Enterprise Software
A six-step framework helped an enterprise software team evaluate synthetic audiences against human B2B research.
An enterprise software product team wanted a rigorous method for using synthetic audiences in B2B research. The challenge was to establish where AI-generated responses reflected real customers closely enough to support product decisions.
The Experiment
ArcticBlue developed and tested a six-step framework for creating and evaluating synthetic avatars. We trained and refined models using small batches of primary research, then compared synthetic responses with human feedback.
A concept comparison covered five concepts, 16 human participants, and 56 synthetic avatars. A separate workflow benchmark compared detailed individual profiles with blended audience profiles.
Results
Synthetic and human audiences agreed on the majority relevance judgment for all five concepts. Response proportions varied, so agreement on the overall conclusion did not imply identical response distributions.
In the separate workflow benchmark, individual profiles reduced error by 49% compared with blended profiles: 11.14% versus 21.71%.
Why it mattered
The team received a research report and practical recommendations for profile design, validation, and use-case selection. It gained a clearer basis for deciding which questions synthetic audiences could help answer and which required further human research.
