Uploaded July 2026 | Updated September 2026, 3 weeks ago
A team ran about a thousand people through a market research survey, then had LLM agents replay the same questions, and the agents matched the humans closely while carrying less noise than the humans did themselves. That is exactly the tell: a synthetic respondent smooths over the messiness that makes a real population real. Ishan Anand walks through where it breaks. Nudge one variable in the prompt template and purchase probability swings, because the model infers latent confounders nobody stated, a little like it is playing improv. So how you ask matters as much as which model you pick, and the persona and the study have to be specified richly, from the participant's point of view.
Accuracy can hide the damage. A model that looks right on average can still distort subgroups and flatten the shape of the distribution, collapsing the variation that mattered. The fix here is a noise floor: score humans against other humans first, so you know the best agreement any method could reach, then judge synthetic against human relative to that floor rather than against a perfect match that never existed. Treat a synthetic persona as an economic actor, not ground truth, since even the human study is not ground truth, and validate it against real outcomes before you let it drive a decision.
Speaker info:
- https://x.com/ianand
- linkedin.com/in/ishananand
- ishananand.com
Timestamps:
0:00 - Introduction: synthetic personas for market research
1:43 - Why this talk: separating signal from noise
2:46 - Forecasting people like we forecast the weather
4:04 - A thousand humans vs their agent replicas
5:22 - Purchase probability and prompt sensitivity
6:41 - Invented confounders and specifying the persona
7:45 - Question order and framing effects
8:50 - Predicting stated attitudes vs experts
10:07 - Prompting techniques and subpopulation methods
13:08 - Reconstructing and scoring the full distribution
16:09 - Calibrating personas against real forecasts
17:38 - Setting a noise floor with human vs human
18:40 - Treat personas as economic actors, and what's next
A team ran about a thousand people through a market research survey, then had LLM agents replay the same questions, and the agents matched the humans closely while carrying less noise than the humans did themselves. That is exactly the tell: a synthetic respondent smooths over the messiness that makes a real population real. Ishan Anand walks through where it breaks. Nudge one variable in the prompt template and purchase probability swings, because the model infers latent confounders nobody stated, a little like it is playing improv. So how you ask matters as much as which model you pick, and the persona and the study have to be specified richly, from the participant's point of view.
Accuracy can hide the damage. A model that looks right on average can still distort subgroups and flatten the shape of the distribution, collapsing the variation that mattered. The fix here is a noise floor: score humans against other humans first, so you know the best agreement any method could reach, then judge synthetic against human relative to that floor rather than against a perfect match that never existed. Treat a synthetic persona as an economic actor, not ground truth, since even the human study is not ground truth, and validate it against real outcomes before you let it drive a decision.
Speaker info:
- https://x.com/ianand
- linkedin.com/in/ishananand
- ishananand.com
Timestamps:
0:00 - Introduction: synthetic personas for market research
1:43 - Why this talk: separating signal from noise
2:46 - Forecasting people like we forecast the weather
4:04 - A thousand humans vs their agent replicas
5:22 - Purchase probability and prompt sensitivity
6:41 - Invented confounders and specifying the persona
7:45 - Question order and framing effects
8:50 - Predicting stated attitudes vs experts
10:07 - Prompting techniques and subpopulation methods
13:08 - Reconstructing and scoring the full distribution
16:09 - Calibrating personas against real forecasts
17:38 - Setting a noise floor with human vs human
18:40 - Treat personas as economic actors, and what's next
