AI 'Digital Twins' Seem to Distort the People They're Meant to Mimic, Study Finds

Some social scientists have hoped to replace costly, easily tired human research subjects with AI-generated "digital twins" that mimic a specific person's likely responses. A study published September 2 in Science Advances finds that these digital twins instead seem to distort the views of the people they represent, creating what the researchers call a "funhouse mirror" effect.

To build the twins, researchers led by Olivier Toubia, a computational social scientist at Columbia Business School, drew on a dataset of more than 2,000 people across the United States who had answered over 500 questions on traits including age, income, education, political preferences and personality, and completed tests of thought patterns and biases. For the new study, the team fed each person's answers into a large language model (LLM) and prompted it to respond as if it were that person, then compared the twins' answers with the real people's across 19 social science experiments — including reactions to political donors and to algorithmic hiring.

The digital twins performed better than random chance but were wrong about a quarter of the time on average — roughly matching chatbots given only demographic information, though the twins better captured real variation between individuals. The researchers found the twins' answers were more homogeneous than real people's, skewed toward demographic stereotypes, grew more accurate for wealthier, more educated participants, and showed biases such as greater trust in others and less concern about technology risks — appearing more rational overall than the humans they modeled. Penn State AI researcher and economist Hadi Hosseini said he has seen a similar shift toward artificial rationality in his own research on AI agents making health care decisions.

Toubia said the static, one-time survey format used to build the twins may partly explain their limits, and that more complex training methods could help. He said twins could still be useful for pretesting experiments or producing detailed answers where tired human subjects might give only a sentence, but urged researchers to stay realistic about what synthetic survey data can predict about real human behavior.