Are AI synthetic audiences accurate?
The short answer: they are good for exploring a range of plausible reactions, and they are not a stand-in for polling real people. The peer-reviewed work is clear about that, and most vendors bury it under a self-declared accuracy score. Here is what the research found, how to vet a vendor, and where Kapari stands.
A synthetic audience is a panel of AI-simulated profiles that react to a question or a decision. How reliable it is depends entirely on what you ask it to do. As a numeric stand-in for a poll, it breaks down. Researchers took a synthetic panel whose averages looked right and found that 48% of the statistical relationships did not match reality, including 32% that ran the opposite way (Bisbee et al., Political Analysis 2024). Measured against real polling, the error runs from 4 points to more than 23, depending on the topic (Verasight reports, 2025-2026). Models also pull minority views back toward an average opinion (Santurkar, ICML 2023; Wang, Nature Machine Intelligence 2025). Point the same panel at a different job and it earns its keep. To explore the range of plausible reactions to a decision, to surface the objections, the fault lines and the blind spots before you announce, a synthetic audience is useful and honest, as long as it claims neither to measure, nor to predict, nor to represent. So the question is not whether a tool is accurate to some percentage. The question is whether it explores honestly or sells you a number.
What the research found
The peer-reviewed work does not say these panels are worthless. It says something harder to live with: the averages can look right while everything underneath them is wrong.
That is why Pew Research Center, the gold standard in survey research, rejects silicon sampling and still fields its studies with real people (2026). The research rules out one thing: using a synthetic panel in place of a poll. It does not rule out using one to explore a range of reactions honestly.
How to vet a tool
Money is pouring into this market. Simile raised $100 million, Aaru more than $50 million, and the race is on for a bigger accuracy number. Most vendors claim 80 to 95%, self-declared, never checked against a public benchmark. Four questions separate proof from marketing.
A public benchmark you can replay
Does the vendor publish results on real cases, with the method and the evidence, so you can run it again yourself? Almost nobody does. That is the hole in this market.
Cases held out of the tuning
Were the test cases kept away from the tuning? A score measured on the cases used to build the tool always flatters it. Only held-out cases mean anything.
A score published even when it is bad
Does the vendor show the misses as well as the wins? A vendor who publishes only the good cases is hiding the variance you actually need to see.
No crystal ball
Is the vendor selling you the future, or the range of reactions? In 2025, the FTC sanctioned an AI vendor for an accuracy claim it could not back up. An over-promise is a risk you inherit, not a guarantee you buy.
Where Kapari stands: explore honestly, then prove it
Kapari is a test bench for decisions. You put a decision in front of a simulated panel, each voice with its own role and its own stake, and you get back the range of reactions, the fault lines and the ways to adjust. The math is deterministic. The AI writes the voices, it never does the arithmetic. Kapari shows no single opinion number and makes no claim to predict. It also answers the four questions above. On 16 famous US decisions submitted without their ending, Kapari recovers on average 74% of the objections that were actually documented at the time, across 3 full passes, with every case published alongside its evidence, the wins and the misses, in a versioned and timestamped record. That is the narrow ground between the vendor who sells a fake number and the vendor who shows no proof at all.
The full exam, case by case: the US exam sheets.
Common questions
Can a synthetic audience replace a poll?
No. On a synthetic panel whose averages looked right, 48% of the statistical relationships did not match reality, including 32% that ran the opposite way (Bisbee et al., 2024). A simulated panel is for exploring a range of plausible reactions. It is not for measuring opinion in place of real people.
How accurate is an AI panel?
It varies, and the claims run ahead of the evidence. The error against real polling runs from 4 points on a heavily covered topic to more than 23 on an everyday one (Verasight, 2025-2026). Most tools claim 80 to 95%, self-declared, never checked against a public benchmark.
How do I tell if a tool is reliable?
Ask for four things: a public benchmark you can replay, cases held out of the tuning, a score published even when it is bad, and no crystal ball. In 2025, the FTC sanctioned an AI vendor for an accuracy claim it could not back up.
Is Kapari accurate?
Kapari publishes its score, including the bad runs. On 16 held-out US decisions it recovers on average 74% of the objections actually documented, with every case published alongside its evidence. It shows no single opinion number and makes no claim to predict.
Reliability is proven, not declared.
Put your decision on the bench. A panel reacts, you read the range before you announce, and nobody hands you a fake percentage.
Try the demo →