How many voices does a decision panel need?

Bigger is more reliable. That is true, and it is beside the point.

If you have ever commissioned research, your instinct says a panel of 30 is thin. That instinct is right, and it deserves a fair hearing before we take it apart.

Sampling works because there is something to estimate. A real population exists. It holds a true proportion you cannot see directly. You draw people from it at random, and the arithmetic tells you how far your draw is likely to sit from the truth. Draw more people, sit closer. The rule is sound and the math is not in dispute.

Kapari has no population behind it. It builds a set of voices deliberately, picked to clash with each other and to cover the roles and stances a decision has to survive. There is no hidden true number for those voices to approximate. Two different things, so two different rules.

If you have seen a demo running thousands of agents at once, the scale is real and it looks impressive. It buys precision on a quantity. This page is about a different question.

The margin of error, stated fairly, and why it does not apply here

Here is the rule people reach for. At 95% confidence and the worst-case split, the margin on a random sample of n people is 1.96 × sqrt(0.25/n). At n=30 that comes to 17.9 points. At n=2,000 it comes to 2.2 points. So yes: if you are estimating a proportion in a real population, 30 is nowhere near enough, and 2,000 buys you a great deal.

The formula comes with two conditions. First, a real population with a true proportion sitting inside it. Second, draws that are random and independent of one another.

Kapari fails both, and it fails them by design. Its voices are not drawn at random: they are built to clash, which is the opposite of a random draw. And they are not independent: every one of them is generated by the same model, so they share whatever that model leans toward. Run the formula on them and it returns a number. The number means nothing.

The sampling formula answers a question this product does not ask. Kapari does not estimate what share of a population holds a view. It shows a range of plausible reactions and the fault lines running through them.

So we measured it instead of arguing about it

The protocol first, because a result you cannot inspect is a claim.

• InputsThree decisions, each already simulated at 100 voices on our engine.
• MethodSmaller panels drawn from each full panel offline, with zero additional model calls. Nothing new was generated, so nothing could drift between sizes.
• Draws400 draws at each size, using a seeded pseudo-random generator, so the whole run can be replayed exactly.
• Sizes6, 8, 10, 12, 16, 20, 24, 30, 40, 50, 70, and 100.
• ReferenceThe full 100-voice panel. At each size, one question: does the smaller panel read the same?

Averaged across the three decisions. The table comes out of our internal saturation script, checked into our repository so the run can be replayed.

What this does not measure

Two limits, stated here rather than parked in a footnote.

This measures internal stability. It shows that a 30-voice panel reads like the 100-voice panel it came from. It says nothing about whether either one is right about the world. For a simulated panel, that check is structurally out of reach, and no amount of subsampling changes it. External validity is a separate question, and it gets a separate answer on the exam page.

The three test decisions had settled verdicts. None of them sat near a boundary. A decision balanced between two verdicts would not come with that guarantee, and we do not claim it does.

One last disclosure. These three panels ran in French, on the same engine. What the table measures is how the engine behaves as panel size falls. This is not a measurement taken on US panels.

The finding that set the floor

Below 30, the thing that breaks is not precision. Precision degrades gently, from ±4.6 points at 10 voices to ±2.3 at 30, and the verdict never flipped once across the sizes we tested. Coverage breaks hard. The stakeholder families are unequal in size, and the smallest one carries only a few voices. Draw 10 and you will usually leave it out entirely: roughly three-quarters of small draws missed a family.

A small panel does not hand you a blurrier picture. It hands you a sharp picture with a piece cut out of it. That is why 30 is the floor, and coverage is what set it, not margin.

The result we like least

Three decisions went to the same panel. All three came back at nearly the same average position, on the same verdict, with a similar spread. The model has a center of gravity, and it pulls.

More agents do not correct that. They sharpen it. Run the same thing across a much larger crowd and you get a very precise number sitting on the same average answer. That is the worse outcome, because precision reads as accuracy.

One caveat, and we would rather state it than have it read back to us. Those three decisions were three candidate versions of the same annual choice, all pointing the same way. On a varied set of real past decisions, the engine's readings spread out and land on different verdicts, so this is not a case of everything collapsing onto one answer. The pull shows up between decisions that already lean the same way.

What we built from that is the whole point of this page. If the average is where similar decisions look alike, the average is not what you came for. What separates one decision from another is where it splits, who defects, and who was never in the room. Panel diversity surfaces those. Panel volume does not.

Two design choices that came out of it

In the product, panels run from 24 to 100 voices, every profile editable. Thirty is where the reading stopped moving in our test, so thirty is what we recommend for a decision that matters. Go below it and you are trading coverage for speed. That is a fair trade on a quick pass and a poor one on an announcement you only get to make once.

Common questions