Pairing two items to see what the same people said to both is a powerful move, and it comes with a cost that is easy to overlook. The cost is arithmetic. Every time a group is divided, the pieces get smaller, and smaller pieces support weaker conclusions.

Survey results are usually published with a margin of error, a figure describing how far an estimate might sit from the truth because the survey interviewed a sample rather than everyone. Pew Research Center (2016) explains that this reported margin applies to the whole sample, and that estimates for subgroups within that sample carry larger margins, sometimes much larger, because those estimates rest on fewer cases.

This is the part that gets skipped. A reader sees a margin of error printed at the bottom of a report and applies it to every number in the report, including the ones describing small slices of the sample. Those numbers are less precise than the headline figure, often dramatically so, and nothing on the page usually says which is which.

The problem compounds when subgroups are also harder to reach in the first place. Pew Research Center (2016) notes that some groups, including young people and minorities, are less likely to respond to surveys, so their estimates end up resting on even smaller numbers of actual interviews than their share of the population would suggest.

Bivariate work is a subgroup problem wearing different clothing. Asking what the people who gave one answer also said to a second item means confining the analysis to the people who gave that first answer. If that group is small, the resulting number rests on a small group, and it deserves the caution any small group deserves. A crosstab with many cells divides a sample many times over, and the cells at the far corners can end up describing a handful of people while looking, on the page, exactly like the cells describing hundreds.

Some organizations have begun marking this visually. Pew Research Center (2021) announced it would display error bars in some charts specifically to signal when subgroup estimates have low precision, and set a threshold for doing so at an effective sample size below one hundred people for any subgroup shown. The threshold is a judgment rather than a law of nature, but it gives a rough sense of where researchers themselves start to worry.

Error bars carry a second lesson that runs deeper than sample size. Pew Research Center (2025) is direct that these bars illustrate sampling error only. They describe how far an estimate might fall from the truth because the survey talked to a sample instead of the whole population. They say nothing about the other ways an estimate can go wrong.

That distinction is the one worth carrying out of this passage. Sampling error is the error researchers can calculate. It is not the only error present, and in many surveys it is not the largest. Wording effects, social desirability, the boundaries of the answer list, and who did or did not respond all push results around, and none of them appear in any margin of error. Pew Research Center (2025) points out that some kinds of people are simply less likely to respond, and that this too moves an estimate away from the true value.

A margin of error, then, is a floor on uncertainty rather than a ceiling. It says the result is at least this uncertain. Nothing about it promises that the result is only this uncertain.

None of this argues against pairing items. It argues for saying out loud how many people a paired result rests on. A finding drawn from twenty-eight respondents is a real finding, and it is a smaller one than a finding drawn from four hundred. Reporting the count alongside the result lets a reader weigh it correctly, and withholding the count invites them to weigh it wrong.

This is what makes a stated limitation more than a formality. A writer who names the size of the group behind a number, and names the sources of error a margin cannot capture, has given a reader the tools to disagree intelligently. That is a stronger position than it sounds, because an argument a reader can inspect is an argument a reader can be persuaded by. The alternative, a confident number with nothing said about its edges, asks for trust it has not earned.