Alan Zendell, August 12, 2020

Any statistician or analyst will tell you that statistics can be a dangerous weapon in the hands of unscrupulous people. It’s almost a cliché that any analyst worth his salt can use the same data to support multiple mutually contradictory conclusions. (Full disclosure: I’ve been guilty of it myself on a number of occasions.)

Does that mean we can’t trust statistics? No. It means you should think before you draw any conclusions, something fewer and fewer Americans do these days. Gullibility about polls and statistics Is one symptom of the intellectual laziness that has made our nation susceptible to deceptive social media and political propaganda based on lies and distortions, both foreign and home-grown.

The first rule to keep in mind is that a trend cannot be inferred from one or two data points. Yesterday, the president offered as evidence that mail-in voting cannot be trusted, a claim that one state had mailed a ballot to a dog. In the early 1960’s when I was an undergraduate at Columbia, the Ivy League schools constantly tried to outprank each other. The hands down winner was a group of Princeton students who created an application for admission for a pet dog, complete with fake transcripts, awards, and letters of recommendation. Princeton accepted the dog. Even a school smart enough to have Albert Einstein on its faculty can be fooled.

Whether or not either story is true, they’re both irrelevant except for amusement. Highly unlikely events are not impossible, and they will occur in small numbers in all situations. The key is “small numbers.” They’re insignificant.

Another good rule is to apply common sense. Some polls are more reputable than others, and some well-intentioned pollsters make mistakes.

When George W. Bush was gearing up his re-election campaign in December of 2003, before cell phone directories were available, polls were conducted solely by interviewing people who had landlines and were willing to interrupt their dinners to answer phone calls. Many people questioned how such a polling universe could be representative of the entire population. I had an opportunity to put the question to Louis Harris, founder of the highly regarded Harris polling organization. He said, “Honestly, I have no idea why it works, but our results have been excellent.” It was an honest response, but not useful to someone looking for a reason to trust the results. That kind of conundrum plagues all polling. Sometimes pollsters are just lucky.

Rule three – before you believe the results of a poll or statistical conclusion, make sure you know the reputation of the reporting organization, who paid for the study, and what their biases might be. Ignore all social media polls. Major news organizations – NBC, CBS, ABC, FOX, CNN – all have very respectable pollsters. It’s the fringe services, PACs, and party organizations asking for money you need to watch out for.

Rule four – pay attention to the margin of error. For most people that goes in one ear and out the other. Projections based on statistics are imprecise by definition. The margin of error defines a range of values, all of which are equally likely to be the right answer. If someone reports that the president’s approval rating is 40% with a margin of error of 3%, the only reasonable inference is that the right number could be 37%, 43%, 41.2%, or any other number between 37% and 43%. That becomes very important when his approval rating appears to change from week to week. If his approval rating was 40% last week with a 3% margin of error, and 42% this week, that isn’t necessary an increase, because last week’s correct number might have been 43%. On the other hand, it could be an even larger increase than it appears.

As the election draws closer and we’re flooded with poll results about candidates, whether parents are willing to send their children to school during the pandemic, how likely we are to have a vaccine this year, or how many hurricanes we’re likely to see before the election, your best defense against being misled is to use your brain. Ignore everything you read on social media and ask yourself if what’s being reported makes sense. Think for yourself and don’t accept anything just because you see it on a screen.