How Statistics May Help Select a Reliable Election Poll

While George Gallup did not invent the notion of polling, he did apply statistics to improve the accuracy of his organization’s predictions. Can statistics help determine the most reliable pollster in today’s competitive market?

Are Conflicting Polls caused by Conflicted Methods?

In Obama Approval Polls Up or Down? Gallup vs. Rasmussen, Decoded Science recently reported that two polls gave different results.

In that article, author Marissa Selner explained several factors that could account for the discrepancy. These included: phrasing the questions differently; including or excluding weakly-held opinions from the results; and whether “likely voters” or “random Americans” were polled.

Political analysts may argue about the best methods for conducting a poll, and place greater confidence in the organization that follows the better discipline.

However, the current question is whether statistics can indicate which pollster has displayed greater accuracy in the past, so as to help decide which poll results to believe.

Define the Problem Carefully

The first problem is to carefully define the problem, and ensure it can be solved with the available tools. A statistical analysis might show that poll results differ week by week; but each week’s report might accurately reflect current opinions.

One assumes that any polling organization will report responses accurately, based on its questions and methodology. After a week, polling results could differ, based on changes in voters’ opinions. It would be difficult to determine that any one opinion poll is accurate or not, since there is no way to independently determine voter sentiment without a trustworthy alternative poll or an actual election.

Instead, this article will focus on the statistics indicating how trustworthy the Gallup or Rasmussen polls might be when predicting voting results just before a presidential election.

Let’s Not Compare Apples to Oranges, or to PCs

The temptation is to blindly throw numbers into a computer, and trust that mathematics will provide the correct answer.

Unfortunately, this can result in comparing apples to oranges… or Apples to personal computers. To solve this problem, the question needs to have clarity and focus.

Limiting the Statistical Problem

How can an opinion poll be shown to be accurate? Let’s consider the summary of results that The Gallup Organization provides across all the American presidential elections they have forecast.

The Gallup page shows the “deviation” between their “Final Survey” and the “Election Result,” which presumably is the percentage of the popular vote. The deviation is the Gallup prediction minus the actual election result.

Click to Read Page Two: The Limited Statistical Analysis for Gallup’s Average