Election Polling: Why So Much Variation in the Polls? (Pt. 1)
Editor's Note: With the many complaints over election polls in recent weeks, The Christian Post spoke with Scott Keeter, director of survey research at Pew Research Center, to better understand election polling. Part one of this series will look at why election polls vary. Part two will address the issue of whether the polls are oversampling Democrats.
The previous seven polls for the presidential race at RealClearPolitics.com ranges from a seven percentage point advantage for Obama (National Journal) to a one percentage point advantage for Obama (Associated Press). With a little more than a month to go until the election, Keeter does not believe this amount of variation is unusual.
While Pew Research has not done an analysis, "it's our sense that this is pretty normal," Keeter said.
Get Our Latest News for FREE
Keeter also expects the polls to become more similar as Election Day nears. There are many reasons that polls can have different results.
Like many opinion polls, election polls use a sample of the population they are attempting to understand. From that sample, they estimate the opinions of the entire population. In the case of election polls, the population is the electorate, or those who will vote on Election Day, and they assess which candidate they will choose.
To determine the reliability of a poll, there are several factors to consider, such as the question wording, the size of the sample and the randomness of the sample.
Sample Size
All else being equal, the larger the sample the more reliable the poll. Most polling organizations will sample at least 1,000 respondents for a national poll.
The margin of error of the sample will vary depending on the size of the sample. A recent Fox News poll, for instance, had a sample of 1,092 likely voters and a plus or minus three percentage points margin of error.
Since a sample of the population is polled, instead of the entire population, the margin of error indicates the degree of uncertainty about the results of the poll. In the Fox News poll, 48 percent of respondents indicated they would vote for Obama and 43 percent of respondents indicated they would vote for Romney. With the margin of error, that true answers for the population are most likely somewhere between 45 to 51 percent for Obama and 40 to 46 percent for Romney.
By comparison, a recent Bloomberg News poll had only 789 likely voters. For this reason, the margin of error is larger, plus or minus 3.5 percentage points, than the Fox News poll.
Election pollsters generally use a 95 percent confidence interval. This means that if the same poll were conducted 100 times, 95 of those polls would fall within the margin of error and five would fall somewhere outside the margin of error. Another way of saying that, Keeter explained, is that one poll out of 20 will be off more than usual.
"We live by the random sample and we die by the random sample," Keeter said. "Anybody can get a screwy poll at any time. ... Just think about the number of polls you consume ... lurking in those polls will be some really bad samples. Not through any fault of the pollsters, but just the luck of the draw. All pollsters pray that they don't get that bad one at the end when they're making their final forecast."
Randomness
If the sample is to accurately reflect the population, it must be a random sample. This means that everyone in the population has the same chance of getting selected for the sample.
Pollsters generally use random digit dialing. A computer program will randomly choose phone numbers from across the country. This does not make a perfectly random sample for several reasons. Some voters do not have a phone, some voters do not answer their phone, and some voters will refuse to participate in the poll. While understanding that perfection is impossible, pollsters will do what they can to make the poll as random as possible.
Cell phones are an important technological change that have impacted the randomness of polls. Congress passed a law that disallows automated calls, or "robo-calls," to cell phones. For this reason, any polling organization that uses robo-calls, such as Rasmussen Reports, can only call land-line phones. This impacts the randomness of the sample because those who only use cell phones will not have a chance of being included in the sample.
According to Keeter, samples that only use land-lines are likely to include a higher proportion of Republicans than likely voters in the general population. And indeed, Rasmussen Reports polls often show Republicans performing better than other polling organizations show.
Question Wording
Question wording can also impact the results of a poll. For accurate results, election pollsters try to determine which respondents will actually show up to vote on Election Day. Each polling organization has different questions that they use to determine who is a likely voter, Keeter said, and this can explain some of the variation in the polls.
Keeter believes that the measures used to decide who will vote will become more accurate as Election Day nears. Between now and Election Day, potential voters will hear more about the election from the campaigns, the media and non-partisan groups that seek to inform voters and encourage them to register. This information may help them decide whether they will vote and for whom they will vote.
Some Republicans have recently complained about the proportion of Democrats in election poll samples. The polls are not an accurate reflection of what the electorate will look like on Election Day, they argue, because they believe there will be fewer Democrats turning out to vote than what the polls suggest. Part two of this series will take a closer look at this issue.
