January 15, 2008
Purdue professor: New Hampshire primary surprise not that surprising
WEST LAFAYETTE, Ind. - The inability of pollsters to predict the New Hampshire primary highlights a lack of understanding about forecasting elections, a Purdue University professor says.Surveys just before Tuesday's (Jan. 8) New Hampshire Democratic primary showed Barack Obama leading Hillary Clinton by as much as nine percentage points. Clinton won the primary with about 39 percent of the vote, and Obama finished second with about 36 percent.
"The surprise and inconvenient truth is not that the polling surveys do not accurately make predictions sometimes, but that they are ever accurate," says Richard Fienberg, a consumer psychologist and director of the Center for Customer Driven Quality at Purdue. "The surveys used are very standard marketing and consumer behavior techniques that my colleagues and I use all the time."
The most frequently used technique to predict primaries and elections are telephone surveys, which are done as the day of voting draws closer. The numbers in the surveys are tracked and used to gauge a candidate's strength. The accuracy of those surveys depends on three things, Feinberg says: Timing, samples and events.
"Asking people three weeks before an election day and assuming that they will actually vote may not be valid," he says.
Surveys' samples and how representative they are of the population is the second factor.
"If the people who answer the survey are not representative of the people who actually vote, then the results of the survey are probably not going to be accurate," he says.
"Voting surveys of 5,000 possible voters who are not selected via scientific sampling techniques are not representative. A poll can be representative of a large population with only 400 respondents if they are selected scientifically."
Most surveys have a reported margin of error, which means, for example, the results are accurate within plus or minus five percentage points. That means if a survey reports a candidate at 35 percent, they may actually have a vote total of 30 percent to 40 percent. If an opponent polls at 45 percent, he or she may really have 40 percent to 50 percent. Feinberg says this means that a prediction really cannot be made since the intervals overlap.
He also says the error rate for a survey is true 95 times out of 100, which means 5 times out of 100 the survey will be completely wrong no matter how well it is done.
Intervening events also affect a poll's accuracy, Feinberg says.
"The time between a poll taken three weeks before an election can be filled by any number of influential events. It is not the poll that is inaccurate, but that things change," he says.
The focus group is probably the most misused and misunderstood tool employed by political professionals, Feinberg says. In this technique, a moderator leads a group of people in discussion of topics, which can include issues, candidates or political parties.
"A focus group can never be representative of the general population which the professionals believe it represents," Feinberg says. "A focus group represents only the opinions and spoken intentions of the individuals in that group at that moment. No matter how many focus groups are being run, no matter how the participants are selected, no matter how many people are in them, focus groups cannot be used to represent voters in any way.
"The focus group can give professionals an indication of how people think about candidates and issues and the language that they use, but this does not mean that a significant or representative percentage of any population believe that."
Panel evaluations, where people watch an event with a handheld dial and move the dial to represent how much they agree or disagree or like and dislike something, also can sometimes be used inaccurately, Feinberg says. A screen displays the averages of these dial movements. This technique is often used to assess the outcome of debates and speeches.
"There is something very compelling about the visual display of these moving lines. But political professionals may have let the visual power of these moving lines obscure the fact that the averages displayed do not represent voting populations," Feinberg says. "The lines represent how these particular voters in this particular environment feel at the moment. But since these lines/averages can never represent the whole population of voters, the results are simply not representative and cannot be used to approximate some larger voting population."Writer: Greg McClure, (765) 494-9394, gmcclure@purdue.edu
Source: Richard Feinberg, (765) 494-8301, cell (765) 491-5583, xdj1@purdue.edu
Purdue News Service: (765) 494-2096; purduenews@purdue.edu
To the News Service home page