Study: the likely voter model in that poll you’re reading about is probably garbage

Pew is out with a new study today comparing the relative merits of the various ways in which pollsters try to determine which of their respondents will actually cast ballots on Election Day.

The results don’t look good for most commercial pollsters. As in, the vast majority of the polls the media reports on, and are included in the polling averages that we use to gauge how the horse race is shaking out, rely on likely voter screens that fail to accurately portray the actual likely electorate. As Pew writes:

Vote via Shutterstock

Vote via Shutterstock

Public pollsters, such as Pew Research Center and the major news organizations that conduct election polls, typically have used random digit dial (RDD) samples to reach a random sampling of all Americans, then narrowing down to prospective voters by asking people a series of questions that gauge interest in the election, past voting behavior and intention to vote. Campaign pollsters tend to use samples from databases of registered voters and incorporate past vote history from those databases into their forecasting models, ensuring that they know whether the respondent has voted in the past. The sample employed in this study was originally obtained from an RDD survey and later matched to a voter file so that both the survey questions and the past vote history could be used in the analysis.

All of the methods examined here result in more-accurate forecasts than using either all those respondents who say they are registered to vote, or else all those who say they intend to vote, both of which include far too many people who ultimately will not cast a ballot.

The upshot here serves as confirmation for a point I’ve made before: non-voters are really bad at self-reporting their voter status, vote history and voting intentions. Voting is a democratic norm, and people who rarely, if ever, participate in the electoral process still feel social pressure to tell a pollster that they are a good democratic citizen. This being the case, asking a random sample of US adults if they are registered to vote, if they voted in x out of the last y elections and if they are planning to vote this coming November is bound to overestimate the eventual size of the electorate. But that’s exactly how most commercial pollsters operate.

They operate this way because keeping up to date voter files, and using those voter files’ voter histories to model the electorate, is time consuming and expensive. However, that’s time and resources that national party organizations and political campaigns are willing to spend, for a few reasons. For starters, they need to maintain up-to-date voter files for reasons that go beyond polling, so the data from which to pull is already available. But they also have a different set of incentives: Commercial pollsters have to make money, and they do that by churning out a large number of polls that (hopefully) turn out to be somewhat close to the end result. Campaigns, on the other hand, are infinitely more concerned with making their polls accurate than they are with making them profitable.

That said, the study also finds that any attempt to screen for likely voters is better than none at all — something that prior research has shown to be more true the closer to Election Day the poll is conducted. However, as commercial pollsters continue to produce popping headlines about shocking twists and turns in the presidential race, it’s always worth checking their sample, seeing if it screens for likely voters and checking whether that screen is based on self-reports or verified voter registration and history.

As one might have guessed, details like that matter.

Jon Green graduated from Kenyon College with a B.A. in Political Science and high honors in Political Cognition. He worked as a field organizer for Congressman Tom Perriello in 2010 and a Regional Field Director for President Obama's re-election campaign in 2012. Jon writes on a number of topics, but pays especially close attention to elections, religion and political cognition. Follow him on Twitter at @_Jon_Green, and on Google+. .

Share This Post

© 2018 AMERICAblog Media, LLC. All rights reserved. · Entries RSS