↓Skip to main content

Political Polling Today

Pollsters can employ one or more data collection modalities:

  • Live. Telephone interviews (including cell phones)

  • Landline. Live telephone interviews (not including cell phones)

  • IVR. Interactive voice response, otherwise known as automated polls or “robopolls”

  • Online. Polls conducted through the Internet (that is, a web browser, inclusive of text messaging and application-based polling on mobile phones)

  • Mail. By the United States Postal Service or other physical mail service

Most contemporary polls are conducted online, as described by Nate Cohn (2024). And, unfortunately, many employ opt-in, non-probability sampling. Respondents cannot be assumed to be a random sample of the population of likely voters. Furthermore, few pollsters document their procedures for sampling, survey participant validation, and weighting.

Well-designed statistical samples are essential to the proper operation of political polls. Accuracy can be affected by sampling and non-sampling errors, including poor coverage, non-response, measurement errors associated with respondent and interviewers, and post-survey processing errors (Weisberg 2005, Groves et al. 2009).

Modality differences are an additional concern because telephone and online surveys can yield different results (Miller 2001, Miller and Dickson 2001). To provide better coverage of target populations, some organizations employ a mixture of modalities.

Polls are costly and inefficient. Even with newer data collection modalities, polls can take days or weeks to complete. There is the initial planning of the survey instrument and sampling scheme, followed by data collection, which may include multiple attempts to reach respondents. The analysis may require weighting of survey responses to ensure that reported results are representative of the voting population. Polls are often out-of-date as soon as they are published. There is wide variability in polling results due to differing methodologies, statistical variability, and media effects (Gelman and King 1993).

Despite the many problems with polling, averages across polls conducted just prior to a presidential election can provide accurate election forecasts of nationwide popular vote percentages. Many think the 2016 US presidential election was a failure of opinion polling. Not so. Popular vote forecasts for the 2016 election were highly accurate, with national polling averages anticipating the final election results: The Democratic ticket of Clinton/Kaine received 48.0 percent of the popular vote and the Republican Trump/Pence ticket 45.9 percent (Theiss-Morse et al. 2022). To pick the winner of a US presidential election, however, it is not sufficient to anticipate the nationwide popular vote. The winning ticket is determined by the Electoral College.

FiveThirtyEight, a data journalism organization and website initiated by Nate Silver, assigns letter grades to pollsters based on their use of proper survey technologies and their history of accurate election forecasts. The organization also assesses each pollster in terms of its bias in favor of Democratic or Republican candidates. Ratings are based on hundreds of polls conducted from 2014 to date, with complete data available from the FiveThirtyEight GitHub site. FiveThirtyEight’s election forecasts and analyses, currently available from ABC News, rely heavily on polling data.

Nate Silver’s (2012) recommendations for modelers are documented in The Signal and the Noise: Think probabilistically (preferably as a Bayesian), update forecasts as new data become available, and look for consensus (by combining information from many sources). Silver describes a hybrid election forecasting approach that combines polling data with information about the economy, demographics, and voting patterns of states. FiveThirtyEight’s forecasting models in 2020 employed Bayesian updates of aggregate poll results, followed by statistical simulations for each Electoral College market.

Bayesian statistics gets its name from Bayes Theorem of probability, which provides a formal mechanism for updating probability estimates. When doing election forecasting, we start with prior probabilities based on past voting behavior across geo-demographic groups. We collect polling data for each group, and, using Bayesian methods, we update probability estimates based on these new polling data. The updated estimates are called posterior probabilities. This process is repeated with each new wave of polling data.

With heavy reliance on polling data for the 2020 US presidential election, researchers working for The Economist (Gelman and Heidemanns 2020) employed an approach similar to Nate Silver’s. Their models incorporated past national and statewide voting patterns along with contemporaneous national and statewide polling results (Gelman et al. 2020).

Regarding forecasts of the 2020 presidential election, models based on Bayesian updates of polling probabilities performed poorly. Like many of the polls themselves, models from FiveThirtyEight and The Economist were heavily biased in the direction of the Biden/Harris Democratic ticket. In fact, as predictors of Electoral College results, these models performed worse than pure chance or predictions that assumed the 2020 election would play out the same as the 2016 election, as reported under Methods.

References #

  • Cohn, Nate. 2024, September 27. “The Problem with a Crowd of New Online Polls,” The New York Times."

  • Gelman, Andrew, and Merlin Heidemanns. 2020. “How The Economist Presidential Forecast Works,” The Economist, August, 5. Available Online.

  • Gelman, Andrew, Jessica Hullman, Christopher Wlezien, and George Elliott Morris. 2020. “Information, Incentives, and Goals in Election Forecasts,” Judgment and Decision Making, 15(5): 863–880. Available online.

  • Gelman, Andrew, and Gary King. 1993, October. “Why Are American Presidential Election Campaign Polls So Variable When Votes Are So Predictable?” British Journal of Political Science, 23(4): 409–451.

  • Groves, Robert M., Floyd J. Fowler, Mick P. Couper, James M. Lepkowski, and Eleanor Singer. 2009. Survey Methodology (second edition). New York: Wiley. [ISBN-13: 978-0-470-46546-2] Publisher Link

  • Miller, Thomas W. 2001. “Can We Trust the Data of Online Research?” Marketing Research, Summer: 26–32. Reprinted as “Online Results are a Mixed Bag,” Marketing News, September 24, 2001, 20–25.

  • Miller, Thomas W., and Dickson, Peter R. 2001. “On-Line Market Research,” International Journal of Electronic Commerce, 5(3): 139–167.

  • Silver, Nate. 2012. The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t. New York: Penguin Random House. [ISBN-13: 9780143125082] Publisher Link.

  • Theiss-Morse, Elizabeth A., and Michael W. Wagner. 2022. Political Behavior of the American Electorate (fifteenth edition). Thousand Oaks, CA: Sage. [ISBN-13: 978-1071822173]. Publisher Link.

  • Weisberg, Herbert F. 2005. The Total Survey Error Approach: A Guide to the New Science of Survey Research. Chicago: University of Chicago Press. [ISBN-13: 978-0226891286] Publisher Link.

Go to Political Research and Forecasting.

Go to the home page of The Virtual Tout.

↑