AAPOR Says Trump-Clinton Polls Mostly Right
Here is an update on what went wrong with the Clinton vs. Trump polling debacle last fall, according to AAPOR’s official Evaluation of 2016 Election Polls in the U.S. that was released several weeks ago.
The conclusions are based on an analysis and assessment of publicly available polling data plus additional data supplied by top polling organizations committed to AAPOR’s efforts to build ongoing transparency and trust into the polling process.
Overall, the findings are consistent with Versta Research’s conclusions published in January (Survey Says…Trump Won? Research Lessons from the Polling Mess) and they add additional insight around some of the weaknesses and dangers of state-level polling.
In brief, key AAPOR findings are:
- National polls were mostly correct and accurate. Overall, they predicted the popular vote to within less than one percentage point (that’s pretty darn good).
- State polls were not so good, leading to inaccurate forecasts of electoral college votes (which is ultimately how the U.S. President is chosen).
- In states where Trump won narrowly, some voters shifted preference to Trump during the last week before the election after most polling was done.
- Voters with less education were under-represented in polls, and some pollsters, especially at the state level, failed to correct for it with weighting.
- There is no partisan favoritism in U.S. polling. “Whether the polls tend to miss in the Republican direction or the Democratic direction is essentially random,” the AAPOR analysis concluded.
So why were so many people shocked? The AAPOR committee takes a good swipe at the poll aggregators and forecasters who seemed to suggest that polls can predict the future:
Any one poll is a static measure. The benefit of public polls is to provide a narrative about the status of issues and elections for the public, the press, and the polity. In contrast, many polling aggregators are prognosticators. Among these forecasters are a few scholars of polling, but many are masters of other fields. They come together as data scientists or data journalists charged with accumulating numbers from a multitude of sources to identify patterns and predict future events. In politics, that means innovating experimental models to predict elections.
As researchers, there are many lessons we can learn from all this. Take a look at our January newsletter for our list of five lessons for market research worth contemplating.