2016: What the Polling Actually Said
The common claim that the polls were wrong in 2016 is half true. The national polls came close, the state polls missed, and the reasons were specific enough to fix.
Published 10/06/2026
National polls in the 2016 United States presidential election were among the most accurate since 1936, with an average absolute error of 2.2 percentage points on the popular vote margin. State-level polls in the same election had an average absolute error of 5.1 points, the largest of any presidential election since 2000, and pointed to the wrong winner in Pennsylvania, Michigan and Wisconsin. Both figures come from the AAPOR review of 2016 polling, published in May 2017. Both of those sentences are true, and the distance between them is the whole subject.
The standard account says the polls failed in 2016. The evidence says national polling performed close to its historical best while state polling in a specific set of states did not, and that the largest identifiable cause was a weighting decision many state pollsters had never previously needed to make. A campaign that treats 2016 as proof that polling is broken draws the wrong conclusion from it and will make the same purchasing mistakes the losing side made.
The Federal Election Commission’s official tally records Donald Trump with 304 electoral votes to Hillary Clinton’s 227, and Clinton with 65,853,514 popular votes, or 48.18 percent, to Trump’s 62,984,828, or 46.09 percent. The official figures are in Federal Elections 2016. The American Association for Public Opinion Research convened a committee in the spring of 2016 to evaluate that cycle’s polling and published its report in May 2017. The committee, chaired by Courtney Kennedy of Pew Research Center, reviewed 39 national polls and 423 state polls completed within 13 days of the election, along with 457 primary polls. Its findings are the basis for everything below.
What national polling got right in 2016
National polls collectively indicated that Hillary Clinton would win the popular vote by about 3.2 points, and she won it by 2.1. The average absolute error across the final national polls was 2.2 percentage points and the average signed error was 1.3 points. Those are the committee’s headline accuracy measures. That was better than 2012, when national polls averaged 2.9 points of absolute error, close to 2004 at 2.1, and less than half the 4.4-point average across national polls since 1936.
National poll error against state poll error, 2004 to 2016
Average absolute error on the vote margin, in percentage points, as computed by the AAPOR review committee.
| Election | National polls | State polls |
|---|---|---|
| 2016 | 2.2 | 5.1 |
| 2012 | 2.9 | 3.5 |
| 2008 | 1.8 | 3.4 |
| 2004 | 2.1 | 3.2 |
| 2000 | Not reported | 4.6 |
The committee also tested and rejected the theory that this accuracy was luck. The claim circulating afterward was that national polls had been saved by two large errors canceling, an under-estimation of Trump support in working-class white states offset by an over-estimation where Hispanic populations are large. The review found no support for it, and no evidence that pro-Trump areas were under-represented in national or state samples.
Aggregators differed from one another and from the polls. RealClearPolitics put Clinton’s lead at 3.2 points using a simple average, FiveThirtyEight’s polls-only estimate was 3.6, and the Huffington Post estimate was 4.9. None of the aggregates beat the plain average of the final national polls.
Why state polls missed in 2016
The 423 state polls reviewed had an average absolute error of 5.1 points and an average signed error of 3.0 points in the direction of over-stating support for Clinton. In the four previous presidential elections the state average absolute error ranged from 3.2 to 4.6, so 2016 was genuinely the worst recent year for state polling. The state-level comparison is in section 2.2 of the review. Wisconsin was the largest single miss, with polls there over-stating Clinton’s margin by an average of 6.5 points before she lost the state 47.2 percent to 46.5.
State poll error in 2016, state by state
Average signed error in the 423 state polls completed within 13 days of the 2016 election. Positive means the polls overstated Hillary Clinton’s margin.
Error was not uniform across the map, and it was smaller where the money was. The 207 polls in the 13 battleground states averaged 3.6 points of absolute error, against 6.4 points for the 206 polls elsewhere. Appendix Table A.1 reports both splits. A 3.6-point average error is unremarkable by historical standards. It was still large enough to name the wrong winner in three states whose combined electoral votes decided the presidency, because the margins there were under a point.
Competitive states were polled more accurately
Average absolute and signed error in 2016 state polls, split between the 13 battleground states and the rest.
The state polling, read as polling rather than as prophecy, said the Electoral College was close. The committee quotes Sean Trende’s observation that the final RealClearPolitics battleground averages had Clinton leading 272 to 266 in the Electoral College, with eight states holding 107 electoral votes at average margins of three points or less. The published predictions putting Clinton’s chance of winning between 71 percent and above 99 percent came from forecasting models, a different instrument answering a different question.
How education weighting changed the answers
Education was strongly correlated with presidential vote in the key states of 2016, and people with more formal education are significantly more likely to answer surveys, so a poll that did not weight on education over-represented college graduates and therefore over-stated Clinton. Many state polls did not make that adjustment.
Two cases in the review show the size of the effect. The University of New Hampshire Granite State Poll weighted on age, gender and region but not education, a protocol its director Andrew Smith said had served the poll well for many cycles. It showed Clinton ahead by 11 points in New Hampshire, which she carried by 0.4. Adjusted for education, the same data produced a tied race, which removed essentially all of the error. Michigan State University’s State of the State Poll showed Clinton ahead by 17 points in Michigan; with an education adjustment her lead would have been 10. Trump carried Michigan by 0.2 points.
Two polls reweighted for education
Two live-telephone state polls as published, the same data adjusted for education, and the certified result.
The honest version of this finding comes with a limit. Among five online opt-in pollsters that released estimates in seven shared contests, three weighted on education and two did not, and the committee found no relationship between that choice and accuracy. Education weighting was a real cause of error in live-telephone state polls and it is not a complete explanation of 2016. The Michigan State poll also completed most of its interviews before mid-October, which means it missed the late movement described below.
Late deciders and the shy voter theory
About 13 percent of voters in Wisconsin, Florida and Pennsylvania decided their presidential vote in the final week of the campaign, according to the national exit poll conducted by Edison Research and archived at the Roper Center, the archive of that survey and that group broke for Trump by close to 30 points in Wisconsin and by 17 points in Florida and Pennsylvania. In Michigan roughly 13 percent decided in the final week and went for Trump by about 11 points. A poll completed 12 days out can be a correct measurement of an electorate that no longer exists on election day.
Three days after the election, before any of that evidence existed, Pew Research Center’s methodologists published a list of candidate explanations: nonresponse bias among harder-to-reach voters without college degrees, a shy-Trump effect modeled on the 1982 Bradley race in California, and likely-voter models distorted by unusually low Democratic enthusiasm. Which direction state polls missed, by election Average signed error in state presidential polls. A positive figure means the polls overstated the Democratic margin.
The shy voter theory fared worse under examination. The committee found that some Trump voters who took pre-election polls did not identify as Trump voters until post-election callback studies, and that they outnumbered late-revealing Clinton voters, which is consistent with either late deciding or misreporting. Every other test the committee ran produced no evidence for misreporting, including comparisons between polls with live interviewers and polls with no live interviewer, where a social-desirability effect should have shown up.
One more finding deserves to travel further than it has. Across elections, the direction of polling error is close to random. State polls over-stated the Democratic margin by 3.0 points in 2016 and by 0.8 points in 2004, and over-stated the Republican margin by 2.3 points in 2012, 1.9 points in 2000 and 0.1 points in 2008.
Pew published that list on 11/09/2016What the polling industry changed after 2016
Education weighting became close to universal in state polling. In the final two weeks of the 2020 election, AAPOR’s follow-up task force identified the weighting scheme for 317 of 438 state-level presidential polls, and 292 of those 317, or 92 percent, used education. The 2020 task force reported those counts. The 2020 task force concluded that the polling error of that year was not caused by a failure to weight on education, which is the sentence that tells you the 2016 lesson was absorbed. Pew Research Center described the same shift: many pollsters added an education adjustment after 2016, while noting that not all of them did. Pew set out the change in its 2020 guide to state pre-election polling
The reporting recommendations have traveled less well. The 2016 committee asked for margins of error calculated for the gap between the top two candidates rather than for one candidate’s share, and for an end to decimal places on estimates that are lucky to land within several points. The scale of the problem is easy to state: the 2016 primary polls averaged 636 respondents, implying a margin of error of about 4 points, while their average absolute error on the margin of victory was 9.3 points.
The committee’s structural proposal has gone almost nowhere. It observed that state polls are under-budgeted, conducted haphazardly by whoever happens to field one, and are the polls by which the whole profession gets judged, and it suggested that well-resourced organizations might collectively finance a set of high-quality state polls. Nothing in the intervening years has changed the economics that produced the 2016 state polling landscape.
What a campaign should take from the 2016 polling review
Four habits follow from the record. Match the instrument to the question, because a national estimate cannot answer a state question and the error grows at every step down. Ask for the weighting variables in writing before accepting any numbers, and treat the absence of education as a disqualifying answer wherever educational attainment sorts the vote.
Field late, because the final week moved double digits among voters who had not decided. And read a reported margin of error as a floor on uncertainty: sampling variation is one source of error among several, and in 2016 the gap between the stated margin and the realized error was a factor of two.
Frequently asked questions
Were the 2016 polls wrong?
The 2016 national polls were close to right and the state polls were not. National polls indicated Hillary Clinton would win the popular vote by about 3.2 points and she won it by 2.1, an average absolute error of 2.2 points that AAPOR rated among the most accurate since 1936. State polls averaged 5.1 points of absolute error and pointed to the wrong winner in Pennsylvania, Michigan and Wisconsin, which is where the claim that the polls failed comes from.
How accurate were the 2016 national polls compared with state polls?
National polls in 2016 had an average absolute error of 2.2 percentage points on the vote margin, better than the 2.9 points recorded in 2012 and far better than the 4.4-point average for national polls since 1936. State polls had an average absolute error of 5.1 points, the largest in any presidential election since 2000, when the figure was 4.6.
Why did the 2016 state polls miss Wisconsin, Michigan and Pennsylvania?
AAPOR identified three causes with evidence behind them. About 13 percent of voters in Wisconsin and Pennsylvania decided in the final week and broke heavily for Donald Trump. Many state polls failed to weight for education, which over-represented college graduates and overstated Hillary Clinton. And some Trump voters in pre-election polls did not identify themselves until post-election callbacks. Wisconsin polls overstated Clinton's margin by an average of 6.5 points.
What did pollsters change after 2016?
Education weighting became standard. Of the 438 state-level presidential polls conducted in the final two weeks of the 2020 election, AAPOR could identify the weighting scheme for 317, and 292 of those, or 92 percent, weighted on education. The 2020 task force concluded that the polling error that year was not caused by a failure to weight on education.
More from the blog
Reading an instrument correctly is most of the work. Our Insights work states the weighting variables, the assumed electorate and the realized uncertainty on the front page of every deliverable, and our Modeling work holds those estimates against precinct returns afterward so the next ones are better. If you are about to commit a budget to numbers somebody else produced, tell us which race you are working and we will come back with a written scope.