It Wasn't the Polls That Missed, It Was the Pundits

It Wasn't the Polls That Missed, It Was the Pundits
AP Photo/John Locher
Story Stream
recent articles

There is a fast-building meme that Donald Trump’s surprising win on Tuesday reflected a failure of the polls. This is wrong. The story of 2016 is not one of poll failure.  It is a story of interpretive failure and a media environment that made it almost taboo to even suggest that Donald Trump had a real chance to win the election.

At RealClearPolitics, we know that the RCP Poll Average is a powerful tool for gauging election trends and projecting outcomes.  But we also recognize that it is not a perfect tool, and that even when aggregating multiple polls, the margin of error within each survey allows for a range of possibilities to exist. This is especially true in an election year that was as unique and volatile as 2016.

That is why, early on, we took what some thought was an unusually broad view of what constituted a tossup state, before returning to our traditional definition of a tossup as a state whose separation is less than five points in the RCP Poll Average. So when a state like Pennsylvania, where Clinton led by 1.9 percentage points in the final RCP Poll Average, goes the other way, we aren’t shocked.

In fact, despite the hue and cry, the national polls were actually a touch better in 2016 than in 2012.   Four years ago, the final RCP National Average gave President Obama a 0.7-point lead; he won by 3.9 points, for an error of 3.2 points. The final RCP Four-Way National Poll Average showed Hillary Clinton winning the popular vote by 3.3 points. She will probably win the popular vote by a point or so, which would equate to an error of around two points.

What about the swing state polls? Again, a close look shows that the 2016 polls performed just as well as they did in 2012 – no better and no worse.  You can see this by calculating the “mean absolute error,” which measures how far away the polls were from the actual result, regardless of in which direction they were wrong. 

So if Hillary Clinton won a state by one percentage point, a poll average that showed the state tied would have the same error as a poll average that showed Clinton winning by two.  To put it simply, it is the size of the miss that’s important.  The mean absolute error is calculated by taking an average of the misses. This gives you a sense of how far away the polls were from the actual results. 

As you can see in the table below, the battleground states of 2016 had the same mean absolute error of 2.7 as the 2012 polls. (And if you look at only the states that were rated as tossups in 2012, the mean absolute error was 2.9 points, even worse than the tossups this year.) 

Though the conventional wisdom had written off Trump’s chances by Election Day, there was evidence showing it was still a very competitive race.  The final RCP No Tossup Map, based off the final RCP Poll Averages in the battleground states, had Clinton leading by the slimmest of margins in the Electoral College 272-266.

At 266 electoral votes, Donald Trump stood just one state short of winning the presidency. And with the final RCP Poll Averages in a handful of battleground states showing Trump trailing by just 0.6 percent in New Hampshire, 1.9 percent in Pennsylvania, 2.9 percent in Colorado, and 3.4 percent in Michigan, it was pretty clear that Donald Trump had a very real path to the White House.

As I noted over the course of 2016, if the polls had been off in 2012 in the Republican's favor as much as they were off in Obama's favor, Romney would have been elected president. It shouldn't have been a huge surprise when that actually happened this year. Sometimes polls are a little more favorable to toward Democrats, while other times they are more favorable toward Republicans. 

What occurred wasn’t a failure of the polls.  As with Brexit, it was a failure of punditry.  Pundits saw Clinton with a 1.9 percent lead in Pennsylvania and assumed she would win.  The correct interpretation was that, if Clinton’s actual vote share were just one point lower and Trump’s just one point higher, Trump would be tied or even a bit ahead. 

Instead, people gravitated toward unreliable approaches such as reading the tea leaves on early voting or putting faith in Big Blue Walls, while ignoring things like the high number of undecided voters.  They selected these data points rather than other possible indicators, such as the significant late break in the generic ballot that could have led them in a different direction. To be blunt, people saw what they wanted to see, and then found the data to support that view.

So don’t blame the polls.  They did what they were supposed to do, and in fact, did their job as well as they did in 2012.  Instead, blame the analysts and pundits, and their stubborn resistance to considering the possibility of a Trump presidency.

Sean Trende is senior elections analyst for RealClearPolitics. He is a co-author of the 2014 Almanac of American Politics and author of The Lost Majority. He can be reached at Follow him on Twitter @SeanTrende.

Show commentsHide Comments