About this Blog
About The Author
Email Me

RealClearPolitics HorseRaceBlog

By Jay Cost

« A Note on the Polls | HorseRaceBlog Home Page | Return To Pennsylvania »

More on the Polls

On Friday, I noted that the differences among the national polls is large enough to suspect that something other than random variation is causing the disagreements.

I'd like to expand on this point by examining today's Pew poll, which pegs McCain's share of the vote at 38%, with a margin of error of 3.5%. That means that Pew predicts with 95% confidence that McCain's true share of the vote is somewhere between 34.5% and 41.5%.

While we don't know McCain's true share of the vote, we do have an estimate of it - the RCP average. Right now, it puts McCain at 43.6%. This figure is far outside Pew's 95% confidence range. So, if we use the RCP average as our estimate of McCain's true share of the vote, we would conclude that Pew is an outlier.

The question then becomes whether it is outlying due to random variation, or some non-random cause. We can never know for sure, but we can make a few points.

First, the level of disagreement between the Pew poll and the RCP average is great. Indeed, if we assume that the Pew poll has an accurate read on the electorate, the chance that McCain's true share of the vote is 43.6% is less than 0.5%. Given the number of polls that cycle in and out of the RCP average, we should expect at least a few outliers. However, it would be pretty rare to find one that disagrees with the RCP average by such a large amount.

Second, the previous Pew poll, which had McCain at 39% of the vote, was also an outlier when compared against the RCP average. So, Pew has twice in a row pegged McCain's number at significantly less than the RCP average. It is very unlikely to see this kind of result if random variation is the only cause.

Does this mean that Pew is wrong? No. We could only conclude that Pew is wrong if we know McCain's true share of the vote right now. We don't know that. Instead, what we can conclude is that the difference between Pew and the RCP average is likely produced by something other than random variation.

Pew is not the only poll behaving in this fashion. Today, the Gallup traditional model pegs McCain's number significantly higher than the RCP average. It has done this several times over the last three weeks - and every day since it began it has shown McCain doing better than the RCP average. It is unlikely that random variation would produce these effects. Today's Rasmussen poll shows McCain significantly higher than the RCP average, and it has consistently been higher than the RCP average for the last three weeks. IBD/TIPP frequently pegs Obama's number significantly lower than the RCP average, and it has shown him lower than the RCP average every day since it began. The GWU/Battleground poll has shown McCain consistently higher than the RCP average for 10 of the last 10 release dates, frequently at significant levels.

None of this is consistent with what we would expect from random statistical variation. These considerations reinforce the point I made on Friday. In all likelihood, something else is going on here. The pollsters have different "visions" of what the electorate is, and these visions are inducing such divergent results.

This is why I would urge caution when interpreting all this polling data. We're talking about disagreements among good pollsters. I take all of these firms seriously whenever they produce new numbers. They are disagreeing with one another in ways that can't be chalked up to statistical "noise." That gives me great pause.

-Jay Cost