In Defense of the Brown-Shaheen N.H. Poll

In Defense of the Brown-Shaheen N.H. Poll

By Sean Trende - August 26, 2014

The New Hampshire Senate race poll released last Thursday night caused quite a stir. It showed Sen. Jeanne Shaheen's lead over former Sen. Scott Brown collapsing to two points. Many people had begun to conclude that Brown’s campaign was on life support. Suddenly it looked downright sprightly.

Here are three points to consider:

1. The poll might be an outlier. FiveThirtyEight Senior Political Writer Harry Enten has made this case better than anyone else out there. Enten writes: “It’s possible that Brown is slicing into Shaheen’s lead, but there hasn’t been any sign from any other pollster that Shaheen’s edge is down to 2 percentage points. The other three surveys in the race taken after July 1 gave Shaheen an advantage of 5, 8 and 10 points. Before Thursday, no pollster over the past six months has given Shaheen anything less than a 3-point lead.”

2. But there are good reasons to suspect that the movement in the polls reflects actual movement of the electorate. The University of New Hampshire’s Dante Scala points out that the poll specifically shows tightening among Republican respondents.

Remember, when Brown entered the race, there was some resistance to his candidacy within the party. This shows up in the first WMUR/UNH poll from January, which found that Brown’s favorability among Republicans was a tepid 46/23 (+23), and that his lead among the group over Shaheen was 66 percent to 11 percent (+55).

Today, Brown’s favorability among Republicans is 63/15 (+48) and his lead over Shaheen among Republicans is 81 percent to 10 percent (+71), nearly matching the rate at which Democrats prefer Shaheen to Brown (85 percent to 12 percent). This suggests that the movement we see in the topline could also reasonably be viewed as a combination of two factors: First, a natural, inevitable tightening as Republicans “tune in” in advance of the primary and “come home,” and second, an increased Republican population making it through the likely voter screen, as Republican intensity increases in advance of the primary.

In other words, while we might initially conclude that this is “noise” – and should certainly wait for another poll before drawing conclusions of any sort – the fact that there is a logical, expected explanation for the movement should temper that conclusion, and perhaps move us to treat this poll with less skepticism than we otherwise would.

3. WMUR/UNH’s “bounciness” should be a source of comfort, not skepticism. I’ve seen various commenters suggesting that this tendency to fluctuate substantially from poll to poll is a reason to doubt the poll. For my money, nothing could be further from the truth.

Remember, these polls are supposed to move around the “true” value and to do so randomly. Given that we have two points around which they are supposed to move, we should actually expect to see quite a lot of movement in the spread.

To illustrate this, imagine a race that had “actually” been gradually tightening over the course of five polls. At the time of the first poll, imagine that Shaheen had led Brown 50.5 to 40 among a hypothetical electorate. In each successive poll, Shaheen’s actual standing with the electorate drops by 0.5, while Brown’s picks up by a point.

In other words, we’ve imagined a race that is gradually tightening. Now, imagine that nine pollsters descend upon New Hampshire at the exact same times for each iteration of the polls. Here is what we might see (I’ve generated random values around our stipulated “real” values, within error margins of plus or minus four percentage points).

Forget the first chart, for now. All nine of our pollsters have been polling the same race, but you would never guess it by looking at the expressed values. A few – “pollster” 4 and especially “pollster” 9 – actually show races that are moving toward Shaheen. You can see this more clearly in the chart of the leads:

At the bottom, you’ll see the actual values from the WMUR/UNH poll (it only polled four times, so the first value is repeated; i.e., poll 1 and poll 2 are the same). Note that it resembles “pollster” 1’s results. It might well be the case that neither WMUR/UNH’s last poll nor its more recent polls were really outliers (in other words, outside the 95 percent confidence interval). It could just be that the last poll showed results at the high end of the error margin, and this one ended up at the low end.

What are we to take away from this? The first is that a pollster who bounces around is not to be distrusted. If anything, we should look askance at pollsters who find consistently stable races. Because even if a race is stable, a pollster conducting truly random samples should show quite a lot of noise.

Second, and more importantly, look at the top chart, which shows the average of the nine pollsters. It shows something very close to the “true” shape of the race – a gradual tightening.

This, I think, is the key, and it gets back to what Enten hints at. Individual pollsters can easily find results suggesting that a race is opening up when, in fact, it is tightening, and vice-versa. The best way to avoid this is to look at aggregations of polls. Right now, theRCP Average shows a seven-point lead for Shaheen, which is still a comfortable lead. Until we get more data, this is the best take on the race. 

Sean Trende is senior elections analyst for RealClearPolitics. He is a co-author of the 2014 Almanac of American Politics and author of The Lost Majority. He can be reached at Follow him on Twitter @SeanTrende.

A Reply to Rush Limbaugh
Mona Charen · December 9, 2014
Against the Wave: Three Winning House Democrats
David Byler · December 14, 2014
Obama, Congressional Dems Show Cracks in Unity
Jim Kuhnhenn · December 7, 2014
Democratic Louisiana Senator Mary Landrieu Defeated
Melinda Deslatte and Bill Barrow · December 6, 2014

Sean Trende

Author Archive

Follow Real Clear Politics

Latest On Twitter