Yesterday, the New York Times offered an analysis of their most recent poll -- asserting that Congress' horrible marks are signs of major trouble for Republicans.
There are a whole host of inferential problems with using congressional job approval as an estimate of seat changes in the next election. At the end of its write-up on the job approval number, which is indeed ridiculously low at 25%, the Times backed off the implication, seemingly aware of the trouble. They have good reason to.
Here is a quick list of reasons why congressional job approval is not the first thing I look to in a poll to get a gauge on November. The 4th reason explains why it is not even the last thing.
1. The implication of the poll is that it reflects an anti-incumbency feeling -- and a very great one at that. However, we can rest assured that 90% of incumbents will be reelected in November. Accordingly, the fact that so many people (a) are voicing disapproval of Congress and (b) will eventually vote for their member anyway is a sign that whatever phenomenon it is picking up is not, in itself, necessarily actionable. At the most, we can only conclude that congressional job approval sets up some kind of context for a vote against the incumbent. This begs the question: are the other factors there, and to what extent are they there? That is what really matters.
2. The ecological fallacy (once again) rears its ugly head. This poll is a poll of the nation. But the House vote is not a national contest. It is a series of 435 individual contests. Accordingly, the distribution of sentiment matters a great deal. And a national poll gives us no indication on how these feelings are distributed -- and so does not, in itself, point to anything whatsoever.
3. Relatedly, there is no necessary reason to accept the idea that this will affect Republicans negatively. These numbers are surely being driven, in part, by voters who are disappointed that Congress has not been sufficiently conservative. They would register disapproval, but this disapproval is not actionable on Election Day. At best, the voter can abstain -- but Republican representatives in competitive districts, those who need their conservative base activated, are very effective at making sure the voters distinguish between the institution and them. That's what the "Homestyle" is all about.
4. The first 3 reasons demonstrate the insufficiency of using congressional job approval as a measure of final vote outcomes. I would actually take things a step further and argue that it is entirely unnecessary. I have written about congressional job approval previously -- and my conclusion was that it is probably a spurious cause of congressional election results. The reason for this is not just that it has a horrible track record in predicting final seat swings -- which itself is a sign that, at best, it is spurious. More importantly, it actually does a better job predicting how many seats the party of the President will gain/lose, regardless of whether the party of the President is actually in control. of Congress. This makes no sense, and supports the idea of spuriousness. My theory is that presidential job approval drives both final elections and congressional job approval.
Let me be clear -- I do not doubt that there is some feeling out there in the nation that this statistic is tapping into. And this feeling is ultimately what will cause Republicans to lose a good number of seats. My objection is that the Times, like many outlets, paints congressional job approval as a cause of what will happen in November. When they point to 1994, that is what they are doing. But the data, as well as the logic, indicates that this is likely a spurious cause. At best, all we can say is that congressional job approval, sometimes but not always, somewhat, but not very much, co-moves with final November results.
Generally, this has been something I have noticed in this season's coverage of the Election. Pundits tend to make a laundry list of causes of GOP trouble. I do not doubt that the GOP is in trouble. What I do doubt is the laundry list. I think that many of the things they point to are in fact all effects of something that is going to cause the result in November. In other words, there is some singular cause that (a) induces congressional job approval to tank, (b) induces right track/wrong track to tank, (c) induces "do you want to reelect your incumbent?" to tank, and (d) induces final election results. (a) through (c) do not cause (d). But, if you are only taking a brief look at the data (for instance...oh I don't know...maybe just looking back to one previous incidence, that happens to be the outlier), and are not careful to distinguish between correlation and causation, you'll conclude that (a) through (c) do indeed cause (d).
This is a symptom of a broader problem I have noticed. Journalists and their editors recognize that they are not able to put together externally valid samples to gauge public sentiment. They know that they need pollsters. However, for some reason, they believe that they can interpret the results. In actuality, interpreting polling is as difficult as creating polling. It requires a thorough grounding in the logic of causal inference -- which is something that journalists obviously lack. They seem to completely lack it, too -- if they knew anything about this logic, they woukd know how difficult and tricky it can be, and they would either outsource the task of interpretation, study up, or express some hestitation. But they don't. They interpret polls just as college freshmen interpret Aristotle -- with a gusto and authority that only comes with complete ignorance of the subject.
A professor friend of mine once told me that the problem with political journalism is that journalists are doing political science with an M.A. in English. I think there is a lot of truth to that -- and the misuse of polling data in journalist circles is one thing I would point to.
