---

April 06, 2006

Will Congress' Poor Marks Doom the GOP?

By Jay Cost

There has been much talk recently about Congress' low job approval numbers. Last month, Gallup found that just 27% of the public approves of the job Congress is doing. Many pundits think that this is a sign that the Republicans are in trouble. Indicative of this is Charlie Cook's most recent column. He wrote:

A report released last week by the Gallup Poll's Jeffrey Jones said that in elections from 1974 to 2002, when the approval rating for Congress as an institution was 40 percent or higher (1986, 1998, and 2002), the party in control of the House lost an average of only five seats. But when the approval rating of Congress was lower than 40 percent (1974, 1978, 1982, 1990, and 1994), the average loss was 29 seats.

In actuality, job approval seems to coincide with seat changes from one party to another - not necessarily from the in-party to the out-party. In 1974, the Democratic-controlled Congress had a job approval in the 30s. However, the Democrats gained more than 40 seats that year. Nevertheless, the argument that Jones and Cook make is that congressional job approval gives us a strong indication of what will happen in congressional elections. At first blush, this evidence seems compelling.

Upon reflection, however, I find it not to be. I have written in the past that, by itself, congressional job approval is not a reliable indicator of what will happen in midterm elections. This is an appropriate time to amplify this argument. There are two reasons for my conviction. The first is statistical. Jones and Cook endeavor to show that there is a strong relationship, but they do not use the right statistic. If we want to know congressional job approval's predictive power, citing an average or two will not do. We need the coefficient of determination. This is a percentage value that tells us the extent to which a change in job approval implies a change in seats. The coefficient of determination is the most direct, simple and precise way to understand the relationship job approval has with seat changes.

There are different ways to compute this value. We could ask how well job approval does predicting the seats gained/loss for the party of the President. We could ask how well it does for the party controlling Congress. We could examine it for all congressional elections since 1974 (the first year Gallup asked the question) or just the midterms. The following table summarizes the predictive power of congressional job approval under different circumstances.

Type of Election	Type of Party	Predictive Power
All Elections	Party Controlling Congress	6%
All Elections	Party of President	23%
Off-Year Elections Only	Party Controlling Congress	8%
Off-Year Elections Only	Party of President	46%

This is nothing to get excited about. The best academic theories have a coefficient of determination between 70% and 90% for all elections since 1946. None of them use congressional job approval as a measure.

Why, then, does job approval seem so compelling? It gets to the statistic Jones and Cook have used. Statistics are like photographs. Each offers a picture of the world. Sometimes, the picture gives a false impression. That is the case here. They are using a particular statistic - the average/mean - to depict reality. However, when one observation differs drastically from another, the mean does not explain much. Take an example. Suppose you have a set of numbers whose mean is 250, but whose values are 994, 1, 2 and 3. Is the mean telling you anything about the real world? Not really. There is too much variability.

The best measure of variability is standard deviation. It is the average amount that each observation differs from the mean. Consider again the mean value Jones and Cook find, now in the context of the standard deviation. The mean number of lost seats below 40% is 29, but the standard deviation is 21. In other words, each observation tends to vary from 29 by an average of 21 seats. Thus, when job approval drops below 40%, we should expect the final result to be anywhere between 8 and 50 seats lost. Given that the Democrats need 15 seats to take the House, we must conclude that, by itself, the history of congressional job approval tells us nothing about who will control the 110th Congress. There is simply too much variation.

There is another problem with their evidence. What justifies the criterion of 40%? Neither Jones nor Cook says. It seems that they use it because, when you divide the data at 40%, the final result looks good. This is the wrong way to go about making arguments. If you are free to group data any way you want, without any justification for the dividing line, you can make anything look impressive. This is why your dividing line has to follow from a preexisting, logically sound theory. What Jones and Cook have done is fit a theory to the data - and you can always look good doing that.

It seems to me that, if we are interested in a theoretically sound dividing line, 50% is best. We should expect the public to start tossing members out rather than keeping them in at 50%, not 40%. If 50% is the line, the relationship between job approval and seat changes looks much less impressive. We would only have only one election (2002) where we would expect the incumbent party to be safe. In reality, there have been three such elections - which means that incumbents have been safe when they should not have been. Another logical point of division might be the median value, which is 32%. In that situation, there have been instances above and below 32% where many seats switched hands, and instances above and below 32% where few seats switched hands. Regardless of which cutoff point you choose, you have to have a reason for the choice.

This is an example of the second type of problem I have. People who argue that congressional job approval is a good measure of what will happen are usually making theoretically unsound arguments. This is one such example. 40%, while it produces good results, is artificial and ad hoc.

As our table shows, correcting the 40% threshold does leave some correspondence between congressional job approval and seat changes. It is a very slight correspondence, but it is still there. Why? Is it because congressional job approval does indeed factor into congressional elections, in some small but real way? Maybe the results are just a coincidence. That is very possible. Also possible is that the relationship is spurious - i.e. there is some third variable that tends to affect job approval and seat changes, but to us it looks like job approval directly affects seat changes. If the results we have are coincidental or spurious, then we should not rely on job approval as a measure. But if they are real, then we must take it seriously.

How can we tell? One way is the following. If the relationship between the two is real and direct, its relationship with seat changes would cohere with what we already know, or can reasonably assume, about congressional elections. In other words, to accept the importance of job approval means that we cannot reject something that we already know to be true.

To appreciate what I mean, consider this example. Suppose I flip a coin seven times in seven days. Every time the temperature is above 65 degrees, the coin lands heads. Every time the temperature is below 65 degrees, the coin lands tails. In this situation, could I infer that the temperature is what is affecting the way the coin lands? Why not? It is not because of any statistical error. In this case, the temperature and heads/tails are perfectly correlated. I cannot make this inference because we already know that temperature and coin tosses have no relationship. Inferring from my experiment that they do is therefore inconsistent with what we already know.

In other words, a good statistic is not just enough to make a good argument. You also have to have a consistent theory behind it: accepting a conclusion cannot force you into rejecting something you know to be true. In the case of congressional job approval, many theoretical inconsistencies emerge.

Perhaps the biggest is how badly job approval does in predicting changes in the Senate. As I mentioned above, it does a lousy job predicting House switches - but it does a really lousy job with the Senate. This is counter-intuitive. The Senate as an institution has a higher turnover rate. It also has elections where voters are thinking more nationally; they are more likely to view their incumbent senator as being part of the bigger system. Senators, all in all, have more trouble avoiding national trends. If congressional job approval made a real difference in elections, we should expect its difference to be felt most strongly in the Senate - not least strongly.

Other theoretical problems abound. (1) Why does it do a better job predicting how the party of the President will do? Should we not expect it to do a better job predicting how the party controlling Congress fares? (2) Given that it does better predicting the fortunes of the President's party, why does it predict best when the President is not on the ballot? If the President matters so much, should he not matter more when he is on the ballot? (3) Why is it that it works so much better for off-years than on-years? The data we have about congressional elections indicates that voters tend to think the same way about their vote choice in on- and off-years. (4) How can this measure be a valid predictor when we know that voters do not think about Congress as an institution when they vote? Is there some intervening variable that connects congressional job approval to vote choice? If there is, should we not look at that to make a prediction?

Recent work by political scientists has indicated that the predictive value of congressional job approval is slightly better when taken in the context of the nation's economy. However, it amounts to only a very small improvement on the percentages reported above and still quite inferior to theories that do not use job approval. What is more, these theoretical problems remain unresolved in this work. Nevertheless, the 2006 prediction for this model, with ongressional job approval at 39% (which Time found last week and which Jones and Cook would identify as reason for Republicans to worry), is for the Republicans to retain 99% of its caucus! Far and away, this estimate is the most conservative of any academic theory I have seen.

I am not inclined toward this recent work; I am certainly not inclined toward such a conservative prediction; and I know that congressional job approval alone has a very weak relationship with seat changes. My sense about any predictive power job approval has, whether in the context of the economy or some other variable, is that it is a combination of coincidence and spuriousness. I think that the key factor is actually presidential job approval. In other words, the President independently affects both seat changes and congressional job approval, which have no real relationship with one another. It seems like congressional job approval affects seat changes, but both are just affected by the standing of the President. How else do you explain the fact that congressional job approval does a better job predicting for the party of the President, not the party controlling Congress? What is more, the coefficient of determination between presidential job approval and congressional job approval is a very impressive 67% (in other words, changes in presidential job approval explains 67% of the change in congressional job approval). My theory is that when voters say they disapprove of Congress, they are basing most of that opinion on their disapproval of the President. Thus, to add congressional job approval to presidential job approval in your evaluation of what will happen in November is to actually count the same factor twice.

This returns to a point that I have made several times. There are, seven months from the election, five indicators that help us get a handle on the result. One of them cuts against the GOP, and that is George W. Bush. He is going to damage their majority. Four of them cut against the Democrats, and therefore mitigate the damage Bush will do. These are (1) disappointing Democratic recruitment, (2) the small number of Republican retirements, (3) the expectation that real income per capita will grow at a good rate, (4) the small difference between the Republican majority in the 109th and their historical average. Congressional job approval does not seem to factor into it.

---

Sponsored Links

About Us | Contact | Advertise | Privacy Policy         © RealClearPolitics 2006