About this Blog
About The Author
Email Me

RealClearPolitics HorseRaceBlog

By Jay Cost

« Dueling Strategies | HorseRaceBlog Home Page | The View »

What Moves the Polls?

Mark Mellman had a very excellent column in the Hill yesterday. The topic involved the validity of horse race polling.

This is what he had to say:

Regular readers have heard me rail against the inaccuracy of early polling, which often fails to presage ultimate electoral outcomes.

Yet I have also maintained that presidential elections are predictable based on the fundamentals -- incumbency, war/peace, prosperity and the like.

Recognizing the implicit contradiction, political scientists Gary King and Andrew Gelman asked, in one of academia's best-titled papers, "Why Are American Presidential Election Campaign Polls So Variable When Voters Are So Predictable?"

Their answer focuses on learning during the campaign, and while they may be right, I fear the problem runs deeper. [Snip]

Though it is heresy for a pollster to say it, the evidence also suggests people are only mediocre predictors of their own behavior. Responses to horserace questions a year out may be a special case of faulty prediction.

Hardly an original thought, it can be traced at least to Russian exile and founder of Harvard's sociology department Pitirim Sorokin, who titled his 1936 paper, "Can One Predict His Own Behavior 24 Hours In Advance?" His answer, based on a study of federal employees, was a resounding no: When asked how much time they would devote to various activities during the subsequent eight-hour workday, the average person was off by five hours.

My hat is off to Mellman - who actually relies upon the work of one of the best political scientists in the country, Gary King, to make an argument about politics. A very rare thing indeed! Usually, political scientists get no more "airtime" than the occasional self-evident quote that journalists integrate into their preconceived storylines.

Mellman's topic - the invalidity of early election polling - is one that I have discussed frequently on this blog. I'd like to extend this conversation because I think the problem with media polls gets to what I think are some serious failings in the way journalists and pundits analyze politics.

The best way to discuss this is simply to review the Gelman and King article - which, I should note at the outset, is an attempt to explain a problem with general election presidential polling. Their title indicates the question with which they tussle. Political scientists have developed models that do a very good job of predicting presidential elections based upon "fundamental" variables like incumbency, partisanship, and the state of the economy. All of these are available a long time before ballots are cast. Meanwhile, the polls run all over the place prior to Election Day. How to explain this?

Gelman and King offer what they call the "Enlightened Preference" Model. They assert that:
(1) Voters do not have full information throughout the campaign about the "fundamental variables" that ultimately drive vote choices.
(2) Voters do use all available information to make their decisions.
(3) Voters do not rationally account for uncertainty during the course of the campaign.

This explains how polls can vary so wildly, and yet final results can be so predictable. Voters base their election decisions on basic variables. Thus, their vote choices are quite predictable. But it is only at the end of the campaign that they have fully grasped the values of the variables. Additionally, they do not factor this lack of knowledge into their thought processes. And so, when pollsters dial them up - they rely on the data they have available, but give answers that are less certain than they realize.

Gelman and King write:

[W]ithout sufficient knowledge of their fundamental variables, and when asked to give an opinion anyway, most respondents act as they will in the voting booth on election day: they use the information at their disposal about their fundamental variables, and report a "likely" vote to the pollster. We believe that this report is sincere, but the survey response is still based on a different information set from that which will be available by the time of the election.

Note that this does not mean that the campaigns are useless. The campaign organizations are the agents that provide the information to the voters. If there is a rough organizational "balance" between the campaigns - then the relevant information will be communicated to the voters through them. The media can play a role here - by providing relevant information to the public so that their choices are as informed as possible. Of course, in the last several cycles, most of the media's work has been in covering the horse race.

Now, as I indicated, I think there is a lesson in this for all of us who produce and consume political news and analysis. Of course, bear in mind that this is not a certification of the Gelman/King theory. My specialties in political science are party and campaign organizations. It is not in public opinion or political psychology. I lack the credentials to certify this theory. My intuition is that they are pretty close to the truth - but I am not up on the latest scholarly research to tell you with certainty.

However, I do know enough about these subjects to know that popular accounts of voter psychology are seriously askew because they falsely assume both too much and too little of the average voter. Gelman and King summarize what they take to be the "journalistic model" of the political campaign. I think there is a lot of validity to the characterization:

Under this model, voters base their intended votes partly on fundamental variables, but considerably more on the day-to-day events of the presidential campaign. Voters are assumed to have very short memories, relying for their decisions disproportionately on the most recent campaign events and last piece of information they ran across. Candidates are thought to be able to easily "fool" voters by changing their policy stance during the campaign or causing the opposing candidate to say or do something foolish. [Snip]

Also according to the journalists' model, voters do not take their role in the process very seriously, have very little information of the campaign and the issues, and frequently do not vote on the basis of their own self-interest.

This, Gelman and King argue, is how journalists implicitly explain the day-to-day movement of the polls. I think that this is a valid explanation of the way journalists and pundits approach politics. And, like Gelman and King, I think it is completely wrong-headed.

I would note three salient features about this false view of the voters:

(1) It assumes too much of them. Implicit in this theory is the idea that average voters pay as much attention to politics as political junkies do. We saw a great example of this early this month when pundits and journalists started talking about how the last Democratic debate "changed" everything. Give me a break! 2.5 million people watched that debate. It didn't change a thing! To think that it did requires one to accept a false premise: that average voters follow politics like the junkies.

(2) It assumes too little of them. It assumes that they can be beguiled - that, for instance, they voted for Bush over Dukakis because the latter put a dumb-looking helmet on his head. Again - give me a break! There is an implicitly condescending attitude toward citizens in media theories about vote choice.

(3) It assumes that average voters are quite like journalists. They love the gamemanship of politics. The twists and turns of the daily political soap opera is what they find to be valuable. They care, for instance, that Hillary Clinton's latest wardrobe choice has a lower neckline. And they really care about the political strategy that influenced the decision.

If these three assumptions are false - then most of the media's horse race coverage is a skewed version of what is really going on. I have been arguing this point for quite a while on this blog. What the media assumes about the voters is simply not true - and therefore its analysis is based upon false premises, and thus is wrong-headed. To appreciate this - try the following experiment: take any standard issue pundit discussion that you see on the news, ask yourself whether they are falsely assuming these three things about the voters, and then ask yourself whether - if they stopped making these false assumptions - the analysis would be different.

What, then, drives general election poll numbers? Gelman and King argue that one of the things that is surely driving it is the partial information that has been collected to date. That is, voters are in the process of collecting the data they need to make an informed choice in November. When queried in, say, July - they have only acquired some of that data, which is what they use to make their selection.

I would argue that there is something else going on in these survey responses. Sitting in the background here is John Zaller, whose theory on public opinion merges very nicely with the theory on vote choices that Gelman and King proffer. His 1992 book, The Nature and Origins of Mass Opinions, is required reading on the graduate level - and I'd wager that a lot of undergrads are forced (much to their chagrin!) to slog through what is a very technical read. Zaller argues that one of the reasons public opinion varies is that respondents "receive" informational tidbits here and there from the dialogue of political elites. If those tidbits are essentially compatible with preconceived notions, they are "accepted" and mentally stored by the respondent - to be "sampled" from when the pollster comes calling.

Receive, accept, sample: the "RAS" Model of public opinion. So, in an ironic twist - the analyzers of public opinion actually are the ones creating some of it. It is not that the last Democratic debate had an independent effect on the polls (if there indeed was an effect at all); the effect was caused by the fact that analysts predicted that it would have an effect. Mass opinion is influenced by the elite conversation - so when elites talk about how an event shaped public opinion, they are in fact helping to shape public opinion in that way. But this kind of self-fulfilling prophesying is just a "game" that is quite separate from the way in which vote choices are formed. Public opinion can be tweaked by the elite dialogue - but insofar as this dialogue is not providing information on critical variables, it is not influencing vote choices. It's just shifting the numbers temporarily.

Now, let me reiterate that my specialty is not political psychology or public opinion. I have read and have understood the "great works" in the field, but I am not up-to-date on the current literature. And, as I said, Gelman and King's model is meant for general presidential elections. Nevertheless, I think that this explanation is probably not too far afield from reality. At the very least, I am certain that there is a disconnection between the way average voters really are, and the way journalists and pundits see them - and that this disconnection induces many, many errors in the way the latter examine politics. These errors ultimately make the media dialogue irrelevant for what really matters: who wins the election.

Or, to quote Gelman and King:

"Journalists should realize that they can report the polls all they want, and continue to make incorrect causal inferences about them, but they are not helping to predict or even influence the election. Journalists play a critical role in enabling voters to make decisions based upon the equivalent of explicitly enlightened preferences. Unfortunately, by focusing more on the polls and meaningless campaign events, the media are spending more and more time on "news" that has less and less of an effect."

-Jay Cost