This article came to me (just as it came to billions of others) via Drudge:

Even Fred Thompson doesn't think he will become president. Chatting off-air to a television reporter, a stunningly candid off-the-cuff quip from the Hollywood actor cemented the impression that his heart is not in the 2008 race.

Trying to encourage his studio to hurry up so an interview could start, Carl Cameron of Fox News said into his microphone: "The next president of the United States has a schedule to keep." Standing beside him, a deadpan Mr Thompson interjected: "And so do I."

As some Thompson aides looked bemused and others cringed, a taken-aback Mr Cameron, Fox's chief political correspondent, exclaimed: "You can't do that kind of stuff!"

I've talked on this blog about Type I error. This is the error of the false positive, or the mistaken belief that something is there when, in fact, it is not there. I've argued that political journalists, political pundits, and others of that ilk have far too great a tendency to commit this error, to turn "nothing into something." I think a lot of this has to do with the pressures of being a journalist, of finding things to talk about every day. I think there is a natural human tendency to make something out of nothing. It takes some discipline to stop yourself from doing that - and this is a discipline that journalists probably do not have an incentive to develop.

This article is probably the single greatest example of Type I error I have seen all year. To infer that headline from this quotation is patently ridiculous.

In the social sciences - at least in the quantitative social sciences - there are all kinds of protocols to prevent the occurrence of Type I error. If you want to get something published in a scholarly journal or an academic press, you really have to prove to the reviewers that you are on to something, that there is a only small chance that what you think you have found is actually nothing.

For a long time, I took this for granted. I did not put much thought into why the rules are set up the way they are. But this election cycle has been helping me understand why academics are so careful about Type I error.

The above example is a silly one. Indeed - only a Brit paper could write something like that, and I don't think even they believed that Thompson was being serious. But there have been serious examples of Type I error. The one that comes to my mind right away is this notion that Rudy Giuliani was giving up on New Hampshire. It never fit the facts well at all. Nevertheless, the press accepted it as true. But - as theories that have trouble fitting the facts so often do - this theory turned out to be false. Rudy has basically planted himself in New Hampshire this month. The journalists and pundits who argued that he was planning to quit New Hampshire had obviously been committing Type I error. They believed a false theory was true. And what was their response to this mistake? They did not go back and conclude, "Hmmm...I guess we were wrong about Rudy giving up on NH." No. Instead, their argument was, "Rudy has changed his strategy!"

This is what you have to do with a false theory when it is faced with a falsifying instance - and you do not want to reject it outright. You have to attach ad hoc addenda to it. You have to alter it so that it fits the observed facts. But the alterations do not make it true; they just make it fit the facts. That is the only reason to accept the addenda. They are not intuitive or sensible. They do not emanate logically from the original theory. They're just attachments to keep it alive. The idea that Rudy changed his strategy is obviously ad hoc. The only reason to accept it is to keep the original theory going - it makes no sense whatsoever (as it requires him to have had three strategies for dealing with New Hampshire in the month of September alone!). And so, look what has happened: they have compounded the error. Before, they were "merely" wrong about Rudy giving up on New Hampshire. Now, they are wrong about Rudy giving up on New Hampshire and Rudy changing his strategy.

This is what happens when you do not have good protocols in place to reject false theories. You accept them. But, as they eventually fail to fit the facts, you are forced to accept false addenda to the original theory. That addenda will eventually fail you - so you have to accept another. Thus, you become more and more wrong. The tendency to commit Type I error is thus like the tendency to make turns down random, unknown roads after you are already lost.

Of course, there is a trade-off her. if you put protocols in place to keep yourself from accepting false theories, sooner or later you are going to reject a true theory. I would argue, this is much less damaging in the long run. The nice thing about true theories is that they are true, which means they fit the facts yesterday, today, and tomorrow. If you have not already accepted a false theory today, you'll be looking at things tomorrow - and you'll have another chance to be persuaded by the true theory. The only damage is that you went a little while not accepting the true theory. When you accept as true a theory that is actually false - you might never correct your error.

I think avoiding Type I error is optimal. This is how I consciously try to argue on this blog. My feeling is that, in the long run, I am much better off if I am skeptical of theories. I'll probably reject a few true theories here and there - indeed, I am sure I have already done that a few times. But, by being skeptical, and by asking for sound reasoning and solid proof before I accept a theory, I think that I am much less likely to commit Type I error, and therefore go seriously askew.

Update, 2 PM EST: I have added a continuation of this post, which can be accessed here.