In Preschool Debate, Politics Trumps Evidence

By Jason Richwine & Lindsey Burke - April 22, 2013

‹‹Previous Page |1 | 2 | 3 |
By Jason Richwine & Lindsey Burke

“What the Impact Study clearly shows is that Head Start does its job.” The National Head Start Association (NHSA) made this pronouncement at the end of last year. It was a strange thing to say, given that the study in question showed the exact opposite.

But such a strong disconnect between rhetoric and evidence is standard fare when it comes to the politics of preschool. As President Obama pushes a major expansion of federal involvement in preschool, it’s worth reflecting on the inflated rhetoric, false promises, and disregard of hard evidence that plague our national discourse on the issue.

Head Start, as the flagship federal preschool program, is the perfect case study. Started as a small summer program in 1965 as part of President Lyndon Johnson’s Great Society initiative, Head Start originally enrolled about 560,000 children on an operating budget of about $1 billion. Today, more than 900,000 children are enrolled in Head Start, and the program’s operating budget has grown to nearly $8 billion per year. Over the same time period, the NHSA has evolved and grown into an umbrella organization that lobbies Congress to spend billions of dollars on Head Start each year.

Perhaps the most effective part of Head Start’s marketing is the name itself. It’s a simple and powerful metaphor for something that all parents want: a way to give their kids the extra preparation they need to face the world. And Head Start’s modus operandi -- targeting young children before they hit kindergarten -- fits with the conventional wisdom that children are easier to influence at younger ages.

But does the program work? At the behest of Congress, the Department of Health and Human Services (HHS) conducted a “gold standard” experiment to answer that question. Researchers compared the growth and development of children randomly granted access to Head Start against a large control group of children who had applied for the program but were not given access due to space constraints.

By first grade, there were essentially no differences in cognitive or social development between the two groups, and the third-grade follow-up study came to the same depressing conclusion. As with many other social programs that sounded promising before being subjected to a rigorous experimental evaluation, the magnitude of the treatment effect is zero.

Those are the facts. Now enter the politics.

After the most recent HHS study was released, the NHSA set the tone with some unusually brazen spin. Here is how its press release begins: “What the Impact Study clearly shows is that Head Start does its job -- it gets at-risk children ready for kindergarten in every aspect that the study measured.”

This is almost Orwellian. The NHSA is referring to the fact that Head Start participants initially show gains over non-participants during the preschool years. The effects then fade out entirely by first grade. The whole purpose of getting children ready for kindergarten is presumably so that they can perform better once they are there. But they don’t. So in what sense at all has Head Start done its job?

The NHSA statement continues:

However, it is particularly troubling that the benefits of kindergarten readiness seem to flatten out from kindergarten through third grade. Our work with students ends when children graduate from Head Start, but it is clear that for many, their circumstances continue to hinder their success; circumstances including, but not limited to, the quality of their primary and secondary education.

Note the deceptive terminology here. The benefits of kindergarten readiness do not “flatten out” -- they disappear. That’s not a trivial distinction.

Furthermore, blaming outside forces for Head Start’s failure is logically invalid. Since the impact study was a randomized experiment, both the treatment and control groups have similar circumstances that hinder their success. If, for example, all children could attend high-performing schools after leaving Head Start, then outcomes for the treatment and control groups would presumably both go up, making the usefulness of Head Start no less questionable.

The fact that the initial test score gains quickly faded out in the HHS study should not be surprising. Researchers have observed for decades that Head Start does not seem to improve academic outcomes.

Perhaps that frustrating lack of cognitive impact is to blame for Head Start’s amorphous mission over the years. President Johnson wrote shortly after its creation that children would be receiving “preschool training to prepare them for regular school,” recounts Checker Finn of the Fordham Institute. But as study after study began to show Head Start was failing in that goal, proponents pushed for a broader mission – one with a greater emphasis on health, motivation and nutrition.

The effort to transform Head Start’s mission from an academic focus to health and social outcomes led to the program being housed at what would become the HHS, not the Department of Education. But the HHS study found that Head Start fails to improve not just cognitive outcomes, but also practically every social and behavioral outcome it examined as well. In short, the program has not lived up to any of its goals, changeable as they have been.

In the face of such depressing evidence, Head Start’s supporters have a fall-back position -- the faith that the positive effects that quickly disappeared may yet reappear later in life. Back to the NHSA press release:

The causes of the “flattening out effect” between kindergarten and third grade are not clear. But, it is clear from hundreds of studies over four decades, that desired long-term effects on life -- such as lowered need for special education, better health and wellness as teens and adults, higher high school and college graduation rates, and greater participation of parents in their child’s education -- are real and strong among Head Start participants.

Unlike the HHS study, the long-term studies referenced in the quote are non-experimental, meaning that participants and non-participants in Head Start were not randomly chosen. Although researchers do their best in non-experimental situations to control for as many factors as possible, self-selection bias often remains a problem.

For the long-term positive effects suggested by non-experimental evidence to fit with the experimental results, the impact of Head Start would have to be entirely hidden in third grade, only to reappear in the teenage years or later. The authors of the HHS study call these “sleeper effects,” but their existence is based more on hope than on evidence. As Russ Whitehurst of the Brookings Institution recently noted, in the unusual situation when the effects of an intervention fade out but then do seemingly reappear -- for example, in the oft-cited Perry Preschool Project -- the initial effects lasted far longer than they ever have with Head Start.

Faith in sleeper effects is emblematic of the larger problem with federal preschool policy. Supporters so strongly desire positive impacts that they go to great lengths to explain away the mounting evidence against them. Unfortunately, the NHSA and other groups with financial interests in the programs are all-too-happy to be enablers of false hope.

There were reasons to think the Obama administration might be more hard-headed with its education policy. When he was at the helm of the Chicago Public School System, Arne Duncan was a reformer who closed down underperforming schools and publicly clashed with the city’s teachers union. As Obama’s education secretary, Duncan has angered some in his own party by supporting charter schools and performance pay for teachers. The president himself once famously remarked that “Secretary Duncan will use only one test when deciding what ideas to support with your precious tax dollars: It's not whether an idea is liberal or conservative, but whether it works.”

When it comes to Head Start at least, someone in the administration didn’t get that memo. On Valentine’s Day, President Obama called for an expansion of government preschool, including more federal funding for Head Start. In its press release, the White House even touted the administration’s “historic investments” in the program.

But what about the fact that Head Start doesn’t work? The White House refers obliquely to the HHS study in saying that it will be imposing new regulations on low-performing Head Start centers. This response is highly inadequate. Are these requirements linked to evidence? Is there any reason to believe that Head Start centers with allegedly high operating standards do get results? Talking vaguely of new regulations is not useful unless they are connected to actual performance-based evidence, which in this case they are not.

President Obama’s proposal for expanded preschool is similarly disconnected from real-world evidence. In his recent State of the Union address, the president asserted something that has become a central White House talking point: every dollar invested in preschool will return $7 in benefits. Such a terrific benefit-cost ratio would seem to make preschool a slam-dunk investment, and one might wonder why the president (and every state governor) didn’t propose it years ago. But the truth is that the seven-for-one statistic is inapplicable to what the president is proposing.

Seven-for-one comes from an experimental evaluation of the famous Perry Preschool Project that began in the 1960s. Perry was a highly targeted and intensive intervention that provided services for participating children and their parents – with just 58 children comprising the treatment group. The children, deemed at risk of “retarded intellectual functioning and eventual school failure,” received structured classroom instruction and home visits on a weekly basis.

Researchers followed up with the Perry subjects through age 40 and found several positive outcomes, but Perry’s apparent effect on criminality drives much of the seven-for-one benefit-cost estimate. Whereas 52 percent of children in the control group had served time in jail or prison, 28 percent of the treatment group did.

Perry and the Abecedarian project, a similar intervention from the early 1970s, have been cited for three decades as examples of effective preschool programs. It may seem strange that two small programs started during the Kennedy and Nixon administrations would be the basis for grandiose claims about federal preschool programs in 2013. Why have there been no modern, scaled-up replications of Perry or Abecedarian that preschool advocates can cite?

In his upcoming book, our colleague David Muhlhausen writes that there are several reasons to be skeptical of Perry’s success. Measured effects of the program vacillated between large and small as the children aged, which is a sign that the sample was insufficiently large to counter the effects of outliers. In addition, there is some question as to whether the treatment and control groups were truly randomized in the first place, as the researchers deliberately arranged for certain siblings and other types of children to be assigned to the same experimental group.

But even if we disregard Perry’s methodological and interpretive limitations, it has little relevance to the kinds of preschool programs that the White House would like to see expanded. Recent state preschool programs are far less targeted and intensive than Perry or Abecedarian. Russ Whitehurst put it best: “Generalizations to state pre-K programs from research findings on Perry and Abecedarian are prodigious leaps of faith.”

We are not aware of any experimental evaluations of the recent state pre-K programs that the president has held up as models for the nation with amazing benefit-cost ratios. As Whitehurst also notes, these programs are even less involved than Head Start, which gives us a sense of their likely effectiveness.

In short, the Obama administration’s seven-to-one figure is the latest in a long series of truth-stretching claims made by advocates of government preschool over the years. Science and politics have never mixed well together, but there is something about preschool that seems to generate an especially strong disconnect between rhetoric and evidence.

The disconnect may be rooted in ideological differences. Some people are more comfortable with government intervention than others, and it’s possible that advocates see preschool as an entry point for further experimentation. Even if the existing preschool programs don’t work, this thinking goes, the federal structure is in place to modify and expand the interventions as necessary until the right formula is obtained.

But there are costs that go along with all of that tinkering -- costs that go beyond just the direct expenditures on the programs. Government programs can crowd out the civil society structures and social capital that have evolved to help communities function. For example, when Quebec introduced heavily-subsidized day care back in 1997, researchers found that it increased child-care enrollment at the expense of both private providers and networks of informal arrangements involving family and friends.

Since some of life’s most lasting satisfactions come from taking on responsibilities within a family or community, we want to strengthen civil society rather than erode it. This happens through initiatives that engage communities with little or no government interference. In our view, the goal of public policy should not be to provide direct government support, but instead to provide the conditions in which private initiatives can flourish.

‹‹Previous Page |1 | 2 | 3 |

Jason Richwine, Ph.D., is a senior policy analyst in empirical studies and Lindsey M. Burke is the Will Skillman Fellow in Education Policy at The Heritage Foundation,

Blurred Lines on Campus
Ruth Marcus · November 12, 2014
UC Silent When Prof Trashes Free Speech
Debra Saunders · November 11, 2014

Jason Richwine & Lindsey Burke

Author Archive

Follow Real Clear Politics

Latest On Twitter