David and Goliath: Underdogs, Misfits, and the Art of Battling Giants

students at the old crowded Shepaug—right?
    It turns out that there is a very elegant way to test whether this is true. Connecticut has a lot of schools like Shepaug. It’s a state with many small towns with small elementary schools, and small schools in small towns are subject to the natural ebbs and flows of birthrates and real-estate prices—which means that a grade can be all but empty one year and crowded the next. Here are the enrollment records, for example, for the fifth grade in another Connecticut middle school:
    1993   18
    1994   11
    1995   17
    1996   14
    1997   13
    1998   16
    1999   15
    2000   21
    2001   23
    2002   10
    2003   18
    2004   21
    2005   18
    In 2001, there were twenty-three fifth graders. The next year there were ten! Between 2001 and 2002, everything else in that school remained the same. It had the same teachers, the same principal, the same textbooks. It was in the same building in the same town. The local economy and the local population were virtually identical. The only thing that changed was the number of students in fifth grade. If the students in the year with a larger enrollment did better than the students in the year with a smaller one, then we can be pretty sure that it was because of the size of the class, right?
    This is what is called a “natural experiment.” Sometimes scientists set up formal experiments to try and test hypotheses. But on rare occasions the real world provides a natural way of testing the same theory—and natural experiments have many advantages over formal experiments. So what happens if you use the natural experiment of Connecticut—and compare the year-to-year results of every child who happens to have been in a small class with the results of those who happened to have come along in years with lots of kids? The economist Caroline Hoxby has done just that, looking at every elementary school in the state of Connecticut, and here’s what she found: Nothing! “There are many studies that say they can’t find a statistically significant effect of some policy change,” Hoxby says. “That doesn’t mean that there wasn’t an effect. It just means that they couldn’t find it in the data. In this study, I found estimates that are very precisely estimated around the point zero. I got a precise zero. In other words, there is no effect. ”
    This is just one study, of course. But the picture doesn’t get any clearer if you look at all the studies of class size—and there have been hundreds done over the years. Fifteen percent find statistically significant evidence that students do better in smaller classes. Roughly the same number find that students do worse in smaller classes. Twenty percent are like Hoxby’s and find no effect at all—and the balance find a little bit of evidence in either direction that isn’t strong enough to draw any real conclusions. The typical class-size study concluded with a paragraph like this:
    In four countries—Australia, Hong Kong, Scotland, and the United States—our identification strategy leads to extremely imprecise estimates that do not allow for any confident assertion about class-size effects. In two countries—Greece and Iceland—there seem to be nontrivial beneficial effects of reduced class sizes. France is the only country where there seem to be noteworthy differences between mathematics and science teaching: While there is a statistically significant and sizable class-size effect in mathematics, a class-size effect of comparable magnitude can be ruled out in science. The nine school systems for which we can rule out large-scale class-size effects in both mathematics and science are the two Belgian schools, Canada, the Czech Republic, Korea, Portugal, Romania, Slovenia, and Spain. Finally, we can rule out any noteworthy causal effect of class size on student performance in two countries, Japan and Singapore.
    Did you follow that? After sorting through thousands of pages of data on student performance from eighteen separate countries, the economists concluded that there were only two places in the world—Greece and Iceland—where there were “nontrivial beneficial effects of reduced class sizes.” Greece and Iceland? The push to lower class sizes in the United States resulted in something like a quarter million new teachers being hired between 1996 and 2004. Over that same period, per-pupil spending in the United States soared 21 percent—with nearly all of those many tens