How can researchers publish a book concluding that government schools are outperforming private schools despite all the evidence to the contrary? By ignoring all the evidence to the contrary, of course.

Writing at Education Next, Patrick Wolf casts a gimlet eye on the claims of Sarah and Chris Lubienski in their book, The Public School Advantage: Why Public Schools Outperform Private Schools:

Research on this question goes back some 30 years. From James Coleman’s early observational studies of high schools to the experimental voucher evaluations of the past 15 years, researchers have routinely found that similar students do at least as well and, at times, better academically in private schools than in public schools. How have the Lubienskis come up with this surprising finding?

Wolf notes that the Lubienskis ignore numerous performance measures — including graduation rates, college matriculation, future income, parental satisfaction — that give a decided advantage to private schools, even after controlling for student characteristics. They narrowly define performance as math scores on two national tests, ignoring the reading data that also show higher private-school performance.

Moreover, both math tests had been altered prior to the Lubienskis’ data collection to more closely align with the way that math is generally taught in government schools as opposed to private schools. By the Lubienskis’ own admission, “the professional development of math teachers changed in the late 1980s to emphasize math reasoning and problem solving and de-emphasize math facts and computations.” Government schools were more likely to embrace the new math curriculum than private schools, which “tended to continue to emphasize traditional math content.” Thus, concludes Wolf, this is a study of “how well private and public school students have learned the brand of math taught in the public schools” rather than a true measure of the differences in math skills.

Wolf also faults the Lubienskis for making apples-to-oranges comparisons of student characteristics that are “measured differently across school sectors” and for methodological flaws in accounting for students who switched from government schools to private schools or vice-versa.

Even more distressingly, the Lubienskis ignore the copious evidence from randomized controlled trials, which are the gold standard of social science research. As Jay P. Greene, Wolf’s colleague at the University of Arkansas, writes:

The net effect of these three methodological choices, plus the fact that standardized math results are more closely aligned with how the subject is taught in public than private schools, strongly skew the results in favor of public schools. The beauty of randomized experiments is that their results are not so easily manipulated by bizarre choices of what is controlled. But the Lubienskis don’t like randomized experiments. Advocates for quack medicine also tend not to like randomized experiments. They don’t let you selectively control for things until you get the answer you want.

Furthermore, Wolf’s assessment of the Lubienskis’ methodological flaws leaves aside the question of efficiency (academic achievement per dollar spent per pupil). Efficiency is an issue that apologists for the status quo prefer to avoid. For example, school-choice critics pooh-pooh results from the over-regulated Milwaukee voucher program because test scores of choice students were not significantly different from government school students (setting aside the choice students’ significantly higher levels of high-school graduation and college matriculation). However, the critics neglect to mention that the private schools achieved those results at about half the cost per pupil.

School choice programs consistently produce similar or better results for much less money. Indeed, in a 2009 review of the global research literature, the Cato Institute’s Andrew Coulson found that every study to measure efficiency in education returned a statistically significant positive result for markets.

Meanwhile, the U.S. has nearly tripled the amount spent per government school pupil in inflation-adjusted dollars in the last 40 years while test scores have remained flat and internationally unimpressive. Perhaps it’s time to give private schools their well-deserved chance?