On Tuesday, I criticized a new economic modeling paper (“Anti-Lemons”) purporting to show that unfettered education markets are bad and that government can fix them with the right regulations.
Andrew Gillen comes to the study’s defense, and I’m delighted that he’s taken the trouble to reflect on it rather than just saying “I like it.” But there are problems with his analysis. First, he faults me for dismissing the “Anti-Lemons” models for being based on false assumptions, citing Paul Krugman:
I am a strong believer in the importance of models, which are to our minds what spear-throwers were to stone age arms: they greatly extend the power and range of our insight. In particular, I have no sympathy for those people who criticize the unrealistic simplifications of model-builders.
Even if we put aside the fact that Paul Krugman is at times less reliable than the Daily Show website, there is an important difference between assumptions that are “unrealistically simplified” and those that are patently wrong. With the former, your model might still huck its intellectual spear somewhere in the general vicinity of the truth, with the latter, you’re just going to put your eye out.
“Anti-Lemons” is in the put-your-eye-out camp. Among other things, it assumes the productivity of all schools is equal. This is both totally false and highly germane — efficiency varies dramatically among schools, and private schools as a whole are consistently more efficient than government schools (as we will see below). Failing to recognize that reality will lead to incorrect results from the model, and this is just one of the false assumptions the paper adopts (see my previous post for others).
Second, Gillen writes that
going by Coulson’s numbers in figure 2 here, we would expect to find a positive impact of markets over government on achievement in slightly less than 2 out of 3 studies (with insignificant findings making up the majority of the others). If the case for free markets over government schools is really so clear cut (and I lean strongly in this direction), than why isn’t this 3 out of 3?
There are many plausible reasons for this result (lack of statistical power, omitted variable bias, other misspecification errors, etc.), but one is particularly worth raising here: government schools in many parts of the world spend several times as much per pupil as their private sector counterparts. This is true in most developing countries, from which a great deal of the inter-sectoral research hails. And when I looked at statewide data from Arizona in 2006 I found that government schools spend roughly 50 percent more than private schools. While it’s true that government school outcomes tend not to improve much as spending rises, the same cannot be said of private schools.
If this is true, you might ask, then wouldn’t the inter-sectoral research on school efficiency be more stark than the research on achievement (that fails to take spending levels into account)? The answer is yes. In fact, if you examine the efficiency bar in the same figure 2 cited by Gillen above, you will see that every single one of the efficiency comparisons between market and monopoly schools is significant and favors the market schools.
So, not only is the “Anti-Lemons” model useless, it is worse than useless: it seems to mislead even intelligent readers into believing that there is some mystery in the literature that needs to be solved by blindly waiving a spear around.
“Anti-Lemons” is neither Camelot, as I said yesterday, nor is it Sparta as Andrew implied. It’s the kid from Christmas Story who nearly puts his eye out by the cavalier application of a potentially powerful tool.
