In a response to my post this morning, Charles Murray remains unconvinced that changes to our school system could result in dramatic improvements in educational outcomes.

He asks to see the scholarly study showing that a school has miraculously boosted achievement above the norm. In one way, this hurdle is too low, and in another it’s too high.

If we could only point to a single study of a single school, it wouldn’t instill much confidence in the generalizability of the phenomenon. A consistent pattern of scholarly results is necessary for that. On the other hand, asking for “miraculous” improvement is a needlessly high standard. My disagreement is with Murray’s earlier, lower threshold claim that: “reforms of the schools can never do more than produce score improvements at the margin.”

Let’s call a marginal improvement an increase of less than .15 standard deviations above the current mean (typically considered a “small” effect in the social sciences). Taking that as our litmus test, is there a consistent pattern of scholarly evidence that better school system design can boost achievement by more than .15 standard deviations? Yes.

That pattern is presented in the figure above, drawn from my recent review of the global econometric literature comparing educational outcomes across different types of school systems. The figure relates the number of statistically significant findings favoring free education markets over state school monopolies (in white), significant findings of the reverse (in light grey), and insignificant findings (in dark grey). Markets beat monopolies by a ratio of 15 significant findings to 1, across the seven educational measures for which data are available.

While a few of these findings have small effect sizes, many are above .15 standard deviations — some of them well above it. A paper by Tooley, Dixon, Bao, and Merrifield (under consideration by the journal Economics of Education Review), for instance, finds that in Nigeria private schools outscore public schools by double that amount, after controls, while “in Delhi and Hyderabad private unrecognized schools top state-run schools in math instruction by about 2/3 of a standard deviation.” A recent randomized assignment study of the DC voucher program finds that voucher students who’ve been in the program for three years are reading two grade levels ahead of their public school peers (.42 std deviations), though the average voucher is worth only a quarter of what DC spends per pupil on public k‑12 education.

These are more than marginal improvements, and they are part of a consistent pattern. That pattern strongly suggests that moving from our current monopoly school system to a free and competitive education marketplace would shift the bell curve of academic achievement significantly to the right, raising the mean achievement substantially above its current level.

No one should be surprised by that. Imagine how far the bell curve for median income across modern nations would shift to the left if all free markets were supplanted with centrally planned monopolies such as have ruined the economies of Cuba, North Korea, and until recently many other nations.