I’m indebted to George Lucas for conceiving Star Wars and the Indiana Jones franchise —episode 4 in 2008, whoooo! But the latest issue of his education magazine, Edutopia, rediscovers a cult science fiction classic that’s particularly dear to my heart: The 1991 Sandia National Labs report titled “Perspectives on American Education.”
The “Sandia Report,” as it’s known to its devotees, claimed that America had not suffered an academic decline as critics alleged. Though the original Sandia Report was never published, it became an instant hit. Anyone wanting to defend the record of U.S. public schools seemed to have — and quote — a copy.
By far, the Sandia Report’s most popular claim was that despite declining average SAT scores, SAT performance was actually going up! How could that be? “Simple” explains Lucas’ Edutopia:
[S]tatisticians call it Simpson’s paradox: The average can change in one direction while all the subgroups change in the opposite direction if proportions among the subgroups are changing.
Sandia claimed, as Edutopia repeats, that “[b]etween 1975 and 1988, average SAT scores went up or held steady for every student subgroup.”
The funny thing is they didn’t. The scores for at least one of the ethnic subgroups went down, the subgroup that made up the majority of test takers: white students.
I first came across, and debunked, Sandia’s claim in 1994, after it was cited by David C. Berliner in his essay titled (ironically, it turned out) “Educational Reform in an Era of Disinformation.”
I looked at Berliner’s figures, looked at the Department of Education figures, and called “Baloney!” Berliner subsequently added a correction to his essay.
There are several other problems with Sandia’s assertions, but claiming that a score had gone up when it had really gone down is a pretty tough act to top.
I don’t mean to come down too hard on Edutopia. After all, the magazine has just repeated a myth that has taken on a life of its own. But it’s a good lesson for anyone writing about the U.S. public education monopoly: “If it doesn’t sound bad enough to be true, then it probably isn’t.”
