Because I work on federal education policy, I receive a steady stream of emails from the White House Office of Public Liaison (read: Office of Spin) suggesting that the No Child Left Behind Act is saving American education. Today, only moments after I’d read the story on the Washington Post’s website, the OPL folks sent me an “in case you missed it” copy of a Post report suggesting that thanks to NCLB, achievement scores for low-income kids in some DC suburbs have increased:
Since enactment of the No Child Left Behind law, students from poor families in the Washington area have made major gains on reading and math tests and are starting to catch up with those from middle-class and affluent backgrounds….The achievement gap between economic groups, long a major frustration for educators, has narrowed in the region’s suburban schools since President Bush signed the law in 2002, according to Maryland and Virginia test data.
It’s entirely possible, I should make clear, that achievement gaps on Virginia and Maryland tests have closed. The problem is, it’s not possible to attribute such changes to NCLB, because so many other factors than the law—changing demographics, state reforms, etc.—could have been at work. And that’s not the biggest problem with dubbing NCLB a savior. The biggest problem is that states have been changing their own tested populations, test content, cut scores, and other important testing variables throughout the NCLB era, rendering truly comparable scores a seriously threatened species.
For well-documented evidence of this, check out the Center on Education Policy’s compilation of state-testing policies and outcomes. CEP has files for every state, and to get through them all requires a fair amount of downloading and scrolling. Examining each state’s “major changes in testing system,” however, will reward the effort, as will taking in the trends of comparable results from year to year. Bottom line: You can tell almost nothing about outcomes under NCLB because so many state testing systems have been significantly altered—and re-altered, and altered again—during the NCLB era.
That said, there is a bit of good news for Maryland: The state has comparable data generally going back to 2004—a pretty abbreviated trend line, but not the shortest of any state—and the percentage of kids overall hitting “proficiency” seems to be going up. That might still reflect test questions getting easier, better test “gaming” by teachers, and other less-obvious changes than new cut scores or completely new tests, but at least it’s a few years of general comparability and improvement.
For Virginia, there is no NCLB Santa Clause: The state has only two years of comparable data, and while the percentage of kids hitting proficiency rose between 2006 and 2007, it’s not nearly enough info to identify any kind of a trend.
So here’s the overall NCLB story based on what can reliably be reported: When it comes to the law’s effectiveness, it seems the truth, no matter what is proclaimed, isn’t really out there.
