Gerilyn Slicker, of the Fordham Institute, offers a brief review of my study of charter school philanthropy in the latest “Education Gadfly” mailing, including the following observation:

Note, though, that this analysis is not without fault. The report doesn’t break down spending by pupil (only reporting aggregate grant-giving), nor does it account for student growth over time or for how long the charter networks have been operational.

All three of these concerns are worth raising, and the first two of them were actually addressed in the paper itself. The aggregate vs. per-pupil grant funding question is discussed in endnote 15:

Note that total grant funding, rather than grant funding per pupil, is the correct measure. That is because enrollment is endogenous—it is a product, in part, of earlier grant funding. So, controlling for enrollment (which dividing by enrollment would do) would control away some of the very characteristics we are trying to measure: the charter network’s ability to attract funding.

Student growth over time, as noted on page 5, cannot be measured using the California Standards Tests, because it reports results as averages of subgroups of students at the classroom level, not as individual student scores. And since the CST is the only source that has broad-based performance data for all charter and traditional public schools in the state, it is the only dataset that can be used to measure the performance of all California charter school networks. Fortunately, good controls for both student factors and school-wide peer effects are available, and the study’s results are consistent with earlier research, where it overlaps with that research.

The final concern, network age, is not one that I directly addressed in the study. There are a couple of reasons to expect it would not have much of an impact on the findings, however. First, a cursory look at the age of some of the top networks shows no particular pattern. American Indian and KIPP are both a decade old, and rank #1 and #7, respectively, out of 68 networks. Oakland Charter Academies and Rocketship are just a couple of years old, and rank #2 and #4, respectively. Similarly, some of low performing networks are brand new, while others, like GreenDot (ranked 42nd), are also over a decade old.

Second, in Appendix E, I show that network size and network academic performance are not significantly linked to one another. And since network age and network enrollment are likely to be strongly positively correlated with one another, it would be surprising if network age were correlated with performance when enrollment is not. That said, I’d probably include network age as a control in future, if I repeat the study, just to be on the safe side.