Grading Criteria
This report assesses the completeness, timeliness, and accessibility of K‑12 education spending data made available to the public on each state’s department of education website. Each state is scored on a 100 point scale and assigned a corresponding letter grade as follows:
97–100 points: A+
93–96 points: A
90–92 points: A-
87–89 points: B+
83–86 points: B
80–82 points: B-
77–79 points: C+
73–76 points: C
70–72 points: C-
67–69 points: D+
63–66 points: D
60–62 points: D-
50–59 points: F
0–49 points: F-
Each state’s department of education website is scored across four categories: Per Pupil Expenditures, Total Expenditure Data, Average Salary Data, and Public Accessibility. The scores for the first three categories are divided equally between statewide financial data and district-level financial data. An asterisk in the score box indicates that points were deducted, with an explanation in the text below. Scores are based on the data provided on each state’s department of education website as of December 31, 2012. Data provided by other departments and/or on separate websites are not considered for grading purposes unless the state’s department of education website links to the outside data.
| Grade |
Letter |
| Per Pupil Expenditures | Max 45 points |
| Total Expenditure Data | Max 30 points |
| Average Salary Data | Max 10 points |
| Public Accessibility | Max 15 points |
| Total Score | Max 100 points |
Before detailing the scoring process, it is important to note one significant limitation. While the terms “total expenditures” and “total per pupil expenditures” should include all categories of expenditures associated with operating the public school system — including instruction, administration, transportation, food service, building construction, debt service, and so on — this report does not penalize state departments of education that do not include the full cost of employee pensions in their calculation of total expenditures. This decision stems from the way that the U.S. Department of Education’s National Center for Education Statistics (NCES) accounts for public school employee pensions. The NCES allows states to define public school employee pension costs as merely what school districts contribute annually toward the state employee pension fund (or its equivalent) rather than the present value of future pension payments for which public school systems are liable.
According to a recent Heritage Foundation analysis, the full cost of public school employee pensions is approximately three to four times greater than the recorded NCES estimates.1 To more accurately reflect the true cost of public education, the NCES should revise the way it accounts for pension costs. However, in order to draw a clear distinction between states that otherwise provide total per pupil expenditures and those states that omit capital expenditures and other big budget items from their per pupil expenditures calculation, this report refrains from penalizing states for following the existing problematic NCES accounting guidelines.
Any grading system is open to the criticism that changing the weighting would change the results. In the spirit of transparency, this report is clear about how each factor was scored and why some factors were given greater weight than others. With the data provided, anyone who so desired could change the weights and produce their own grades.
The purpose of publishing government financial data is to understand how, and how well, public dollars are being spent. This report weighs several factors according to their relative importance in fulfilling that purpose. The fairest and most meaningful way to compare spending levels among states and districts of various sizes, and to measure trends in spending over time, is to look at per pupil expenditures (PPE). PPE represents the average “price” of a year’s worth of public schooling, and can be used to determine the average cost of educating a given-sized class of students, making it the most intelligible measure by which the public can evaluate school system spending and efficiency. Accordingly, this report gives PPE the most weight with almost half the possible points awarded (45 points).
The second largest category (30 points) is total expenditures, which also includes the most common major spending items: capital expenditures, total salary expenditures, and pensions. These data are crucial to understanding the cost of public schooling compared to that of other government functions. Nevertheless, it is more difficult to compare total spending across districts or states of varying sizes, and to assess aggregate spending trends that may be affected by increasing or decreasing enrollment. This category is therefore given less weight than PPE.
The smallest category (10 points) is average salary data, which includes employee benefits. Though employee compensation is only a portion of total spending, it is generally among the most significant. These data are necessary to provide context to understand how teachers, administrators, and nonteaching staff are compensated relative to other districts and other professions.
Of course, none of the above data are useful to the public unless the public is able to easily find and understand them. Unlike the above categories, which rely entirely on objective measures, this category relies on objective and subjective measures. The public accessibility category is therefore awarded only 15 points, the equivalent of one and a half letter grades, to balance the importance of accessibility with the desire to limit subjectivity as much as possible. The scoring of all the subjective elements (navigation, ease of public analysis, etc.) is detailed at the bottom of the report for each state.
The weights for each of the subcategories are explained in greater detail below.
|
Per Pupil Expenditures |
|||
| Category |
Metric |
Full Credit |
Score |
| Type of PPE | Total or OperatingPer Pupil Expenditures | Total | Full or Half Credit for PPE Section |
| Years of Data | Number of years of data provided. | 10+ years | Max 20 points |
| Most Recent Year | Most recent year forwhich data are provided. | 2011-12 | Max 20 points |
| Historical Comparison | Whether state provides multiple years of inflation-adjusted PPE figures in a table or graph. | Included | Max 5 points |
| Total |
Max 45 points |
||
Per Pupil Expenditures (45 points):
This section measures the completeness and timeliness of each state’s reported per pupil expenditure (PPE) data. Full credit for the entire section is awarded when a state department of education provides total PPE. States that provide only partial figures, often called “operating” or “current” PPE, will receive half credit.
Twenty points are awarded to states that provide 10 or more years of PPE data. One point is deducted at both the state and district levels for each missing year of data.
Twenty points are awarded when the most recent year of PPE data provided covers the 2011-12 school year. Two points are deducted at both the state and district levels for each year the data is behind.
Five points are awarded to states that provide a table or graph with ten or more years of inflation-adjusted PPE data at both the state and district levels. One half-point is deducted for data that is not inflation-adjusted at both the state and district levels. One quarter-point is deducted for each missing year of data at both the state and district levels. Points are rounded up to the nearest quarter-point.
|
Total Expenditure Data |
|||
| Category |
Metric |
Full Credit |
Score |
| Total Expenditures | Granularity of total expenditures | Object / Summary | Max 8 points |
| Capital Expenditures | Whether capital expenditures are included as a line item | Included | Max 4 points |
| Total Salary Data | Granularity of total salary data and whether data on employee benefits are provided | Object / Summary with Benefits | Max 4 points |
| Pension Data | Whether data on contributions to the employee retirement fund are provided | Included | Max 4 points |
| Years of Data | Number of years of data | 10+ years | Max 5 points |
| Most Recent Year | Most recent year for which data are provided | 2011-12 | Max 5 points |
| Total | Max 30 points | ||
Total Expenditure Data (30 points):
This section measures the completeness and timeliness of each state’s reported total expenditure data, including three important sub-categories: capital expenditures, total salary expenditures, and pensions.
Eight points are awarded to states that provide total expenditures at both the summary and object levels. To be considered object-level detail, the state must provide data at both the summary and object level (e.g., instruction, administration, transportation, etc.). Three points are deducted at both the state and district levels if total expenditure data are provided at the summary level only. One point is deducted at both the state and district levels if total expenditure data are provided at the object level only.
Four points are awarded if capital expenditures are provided as a line item and included in total spending. One point is deducted at both the state and district levels if capital expenditures are given as a line item but not included in the total.
Four points are awarded if total salary expenditures are provided at the summary and object level (e.g., teachers, administrators, and non-teaching staff). One point is deducted at both the state and district levels if total salary data are provided at the summary level only. Half of one point is deducted at both the state and district levels if total salary data are provided at the object level only. Half of one point is deducted at both the state and district levels if employee benefits are not provided as a line item.
Four points are awarded if expenditure data related to the state or district’s contribution to the public employee pension fund are provided as a line item and included in total spending at both the state and district levels.
Five points are awarded to states that provide 10 or more years of expenditure data. A quarter of one point is deducted at both the state and district levels for each missing year of data.*
Five points are awarded when the most recent year of expenditure data provided covers the 2011-12 school year. Half of one point is deducted at both the state and district levels for each year the data is behind.
Note: It is possible that there exists discrepancy in time coverage between Total Expenditures and one of the other categories (Capital Expenditures, Total Salary Data, or Pension Data). Since Total Expenditures is the primary category, when the time coverage score of total expenditures is lower than the other categories, that will be the final score. If the time coverage score of total expenditures is greater than the other categories, the final score will be the total expenditures score minus half of one point per category at the state and district levels.
|
Average Salary Data |
|||
| Category |
Metric |
Full Credit |
Score |
| Categories Covered | Categories of employees for which average salary data is provided. | Teachers, administrators, and non-teaching staff. | Max 3 points |
| Employee Benefits | Whether employee benefits are provided as a line item. | Included | Max 2 points |
| Years of Data | Number of years of data. | 5+ years | Max 2.5 points |
| Most Recent Year | Most recent year for which data are provided. | 2011-12 | Max 2.5 points |
| Total | Max 10 points | ||
Average Salary Data (10 points):
This section measures the completeness and timeliness of each state’s reported average salary data, including employee benefits.
Three points are awarded for providing average salary data for teachers, administrators, and non-teaching staff. Half of one point is deducted at both the state and district levels for each missing category.
Two points are awarded for providing data on average employee benefits as a line item at both the state and district levels.
Two-and-one-half points are awarded to states that provide five or more years of average salary data. A quarter of one point is deducted at both the state and district levels for each missing year of data.
Two-and-one-half points are awarded when the most recent year of average salary data provided covers the 2011-12 school year. Half of one point is deducted at both the state and district levels for each year the data is behind.
|
Public Accessibility |
||
| Category |
Metric |
Score |
| Navigation | Level of difficulty for a layperson to find financial data on the department of education website | Max 5 points |
| Ease of Public Analysis | Level of difficulty for a layperson to analyze financial data on the department of education website | Max 10 points |
| Total | Max 15 points | |
Public Accessibility (15 points):
This section measures the navigability of each state’s department of education website and the ease with which an average citizen can analyze the data. The Navigation component considers a number of factors including the number of clicks necessary to reach the desired data, whether or not the names of the links are intuitive, whether there are “red herrings” that confuse and frustrate users, and so forth. The Navigation component utilizes the following scoring scale:
Full Credit — Very easy for a layperson to locate the desired data. All relevant data are in close proximity with a main menu that is clear and commonsensical.
4 points — Fairly easy for a layperson to locate the desired data. Minor design flaws that slightly inhibit navigability.
3 points — Somewhat difficult for a layperson to locate the desired data. Moderate design flaws that somewhat inhibit navigability.
2 points — Difficult for a layperson to locate the desired data. Serious design flaws that significantly inhibit navigability.
1 points — Very difficult for a layperson to locate the desired data. Major design flaws that considerably inhibit navigability.
0 points — Nearly impossible for a layperson to locate the desired data.
The Ease of Public Analysis component measures the relative ease or difficulty for a layperson to analyze the data. Ten points are awarded for data that are presented in both Microsoft Excel and either PDF or HTML. Excel is the most widely used spreadsheet software and hence it is the most accessible format in which average citizens can analyze data. However, since many citizens do not own the Excel software, it is important that state departments of education also provide the data in either PDF or HTML format.
One-and-one-half points are deducted when a state provides data in PDF or HTML plus in Microsoft Access, but not in Excel.
Two-and-one-half points are deducted when a state provides data in only one of the above formats.
Other common deductions include
- Minus 1 point: Use of jargon that the public cannot be reasonably expected to understand.
- Minus 2 points: Lack of instructions to read data, or unclear instructions.
- Minus 2 points: Excel files are not clearly titled.
A few states reported total PPE in one document while providing only operating PPE in a separate document. This can confuse average citizens who are attempting to obtain PPE data, particularly when the smaller figure is reported in a user-friendly document while the complete figure is hidden in a complicated spreadsheet. The following criteria explain how various scenarios were scored:
- Minus 7.5 points: Operating PPE are presented as “Total PPE” or simply “PPE” without any definition or explanation in a user-friendly document that is clearly intended for lay people.
- Minus 5 points: Operating PPE are presented as “Total PPE” or simply “PPE” without any definition or explanation in a document that is not significantly more user friendly or prominently placed on the website.
- Minus 2.5 points: Operating PPE are identified as such though they are in a user-friendly document intended for lay people and/or it is significantly more likely that a person looking for PPE will find the operating PPE before finding total PPE.
- No points lost: The site provides both Operating and Total PPE. Operating PPE is identified as such and is in a document that is not significantly more user friendly or more prominently placed on the website.
Notes
1 Richwine, Jason, “Official Education Spending Figures Do Not Incorporate Full Cost of Teacher Pensions,” Heritage Foundation, March 25, 2013. http://www.heritage.org/research/reports/2013/03/official-education-spe…