As state education departments and school districts continue to grapple with school closure in the service of limiting the spread of COVID-19, the US Department of Education has waived the federal requirement for states to administer annual assessments of student achievement for this school year. The overwhelming majority, 40 at last count, of the states have either cancelled their state-wide assessments or are in the process of doing so.
My recent report for the Manhattan Institute looks at the performance of students in New York City and State on the 2019 versions of the New York State Assessments and the National Assessment of Educational Progress (NAEP). Both exams are administered in grades 4 and 8, enabling us to compare results from the two instruments. (The state assessments are administered to virtually all students; NAEP is administered to a small, but randomly chosen sample of students, representative of each jurisdiction.) Results are presented in terms of the percentage of students scoring at proficiency level or above.*
Each state determines the cut-off score on their annual exam that establishes the boundary for proficiency. Our analysis of the 2019 results for New York reveals a difference between what the State Assessments tell us about the achievement of New York City relative to that of New York State than does the NAEP.
In both grades, the State Assessments estimate significantly higher proficiency rates than does the NAEP. In 2013, the State Education Department overtly set its cut-off thresholds for proficiency to produce estimates like those seen on NAEP. In the intervening six years, scores on NAEP have remained relatively flat, while those on the State Assessments have risen.
State Assessment scores have risen more steeply in the city than in the state. As a result, the State Assessment results tell us that the average score for New York City students is higher than New York State’s in both grades in English Language Arts.
On the NAEP, however, the trend is reversed; New York State’s scores are 6 to 7 points higher than those for New York City.
What might explain this difference? We can’t be sure, but there are very real and valid reasons for students, schools and teachers to focus on the state assessments. These are high stakes tests; their results are used to assess schools and to inform admission decisions to selective schools and programs. NAEP, on the other hand, serves a single purpose: to estimate the performance of a large group of students. NAEP publishes no scores for individual schools or students.
There is also a familiarity factor. Most teachers and students never see a NAEP exam because it is administered bi-annually to a small sample of students. State assessments are administered annually to every student; the format and presentation of the questions become familiar to all, though the questions vary from year to year.
When exam results are used to inform decision-making, schools and staff have incentives to do some level of test preparation and the results of those exams reflect that emphasis. When the exams are distant and not used for local decision-making, such as NAEP, no such preparation is called for. The results may be purer in that regard, but they are of no utility to schools or teachers.
For unfortunate reasons, 2020 will be a year without state-level assessments. We’ll be able to see how states and districts respond to this lack of information in ways that may inform future discussions about the value of these exams.
Ray Domanico is a senior fellow and director of education policy at the Manhattan Institute. He is the author of the new report, "NYC Student Achievement: What State and National Test Scores Reveal."
*Only results for English Language Arts on the State Assessments and Reading on the NAEP are presented here. New York State exempts its highest-achieving eighth grade students from its math assessments as they are enrolled in, and will be tested in, a high-school-level state math exam, making comparisons with other test results difficult. Math results are presented in my full report
Interested in real economic insights? Want to stay ahead of the competition? Each weekday morning, e21 delivers a short email that includes e21 exclusive commentaries and the latest market news and updates from Washington. Sign up for the e21 Morning eBrief.
Photo by Kali9/iStock