A Teacher Data Compromise: Only Release Scores For Those At The Very Top And Bottom
Several newspapers have filed Freedom of Information Law requests seeking to force the city to release “value-added” scores for public school teachers. The papers want to follow in the footsteps of the Los Angles Times, which recently published the names and scores of about 6,000 L.A. teachers. The United Federation of Teachers has filed a lawsuit to block the release.
Value-added - a statistical tool used to compute an individual teacher’s contribution to his students’ test scores - is a far superior gauge of teacher quality than conventional measures, such as years of experience or credentials. It was a step in the right direction when lawmakers in Albany required student test score performance to play a role in teacher evaluations, and city Schools Chancellor Joel Klein was right to order principals to consider value-added scores when determining whether to offer a teacher tenure.
But Klein and the media must proceed with caution. These analyses have several limitations that should give pause to those looking to publish teachers’ scores. If the names must be made public, the papers would do well to release only the names and ratings of teachers whose value-added scores are very high or very low based on an analysis of at least three years of data.
UFT President Michael Mulgrew asserts that New York City’s value-added scores are inaccurate because they are based on faulty tests. He’s misrepresenting a recent controversy, which had to do with the cutoff scores that students must reach in order to be deemed “proficient.” Another common claim is that value-added scores blame teachers for poor student performance caused by disadvantages at home. That’s wrong. The procedure accounts for such factors.
Nonetheless, there are fair criticisms of value-added. The limitations are strong enough that no one who works with these data believes that they should be used in isolation to evaluate teachers.
As with any statistical tool, value-added is influenced by randomness. Thus, there will always be some bad teachers who get a high valued-added ranking and some good teachers who get low marks. The validity of value-added assessment improves with the number of years and students with which a teacher is observed.
Scores based on three or more years of data deserve our attention; reportedly, 41% of the reports requested by the newspapers meet this standard.
Value-added also isn’t nearly as good at ranking moderately effective teachers as it is at identifying those teachers who consistently perform much better or worse than their peers. It’s as if you took the weight of a hundred people: It would be tough to accurately rank those of about average size, since a person’s weight might be slightly different every time it is taken - 150 pounds one day, 148 another - but you will be able to tell who is thin and who is obese.
So: Newspapers should release only the scores of teachers who fall within a range, say the top and bottom 5%, among those whose scores were produced with three years or more of underlying data.
Looking only at the best and worst teachers is not only more reliable, it is better aligned with how these scores might be used in the future. Teachers whose students are excelling should be rewarded with higher pay and more responsibilities. Teachers whose students consistently lag behind their peers should be removed from the classroom. For the vast majority of teachers in the middle, value-added should be used primarily as a motivational tool.
Let’s not forget that goal: It’s far more important that the value-added assessments have consequences within the school system than that they appear in a public document.
So for now, focus on scores based on multiple years of observations - and expose only the highest- and lowest-performing teachers.
This piece originally appeared in New York Daily News
This piece originally appeared in New York Daily News