‘Reform’ Cities’ Results Are More Complex Than New Report Suggests

Share this

Recently, the Broader, Bolder Approach to Education campaign released a new report aimed at assessing the impact of reforms in New York City, Chicago, and Washington, D.C. A central claim of this report is that “test scores increased less, and achievement gaps grew more, in [these] ‘reform’ cities than in other urban districts.” But a thorough analysis of achievement data for these districts shows a much more complicated picture than this statement suggests.

To understand patterns of achievement and equity in these districts, we used data from the National Assessment of Educational Progress Trial Urban District Assessment (TUDA) to answer the following questions:

  • How much did these districts improve for students overall and for groups of students between 2003 and 2011?
  • How much did gaps between African-American and white students, Latino and white students, and low-income and higher income students narrow in these districts over the same time period?
  • How do these districts compare to other large districts and to the “large city” average in terms of improvement and gap-closing?

As you can see in the attached tables, each of the three districts featured in the report outperformed the large city average in some cases and performed worse than the large city average in others. For example:

  • In Chicago, nearly all groups of students improved faster than the large city average in eighth-grade math. However, nearly all groups improved slower than average in fourth-grade reading, and white/black gaps in the district generally widened.
  • In all four grades and subjects, New York City’s students generally improved less than the large city average. However, in all four grades and subjects, the white/black gap closed at a faster rate than the large city average.
  • DCPS students generally improved faster than the large city average in fourth-grade reading, but the low-income/higher income gap widened in all four grades and subjects.

We also examined state assessment results to understand how New York City and Chicago compare to their respective states in terms of improvement. Here, too, the story is mixed. These districts outperformed their states in some cases, but improved at the same rate as or slower than their states in others. For example:

  • In fourth-grade reading and math, New York City improved faster than New York state for all groups of students. But in eighth-grade reading, the state improved faster than the city for most groups.
  • Chicago consistently improved more rapidly than Illinois for students overall, white students, and higher income students. However, the district never improved faster than the state for Latino students.

Raising achievement and closing gaps between groups requires an honest examination of where we’re making progress, and where we’re struggling. Only with this information can we begin to learn from and scale up practices that are working and tackle those that aren’t head on. Anything less than a comprehensive look at the data does a disservice to this hard, but essential work.


TUDA_Analysis_by_District.pdf66.57 KB