The recently released RAND study, Charter Schools in Eight States, offers a strong contribution to the never-ending, sometimes tiresome debate about whether students in charter schools do better or worse academically than comparable students in traditional schools. The RAND researchers have conducted a very careful study and should be lauded for their efforts. However, such quantitative studies fail to ask why the metric being used to determine "academic achievement" is being used in the first place.

In our recent qualitative study, Inside Urban Charter Schools, we examined several high-performing schools on the well-regarded Massachusetts State Assessment System (MCAS). When looking inside many of the classrooms in these schools, we found a remarkably low level of cognitive demand being placed on students. The instructional emphasis frequently was on procedure, not on conceptual understanding. Students were not being asked to think for themselves, nor were they being asked to conjecture, evaluate, or assess. Why? Because the tests that hold these charter schools accountable do not measure higher-order thinking.

The quality of the high stakes state tests typically used in the quantitative studies presents an unnecessarily low ceiling on what we ask kids to know and be able to do. It also presents an inflated idea of “success.” It is time to spend some of the stimulus money on developing better measures of student knowledge in all of our schools and supporting more nuanced qualitative, as well as quantitative, studies of the practices of these "successful" schools.