Standardized math tests that are too predictable can lead to bad test prep and coaching. But when it comes to reading tests, predictability is essential if we want to level the playing field for kids from less educated families.
For the past several decades, American education reformers have focused on reading and math test scores as the primary means of holding schools and teachers accountable for improving student performance. A recent book by a Harvard education professor argues convincingly that this regime has led to dire and unintended negative consequences while failing to achieve its aims: namely, raising student achievement and narrowing the gap between the highest- and lowest-achieving groups of students. But in his recommendations for correcting the situation, the author–Daniel Koretz–overlooks a crucial distinction between reading and math tests.
Koretz’s book, The Testing Charade, concludes that the system of test-based accountability–which began in earnest in 2002 after the passage of No Child Left Behind–has yielded nothing more than modest gains in elementary math scores. High school math scores haven’t improved, nor have reading scores at any level.
And while the gap in scores between white and black students has narrowed, Koretz says that much of that progress started–and ended–before test-based accountability became widespread. Meanwhile, the gap between the wealthiest and poorest students has only been getting wider.
As Koretz details, requiring teachers to do the impossible–equipping virtually every student to achieve “proficiency” within an unrealistic period of time–has led to narrowly focused test prep, the neglect of important but untested subjects like history and social studies, and outright cheating. Basically, Koretz declares the effort to reform education through high-stakes testing to be an abysmal failure.
But he doesn’t argue that we should dispense with standardized tests entirely. We need some way of comparing the performance of students at schools that vary widely. An “A” at a school serving low-income students, for instance, often means something very different from an “A” at a school in an affluent community. Standardized tests alone won’t correct educational inequities, but they’re very effective at uncovering them.
Koretz advocates supplementing standardized test scores with other measures, including some that allow for human judgment: tests that teachers create themselves, inspections conducted by outside education professionals. While there are risks to introducing subjectivity, Koretz argues that teaching is simply too complex an activity to be evaluated without it. And, he points out, other countries have managed to incorporate human judgment into their evaluation systems successfully.
The argument against predictability
So far so good. But Koretz also argues that the tests themselves need to be reformed–specifically, that they need to be less predictable. If teachers can predict what’s going to be on a given high-stakes test, he says, they’re likely to focus on “the specifics of the test at the expense of the larger domain.” The result will be score inflation: the artificial increase in scores that reflects “bad test prep” rather than an actual increase in achievement.
Koretz’s argument against predictability makes sense when it comes to math tests–and while he notes one instance of apparent score inflation on a state reading test, all of his other examples relate to math. But reading and math tests differ in one fundamental respect, and it’s one that Koretz largely overlooks.
Both kinds of tests purport to test skills, and when it comes to math that claim is justified. In math, the skills are the content. But the content of “reading,” obviously, varies. And reading skill varies depending on the reader’s familiarity with the content. In one famous experiment, kids who were “poor readers” and knew a lot about baseball could read better than kids who were “good readers” but knew little about the sport–when the topic was baseball. That finding has been replicated in numerous other contexts.
If a math test is too predictable, teachers may focus only on the aspects of algebra or geometry they expect will be tested and skip over other important components of those subjects. Teachers may also try to predict what will be on a reading test, but because there’s no way to predict the content of the reading passages, they can only focus on skills. If students don’t have enough relevant background knowledge and vocabulary to understand a test passage in the first place, however, no amount of practice with skills will help them.
Koretz gets the distinction between reading and math, but only up to a point. Compared to “content-rich” subjects like history and science, he says, reading doesn’t lend itself as much to bad test prep. Teachers can’t focus on just some fraction of the content–because the “subject” of reading has no content.
In fact, though, teachers across the country are engaging in bad reading test prep every day, by focusing on skills. If teachers know that the test will ask students to identify the main idea of a passage or connect a claim about the passage to evidence in the text, they’ll have students practice those skills without paying any attention to building their content knowledge.
Why predictability in reading tests can help
Koretz seems to assume that teachers can’t do much to improve “reading skill.” But there’s a lot they could do, if they understood the importance of building students’ knowledge. Instead of spending hours every day–year after year–drilling students on skills of limited utility, they could be teaching them the history and science that would actually increase their chances of understanding what they read.
Similarly, Koretz more or less throws up his hands at the idea that schools can do much to narrow the achievement gap in reading. He notes that “a considerable share of reading skill is picked up out of school, starting with parents reading to young children.” But it’s not so much a skill that’s being picked up by those fortunate kids, most of whom come from relatively well educated and affluent families, as knowledge of the world and the vocabulary that goes with it. As Koretz points out, schools that serve low-income students are the most likely to be engaging in test prep–including trying to teach reading comprehension skills divorced from content. (Koretz also argues that the focus on test prep in such schools has led to more score inflation there, making the achievement gap appear narrower than it really is.)
Here’s where the value of predictability comes in. As long as we’re stuck with reading tests, how about at least giving teachers some clue about what content the reading passages will cover, so that their students will have a fighting chance of displaying their reading “skills”? Right now, the topics of passages are guarded like state secrets. The only thing teachers can predict are the skills that will be tested–so, naturally, that’s what they focus on. But if they could predict that the passages were likely to draw on students’ knowledge of, say, the Civil War or life in the Arctic, they could focus some of their instruction on those topics instead of on endless practice in “making inferences” or “comparing and contrasting.” For students whose general knowledge is limited, that kind of predictability could be a lifesaver.
It’s possible, of course, to go too far in this direction, even on reading tests. I’m reminded of stories I’ve heard about the British system, in which schools are allowed to choose a limited number of literary works to have their secondary students tested on, with the result that students may spend two years doing nothing but intensively studying a couple of novels. But surely it’s also possible to find a happy medium–for example, the announcement in advance of a number of general topics in history, literature, and science, some of which will be covered on the test.
At the very least, this kind of predictability would send teachers and the public the message that reading tests are–as cognitive scientist Daniel Willingham has put it–actually knowledge tests in disguise. And that the best way to prepare students for them is not to teach comprehension skills but to provide them with knowledge.