Why reading tests don’t actually test reading skills

Here’s a quick question: Do reading tests test reading skills? You may think that’s a question on the order of “Who’s buried in Grant’s tomb?” But if you said yes, the real answer may surprise you.

Pretty Young School Girl Reading A Book

Reading tests–of the standardized variety that kids in public schools take–are certainly supposed to test reading skills. The tests that have come into use in the past few years, which are aligned to the Common Core State Standards, are aiming to test skills like “finding the main idea and key details,” and “making logical inferences from text.”

But what if you can’t understand the reading passages, because they assume a lot of knowledge and vocabulary you don’t have? Would you be able to demonstrate your skill at finding the main idea or making inferences if you couldn’t understand what you’d just read?

That’s the problem that many kids–and especially those from low-income, less educated families–confront when they take reading tests. And that’s a huge reason why test scores for that segment of the student population have remained stubbornly abysmal.

This is a truth that some people–although not the majority of those involved in education–have long acknowledged. Cognitive psychologist Daniel Willingham has declared that reading tests are really just “knowledge tests in disguise.”

Now we have some good, hard evidence to back up that assertion with respect to the two most widely used Common Core-aligned reading tests. These are the tests designed by two consortia, one called Smarter Balanced and the other called PARCC. DC students take PARCC in 3rd through 8th grade, and again in high school, usually in 10th grade.

The evidence comes in the form of an analysis by Ruth Wattenberg, a member of the DC State Education, who studied released questions from the two tests and wrote up her findings for a new-ish organization called Knowledge Matters. That organization is making the case that if we want to teach kids to understand what they read–and prepare them for high school, college, and life–we need to teach them history, science, and the arts beginning in the early elementary grades.

Right now, most elementary school students–and especially those from low-income families, who are the least likely to pick up knowledge about these subjects at home–are getting a steady diet of math and reading. “Reading” usually means a lot of work on foundational skills like phonics–which is important. But it also means hours of practicing supposedly abstract reading comprehension skills like “finding the main idea,” with the theory being that once you’ve perfected that skill on simple texts, you’ll be able to apply it to find the main idea of any text, no matter how complex.

That theory–as many experiments have demonstrated–is actually wrong. While some practice with reading comprehension skills can be helpful, the main determinant of whether you can understand what you’re reading is whether you have the relevant background knowledge and vocabulary. And the best way to expand your background knowledge and vocabulary is to learn about things like history, science, and the arts.

The people who design the PARCC and Smarter Balanced tests actually understand this. Since there’s no way of knowing which students have studied what, the idea is to cover a wide variety of topics, to balance out whatever unfair advantages in background knowledge some kids might have. If one group of kids has studied snowy owls, for example, and there’s a passage on snowy owls on the test, they might ace that. But then the next passage might be on ancient China, which they know nothing about–but some other kids might have studied that.

Of course, these days, students in many elementary schools are unlikely to have studied much of anything aside from reading and math skills. So the wide variety of topics on reading tests privileges the kids who have acquired a fair amount of knowledge from home. And those kids tend to be the wealthier ones.

Wattenberg looked primarily at 3rd-grade questions about “informational text” passages, most of which had something to do with geography or science. She found that even these 3rd-grade passages assumed a lot of knowledge and vocabulary that many eight-year-olds would be unfamiliar with–concepts like the names and general locations of continents and units of measurement like kilograms, as well as vocabulary like “rodent, “intestine,” “crystallizes,” and “Aurora Borealis.”

From interacting with low-income high school students and speaking with some of their teachers, I know there are many high school students who are unfamiliar with these words and concepts. I can only imagine the confusion of a low-income 3rd-grader when confronted by them.

Here’s an example of just one paragraph from one of the reading passages Wattenberg analyzed, with the words she identified as challenging replaced by a nonsense word, blunk. See if you can make sense of it:

In one of the most blunk places in the world, the blunk blunk, a people have survived over a blunk of [sic] years. They are the blunk. For the blunk, the blunk is a place blunk with life. Depending on how far north they live, the blunk find everything from blunk blunk and blunk blunk to blunk blunk. The blunk have blunk themselves to the various blunk they blunk. At one time they were considered to be among the healthiest people in the world. This is no longer the case; the blunk blunk has changed blunk over the past blunk. The arrival of blunk and modern blunk resulted in big changes to the blunk diet and blunk blunk blunk.

Here’s the passage with the words included, for the sake of comparison:

In one of the most remote places in the world, the Canadian Arctic, a people have survived over a thousand of years. [sic] They are the Inuit. For the Inuit, the Arctic is a place teeming with life. Depending on how far north they live, the Inuit find everything from Caribou herds and polar bears to beluga whales. The Inuit have adapted themselves to the various regions they inhabit. At one time they were considered to be among the healthiest people in the world. This is no longer the case; the Inuit lifestyle has changed dramatically over the past decades. The arrival of southerners and modern technology resulted in big changes to the Inuit diet and way of life.

The point is not that we should lower our expectations for what 3rd graders can read and understand. The point is that it’s unrealistic to expect kids to meet those expectations if we haven’t given them—and their teachers—the tools that will enable them to do so.

Wattenberg’s analysis is well worth reading in its entirety. You can find it here.

Come hear the cognitive scientist: why our efforts to help struggling readers haven’t worked

Why is it harder to raise reading scores than math scores for students from low-income families? And why do kids who seem to read well in elementary school then struggle with grade-level text in middle and high school?

Students Standing In Classroom.

For decades, most elementary schools have taught reading as a skill: children have practiced reading comprehension strategies like “finding the main idea” or “making inferences” on simple stories. The theory has been that it doesn’t matter what students are reading, as long as they’re reading something. And in many elementary schools, especially those serving low-income students, the curriculum has been narrowed to “the basics:” reading and math.

But reading comprehension is highly dependent on background knowledge–as Daniel Willingham, Professor of Psychology at the University of Virginia, will explain when he comes to DC on Wednesday, June 2. If students don’t learn about history, science, and the arts in elementary school, they’ll be at a tremendous disadvantage in high school, when they encounter texts that assume a lot of knowledge and vocabulary they don’t have. That’s particularly true for low-income students, who are far less likely to acquire academic knowledge at home. But all students will benefit from a broad, content-rich curriculum–and they’ll also find it more engaging than a narrow focus on reading skills.

Willingham is an accessible and engaging speaker as well as the author of several popular books–and the cognitive scientist behind the column “Ask the Cognitive Scientist,” a recurring feature of American Educator magazine. He has become one of the leading advocates of bringing content and a focus on building knowledge into the elementary school curriculum and
was recently cited in a speech by Secretary of Education John B. King. “We know from decades of research from folks like Daniel Willingham at the University of Virginia that knowledge matters for reading success,” King said. “It is not about reading vs. science and social studies.”

Willingham will speak on Wednesday, June 1, from 6:30 to 8 at McKinley Tech High School, 151 T Street NE. Parking is available behind the school. To RSVP, email Ward 3 State Board of Education representative Ruth Wattenberg at ruth4schools@yahoo.com.

This event is co-sponsored by the DC State Board of Education, SHAPPE (Senior High Alliance of Parents, Principals, and Educators), DC Public Schools, the Public Charter School Board, Coalition for 4 DC Schools and Communities (representing all 8 Ward Education Councils), Washington Teachers Union, DC Language Immersion Project, Office of the State Superintendent of Education, and Deputy Mayor of Education.

I hope to see you there!

Why the Education Secretary wants kids to have dessert WITH their broccoli

In a speech last week, US Education Secretary John B. King offered up a fundamental—if somewhat counterintuitive—insight that academics outside the field of education have agreed on for decades: If you want children to learn to read, don’t just teach them reading.

Students In Class Volunteering For Teacher

Inside the education world, that insight has been largely overlooked. And therein lies the root of many of our problems—in particular, the huge divide between rich and poor students that we call the achievement gap.

“We know that students are better able to read a text when they have had exposure to the knowledge and experiences that are referenced in that text,” King said. “We know from decades of research from folks like Daniel Willingham at the University of Virginia that knowledge matters for reading success. It is not about reading vs. science and social studies.”

In a way, King’s observation might sound obvious. We can all think about what it’s like to read, say, an abstract of an article on cellular biology, if you’re not a cellular biologist—or an account of a cricket match, if you’re unfamiliar with the game. Without the relevant background knowledge and vocabulary, the text is impenetrable.

Low-income kids start at a disadvantage

What’s less obvious is how little background knowledge many low-income children have when they enter school. A text that looks simple to an adult can be as impenetrable to a disadvantaged third-grader as a tract on phenomenology. That’s largely because, as King also mentioned in his speech, beginning at birth, low-income kids are exposed to far fewer words and concepts at home than more affluent kids.

Studies have shown that even highly functional low-income parents speak far less to their children, on average, than affluent parents. And when they do speak to their kids, they’re more likely to issue commands or reprimands than to engage them in back-and-forth conversation. Meanwhile, more affluent parents are continuously immersing their children in exchanges that expand their knowledge of the wider world and develop their linguistic abilities.

Some argue that each style of parenting has its advantages and disadvantages. But the way affluent parents raise their kids equips them to do better in school: by the time they enter kindergarten, the skills and knowledge of the most affluent children far exceed those possessed by their low-income peers.

Family doesn’t have to be destiny, though. Schools can help level the playing field, if they start systematically building low-income kids’ knowledge about subjects like history, science, and the arts from an early age.

But most elementary schools—especially those serving poor kids—have instead devoted hours each day to teaching kids reading comprehension strategies like “finding the main idea” and “making inferences,” and then having students practice the strategies on simple books, often of their own choosing. Meanwhile, subjects like history and science have fallen by the wayside.

King embraces the cognitive science on reading

What’s significant about King’s speech is that it marks the first time a top U.S. official has mentioned the cognitive science on reading comprehension. I was particularly struck by King’s reference to Daniel Willingham, a psychology professor who has devoted years to explaining the science of learning to teachers and the general public in easily understandable terms.

A few months ago, I noted that British government officials had enthusiastically embraced Willingham’s ideas, and I wondered why he—and other American academics who have made similar arguments, including E.D. Hirsch, Jr.—hadn’t received the same kind of reception here.

“Why hasn’t Arne Duncan—or the new acting US Secretary of Education, John King—given these home-grown education theories a similar shout-out?” I wrote in January. I’m not suggesting King got the idea from reading this blog, but it’s certainly gratifying to see my wish become reality.

But here’s the rub—actually, two rubs. The first rub is that King’s point about knowledge and reading comprehension didn’t get wide coverage—the New York Times ran nothing but a brief AP story. And, like all but a few reports of the speech, the AP story didn’t even mention this crucial point. That’s not surprising: the bulk of the speech focused on how history and the arts can spark kids’ interest and motivation and develop them into well-rounded individuals who lead rich, meaningful lives.

Why kids need dessert WITH their broccoli

Those points are valid and important. But here’s why I’m worried about leaving out the link between those subjects and reading comprehension: Even if we got rid of high-stakes reading tests tomorrow (which is unlikely to happen), people would still place a huge emphasis on teaching kids to read, especially in the early grades. That’s understandable: reading is the gateway to just about all other learning.

That’s why teachers, parents, and policymakers need to recognize that teaching history—or science or art—actually IS teaching reading. They need to stop seeing reading as the broccoli that kids have to eat before they can get to dessert—dessert being things like music, art, the American Revolution, Ancient Egypt, snakes, or the human digestive system. These are subjects that even very young kids can get excited about if they’re taught in an engaging way.

If we hold off on giving low-income kids this kind of dessert until middle or high school, it’s too late. Every year, the gap in knowledge between them and their affluent peers widens and gets harder to close. So what King is saying, essentially, is this: if you want kids to get the nutrients out of their broccoli, you have to give them dessert at the same time.

The other rub is this: Britain, like most other developed countries, has a centralized education system. If top officials want kids to learn about history, science, and art in elementary school, they can simply pass legislation to that effect.

In our system, the federal government can only indirectly influence decisions about what and how to teach. The actual power to decide what to teach rests with state and local governments—and often, in a very real sense, with individual schools and teachers.

In recent years the federal government has done a very effective job, intentionally or not, of influencing American elementary schools to do the very thing that King is now warning against: focus on reading and math to the near exclusion of other subjects. Whether the government can help undo that damage remains to be seen. But kudos to John King for beginning to try.