What Do Standardized Tests Actually Test?
A headline in the January 26, 2009, issue of Forbes magazine reads: “Bill Gates: It’s the Teacher, Stupid”
The article that follows says that on a conference call with journalists, “Gates pointed out that experience (as measured by years on the job) and master’s degrees (which carry great weight in teacher hiring) show no bearing on whether someone will be a great teacher or a mediocre one.”
Gates’ opinions are important. He’s done as much as anyone or more to shape current education policy in America, and his focus on teachers — the good ones as miracle workers, and the tenure-protected bad ones as the main cause of poor school performance — has pushed aside interest in and dialogue about other social and institutional factors affecting school performance. He’s spent millions trying to pinpoint what makes a teacher great. He’s reached no firm conclusion, but thinks the great ones are easily identified. They’re the ones who raise scores standardized tests — and to school reformers like Gates, test scores are infallible indicators of quality.
The truth is that teaching—trying to shape minds—is hard, complicated work. Claims that class size, school size, teacher education, and teacher experience make no difference in performance is sufficiently at odds with common sense to require an explanation.
Like most people, Gates believes that learning is a product of teaching. That assumption is the bedrock of traditional schooling. It’s taken for granted by newspaper and magazine editors, syndicated columnists, and talking heads on television. It shapes nearly all commercially produced teaching materials. It’s how schooling is portrayed in movies and on television. It’s why traditionally arranged classroom furniture is in rows facing front, why most teachers talk a lot, assign pages in textbooks, ask questions about what’s been said and read. It’s the conventional wisdom.
Teachers teach, learners learn, and standardized tests monitor how well the process is going. The tests measure a quantity—the amount of information taught, minus the amount not learned or learned and forgotten. Subtraction yields a single, precise number convenient for sorting and labeling kids, teachers, schools, school systems, states, nations.
Simple and straightforward. Right?
There’s a now-familiar ancient Chinese proverb which, loosely translated, says, “Tell me and I’ll forget. Show me and I’ll remember. Involve me and I’ll understand.”
That’s three very different approaches to teaching—telling, showing, and involving. The first two lend themselves to standardized testing. The third one—the only one that really works—doesn’t. It says that what needs to be evaluated are the outcomes of personal experience, and personal experience is very likely to be too individual, too idiosyncratic, too much a product of a teachable moment exploited or created by the teacher, for its outcome to be evaluated by machine-scored standardized test items.
Involved learners don’t just read about plants; they’re outside, identifying, examining, and classifying, the weeds and whatever else is growing around the school. Involved learners aren’t filling out worksheets about geometric principles; they’re determining the height of the school’s flagpole by measuring angles and lengths of shadows.
Teachers doing those kinds of things are usually older, better educated, and more experienced, but high-stakes testing’s single-minded focus on scores has reduced them to simply guessing what’s probably going to be on the test and hammering it to near death. Experiences that create understanding? When test scores can dictate what happens to you, your students, the school’s principal, and the school, understanding runs a distant second to filling in the right bubble on the answer sheet.
It took me about 15 years in the classroom—and a federally funded 1960s “think freshly” initiative—to accept that what mattered most wasn’t what I said but what kids did. When I made that radical switch, I began a search that continues, a search for experience-creating activities (a) so interesting, the teacher can leave the room and nobody notices, (b) so useful, the activity’s relevance is self-evident, (c) so complex, the smartest kid in the class is intellectually challenged, (d) so real-world, perceptions of who’s smartest constantly shift, (e) so theoretically sound, the systemically integrated nature of all knowledge is obvious, (f) so wide-ranging, the activities cover the core curriculum (and much more), (g) so varied, every critical thinking skill is exercised, (h) so scalable, concepts developed on a micro level adequately model macro phenomena, (j) so effective, when the activities themselves are forgotten, their benefits are fixed permanently in memory.
The raw material for creating a near-infinite number of activities that meet those nine criteria isn’t hard to find. It lies within the property boundaries of every school or randomly chosen slice of real life. Finding it is mostly a matter of looking at the too-familiar and the taken-for-granted until it becomes “strange enough” to see.
Modern school reform based on test scores as the main accountability measure — supported by the Business Roundtable; the U.S. Chamber of Commerce; the National Governors Association; the Gates, Broad and Walton Foundations; some big-city mayors, among others—have engineered an educational train wreck. They took over an institution struggling to replace the minimally productive 19th Century idea that learning is a product of teaching with the demonstrably better idea that learning is a product of the activities of learners. Then, instead of asking educators how they could help with the transition, they slammed the door in educators’ faces and wrote standards and tests that have locked the sterile 19th Century view of teaching even more rigidly in place.
For millions of kids, it’s too late to undo the damage they’ve done. But if parents and other concerned citizens make enough noise, the giant, tax-wasting, kid-abusing, craft-and-profession destroying, super-standardizing, multibillion dollar testing juggernaut that’s perpetuating a stupid idea of what it means to educate and be educated, can be stopped.
If that can be made to happen, teachers can pick up where they left off before they were rudely interrupted—trying to figure out how kids learn best.
Still, we will come away from this reform era having learned a couple of useful lessons: One is that no machine can measure the quality of complex, emotion-filtered, experience-based learning. And the second: If you’re testing the wrong thing, there’s no reason to keep score.