Education Accountability We Can Believe In
Michelle Obama is very upset.
According to reports from CNN, the First Lady finds it’s “unacceptable” that Republicans lawmakers in the House are pushing to allow schools to opt out of nutrition regulations that were part of a 2010 law that established new requirements for the country’s free or reduced-price lunch program. The new regulations call for these midday meals to have less sodium laced junk food and more healthful fare like fresh fruit and vegetables.
Apparently, the First Lady considers the new standards as a “hallmark” of her campaign against childhood obesity. Her logic is hard to deny. For sure, unhealthy weight gains in children are linked to what they eat, and federal tax dollars should not promote unhealthy eating. Noting that one in three kids are obese, the first lady emphasized that, the report quotes, “folks in Washington should be on our side.”
Surely, here is a fight Democrats can understand: Upholding a federal effort to make sure the nation’s money is spent on positive inputs for our children, so there will eventually be positive effects on their lives down the road. The program is a taxpayer-funded endeavor that feeds “more than 30 million children,” according to the reporter. Eligibility is based on household income.
Of course, government policy can’t make students eat the more nutritious food, can’t control what students eat outside of school, and won’t ensure parents reinforce healthful eating in the home. But at least everyone should agree it’s foolish to wring our hands over children’s rising obesity levels, and then do nothing to address the actual causes.
If that logic seems sensible to you then ask yourself why it isn’t being applied to the rest of our education policy.
For years, education policy at nearly every level has been obsessed with an outcomes-only focus – namely, scores on standardized tests – with less and less emphasis placed on the inputs into our children’s schooling. As the fallacious goals of No Child Left Behind – with its impossible target of 100% proficiency by 2014 – morphed into Race to the Top’s insistence on tying student tests scores to, well, just about everything, governments cut back education funding, narrowed curricula, and constricted the autonomy of schools and teachers to take bold initiatives due to the risks of lowering test scores.
The results have been not only detrimental to children and schools, they haven’t produced much in the way of better outcomes, as the most recent showing on the National Assessment of Education Progress have shown.
Nevertheless, state governments, at the federal government’s urging, are rolling out more new and unproven outcome-based accountability systems – including new teacher evaluation systems tied significantly to student test scores – while continuing to neglect what we feed into our children’s education livelihoods.
Surely it’s time for alternatives to this mindless direction, and fortunately, such alternatives are being proposed from multiple sources. But are leaders paying attention?
Metric Absurdities In Teacher Evaluations
Without doubt, the most nonsensical output-obsession consuming the nation’s schools is the insistence on basing teacher evaluations on student scores on standardized tests – even basing teachers’ performance ratings on students who they do not teach.
Education reporter for The Washington Post Lindsay Layton recently reviewed new research studies on the practice of evaluating teachers based partly on student test scores and found that these studies generally “cast doubt on whether it is possible for states to use empirical data in identifying good and bad teachers.”
Highlighting a recent study published in a peer-reviewed journal of the American Educational Research Association as well as a much-publicized finding from the American Statistical Association, Layton noted, “Thirty-five states and the District of Columbia require student achievement to be a ‘significant’ or the ‘most significant’ factor in teacher evaluations,” yet “teachers account for a maximum of about 14 percent of a student’s test score, with other factors responsible for the rest.”
Further, “Researchers found that some teachers who were well-regarded based on student surveys, classroom observances by principals and other indicators of quality had students who scored poorly on tests. The opposite also was true.”
Layton quoted one of the researchers, Morgan S. Polikoff, an assistant professor of education at the Rossier School of Education at the University of Southern California, who said, “These state tests and these measures of evaluating teachers don’t really seem to be associated with the things we think of as defining good teaching,”
Amplifying Layton’s reporting at her blog for The Washington Post, Valerie Strauss noted, “while there are economists” who still insist a test-based algorithm can measure teacher effectiveness adequately, “testing experts, academics, and other economists say that more than abundant evidence shows that it doesn’t, and that reformers should stop trying to evaluate teachers and principals with unreliable and invalid measurement tools.” Strauss directed our attention to a list of 70 articles and reports compiled by education professor Audrey Amrein-Beardsley, and published on her blog, VAMboozled!, that call into question this approach.
Despite the misgivings about test-based teacher evaluations, school governance at all levels is ramping up their implementations, and states are moving to cement these assessments deeper into their systems by linking them to teachers’ job security. As reported by NBC News, “A growing number of states are using controversial teacher evaluations to determine which teachers earn and hold onto tenure … Sixteen states have now mandated that the results of the evaluations be used in making tenure decisions, a jump from 10 states in 2011.”
Making matters even worse, states are applying this faulty way of judging teacher performance to principals too. As Education Week recently reported, perhaps as many as 36 states have revised school leader assessments to “require that a percentage of a principal’s evaluation include student performance or growth. The amount ranges, for example, from 20 percent in Delaware to 50 percent of the overall score in states such as Georgia and Ohio.”
Again, the push for this has come from the federal Race to the Top grant competition that required student growth to be considered as a “significant” factor in evaluating principals, even though there was “a dearth of valid and reliable evaluation methods, and little emphasis on training for the evaluators.”
When The Post’s Strauss asked the Department of Education to comment on new research revealing the problems with test-based evaluations, she noted, in another blog post, that the Department’s response appears to be an unwavering support for the current approach.
The Outcome Fallacy
It doesn’t appear that any tinkering around the edges will fix the problems with test-based accountability systems, including teacher evaluations. That’s because the focus is wrong from the outset.
Just as it would be absurd for a federal policy aimed at addressing obesity levels in children to repeatedly weigh the kids and mandate they lose pounds, a focus on repeatedly testing students and coercing educators – through evaluations – to raise the scores is off base. At some point a regard for the actual inputs that cause the problem have to be considered.
Writing at his personal blog, classroom teacher and teacher coach Ben Spielberg explained why a focus on outcomes alone is not going to get the accountability we want.
Drawing from numerous examples from a range of activities – from playing poker to getting into college – Spielberg found, “The more factors in-between our actions and the desired outcome, the less predictive power the outcome can give us.” So obsessing over outcomes like student test scores overlooks all the mitigating factors in between what students bring to the classroom and how well they do on tests.
Spielberg noted, “Research on both student and teacher incentives suggests that rewards and consequences based on outcomes don’t work. When we use student outcome data to assign credit or blame to educators, we may close good schools, demoralize and dismiss good teachers, and ultimately undermine the likelihood of achieving the student outcomes we want.
He concluded, “Better policy would focus on school and teacher inputs,” including use of “best teaching practices.”
Education research and policy expert Matt DiCarlo came to a somewhat similar conclusion at the blog for the Albert Shanker Institute. DiCarlo compared valid assessment practices in healthcare to education and argued, “As is the case with student performance on tests, patients’ health outcomes vary widely, but not all of that variation can be attributed to differences in the performance of health care providers. Many factors that are largely outside of providers’ control, such as patients’ behavior and circumstances, also affect outcomes. So, in this kind of formal accountability system, we should not be setting expectations for patients’ health outcomes per se. Rather, we should be setting expectations for institutions’ contributions to those outcomes.”
Evaluating schools and teachers based on the varying inputs they bring to the education process, rather them holding accountable to rigid outcome measures, DiCarlo maintained, “is not ‘setting different expectations’ in the sense of tolerating low performance.” It’s really a more accurate way to look at “true performance.”
Time For A More Authentic Accountability
Most recently, writing at The Huffington Post, education professor Linda Darling-Hammond joined with the president of the American Federation of Teachers Randi Weingarten to propose “a new accountability in American education.”
Although their argument was within the context of implementing new academic standards known as the Common Core, their conclusions were relevant to just about any proposal for a more positive way forward in education policy – namely, that any new innovation is going to fail as long as we adhere to the “old accountability system.”
They called for the current “test-and-punish approach” to be “replaced by a support-and-improve model.” (emphasis original)
Their preference is for accountability to pivot from test score outcomes to the inputs school and teachers can bring to our students’ lives, including “curriculum, teaching, and assessment focused on meaningful learning … adequate resources” and ensuring teachers and school leaders have the opportunity to “develop the knowledge and skills they need to teach much more challenging content in much more effective ways.”
They compared policy mandates in New York that are “stuck in a narrow test-based accountability system adopted under NCLB and reinforced by federal Race to the Top rules” to what’s being enacted in California “that directly addresses what parents and students want from their schools: Good teaching focused on productive learning that is supported with the right resources.”
They concluded the California approach is one that is “more likely to produce a truly accountable educational system – one that ensures all students experience engaging learning in supportive schools that help them pave a path to a productive future, not just another test.”
Politicians and policy makers fearful of proposing new directions for education accountability should understand that an approach similar to what Darling-Hammond and Weingarten propose makes sense to most people. Research shows that parents rarely rely on test scores and other types of outcome data to determine their school preferences. What they tend to rely on are “word of mouth” and what influential people tell them.
Unfortunately, most politicians and policy leaders today devote most of their rhetoric to saying bad things about public schools and calling for measures like “choice” and vouchers that actually eliminate layers of accountability rather than focus accountability on what matters.
Rather than perpetuating an “old system” increasingly taking schools down the wrong path, education accountability we can believe in would call for a healthier menu of inputs to nourish students’ learning.
You have to wonder, why aren’t the First Lady and the rest of the Obama administration fighting for that?