In her Huffington Post blog, Sabrina Stevens Shupe, Denver teacher and education activist, describes a conversation she had recently with a wannabe “accountability maven” over a beer in a Denver pub. The WAM uses the usual arguments that the schools are in terrible shape and if we could just get rid of the ineffective teachers all would be solved. He advocated for the currently popular use of student bubble-in test scores to identify those who need to be eliminated.
Sabrina then introduced an analogy so powerful that all of us who advocate for local control of schools and a return to sanity in the drill-test-punish environment of current school policy formation should be using it in our conversations and letters. Here is what she said:
“Let me ask you a question. Pretend you’re a doctor. If we took my temperature, and the thermometer said it was a little over 100 degrees, what would you know about me?”
“I’d know you were sick,” he responds, his face a mixture of “what’s this about?” and “What a stupid question!”
“Wrong. You’d know my temperature was different from the expected 98.6 degrees,” I say, his face dropping at the sound of “wrong.” I continue. “There could be many reasons for that. I could be ovulating (his face goes completely pale), I could be pregnant, I could have just finished a strenuous workout. Maybe I just drank a cup of hot tea before I put the thermometer in my mouth. Maybe the thermometer’s broken, and my temperature really is 98 degrees.”
“Or maybe you’re getting sick!” he interjects.
“Yes, or I might be sick. But the number alone can’t tell you that. You need a lot more information before you can come to an accurate conclusion.”
His face looks a little confused now, and annoyed. “What does this have to do with anything?”
“Well, a thermometer is a direct way to measure temperature. Yet even then, you don’t know everything you need to know in order to figure out what to do about it. You’d have to ask me about other symptoms, look for other information.”
He still looks lost. “A moment ago, you quoted bunch of achievement Data, pulled from measures far less direct than the thermometer. You said those statistics are how you know that schools are failing, and that teachers and schools need to be more accountable.” His face twists as I ask, “How can you be so sure you know what’s wrong?”
He sputters, incredulous that I would doubt him or the sacred Data. “Because…” He looks up, searching for the words. My gut tells me he’s never felt it necessary to say, “I don’t know,” or “I never thought of that…” I finish my glass, thinking how nice it must be, to live in a world where the way you think, the way you see things, are constantly validated. I’m sure he’s never stopped to ask what those numbers mean, where they come from, who decides, or what they can’t tell him.
He never answers my question, saying we’ll probably have to “agree to disagree on the testing thing.” I’m not comfortable with that. “If you can’t even answer my question, how can you be comfortable with teachers losing their jobs, or schools closing, on the basis of that information? These are people’s lives, you know.”
I actually think it is even worse than what is described here. In Sabrina’s example she suggests that the person reading the thermometer is a doctor who is at least expert at reading the temperature and generating assumptions. In the case of testing, the people reading the scores and making policy based on the results have no inkling of (nor do they care) how the tests were developed, how valid or reliable they are for use in policy development, the conditions under which they were given, or the status of the student taking the test. Would you ask your garage mechanic to take your temperature and prescribe medication? I think not, even if he/she was willing and eager to do so.
I came back to revisit this article and share a related animation:
http://www.schooltube.com/video/b37a6b38fea2580c67f2/Collaborative-Planning