A friend in Florida sent this to me in response to the article posted on our Facebook page, “The Loneliness of the Long-Distance Test Scorer.” The author speaks to his time working on the math questions for the National Assessment of Educational Progress (NAEP). This further exposes the politicization of what purports to be “reform” of the public schools. Read on, but don’t get depressed. Just get mad and start speaking up!
This piece really spoke to me! (The Loneliness of the Long-Distance Test Scorer)
I served on the NAEP National Math Committee for 8 years from 1996-2004.
This was a selected group of approximately 30 university and public school
professors/teachers/math advocates whose task was to review, revise,
rewrite, rethink the questions on the 4th, 8th, 12th grade math NAEP
tests, commonly called the “Nation’s Report Card.” These are the questions
that are scored and reported to “advertise” what progress (if any) the
United States is making vis-a-vis world scores.
I swear those 8 years were a third Master’s Degree in public policy!
First, in my experience, during my 8 years on the NAEP committee, all of
the 30 members of the committee favored the concepts of “reform math” of
the 90′s. We had intriguing conversation about specifics but we were all > together on the general direction. Some of the members of this group were
former NCTM elected policy framers and office holders. We weren’t wimps!!
Keeping in mind that NAEP is a government-sponsored testing scenario, we
were required to follow direction that had less and less to do with
education but everything to do with politics. What questions are on NAEP
have more to do with political direction than with educational
understanding, believe me BIG TIME!! By the time I left the committee in
2004 (they really wanted me to stay but I was retiring and two terms was
the norm), I had seen a dramatic change from creative thought/higher level
thinking to many more questions devoted to rote skills. (Question banks
must be allocated among different math topics by a certain % determined by
“others.” Skill question percentages were rising dramatically.)
Another task of the committee was to review/change/discuss the specific
questions that would appear on the next edition of the math NAEP. Of
course, these would have to be scored by a battery of “hired hands” who
had no idea what we were thinking about when we OK’d the questions. So we
looked at every answer that a student could possibly make, analyzed it to
the nth degree, wrote specifics for what was acceptable for “extended” written answers, gave alternate interpretations of those specifics…I
mean, this was boring and menial work but we came up with templates for
scorers to follow. During these sessions EDS people (they had the NAEP
contract) who would do the “training” to teach the potential scorers how
to read/interpret/score the answers would take notes on our every
discussion to be sure they had the proper understanding of every nuance of
every problem and every multi-step score possibility. (BTW, by the end of
my service, open-ended questions were diminishing in both complexity and
need for student explanation. We math people all bemoaned this but it was
a “political” decision. Big surprise!)
Finally, in the last few sessions I attended there was a new group to be
introduced, the Pearson folks!! That company took over the NAEP
stats/interpretation situation in GWB’s first term, I think. They were
ostensibly there to “observe” but I swear it was like having the “Men in
Black” looking over our shoulders! They might be good human beings but
they sure weren’t into social interaction!>
I know I’ve shared some of this experience with you before. I’m that naive
type who goes into an “assignment” (NAEP) thinking it’s all going to be
groovy. It usually takes me a while to figure things out, but I know now
that all of this educational statistical stuff is politically driven.
Our educational policies and “standardized tests” have less to do with
what is important for kids to learn and to assess that learning in an an
intelligent manner (waaay too expensive unless it’s machine scored so
test questions are continually dumbed down…been there, seen that!) than
it does with producing scores and controversies that result from those
scores to the general populace.
Sigh.
So I hear in this article from those folks who have been “trained” by the
very people who sat in the NAEP committee sessions I was a significant
member of and took notes on how to explain our thinking on what the
students we are assessing were thinking. OK, let’s hope the math people
did it better than the Language Arts people did. Still, if this is being
done by hand, it’s an incredibly mind-numbing experience and I have little
faith in the results. If it’s a machine-scored process, it’s an incredibly
mind-numbing experience and I have little faith in the results.
Heaven help us all! The human mind cannot be filtered down to data, no
matter what anyone thinks…and this comes from a math person.
If the current path continues, and it seems like it will, we are so
screwed!
This conclusion of The Loneliness of the Long-Distance Test Scorer should motivate each of us to change standardized test policy in the US:
If scoring is any indication, everyone should be worried about the logic of putting more of our education system in the hands of these for-profit companies, which would love to grow even deeper roots for the commodification of students’ minds. Why would people in their right minds want to leave educational assessment in the hands of poorly trained, overworked, low-paid temps, working for companies interested only in cranking out acceptable numbers and improving their bottom line? Though the odds might seem slim, our collective goal, as students, teachers, parents—and even test scorers—should be to liberate education from this farcical numbers game.
Right on, Tom. When I was researching the WASL a couple of years ago I interviewed test scorers around the U.S. (who were scoring the WASL) and found exactly the same thing. We fought the WASL and won, but the replacement is not significantly better.