“Test data can be precise, but not accurate.” - Kennewick Superintendent Dave Bond. (see comments)
Kennewick mandates standardized testing for all students two or three times each year using the MAP (Measures of Academic Progress) which is a product of NWEA (Northwest Evaluation Association). Reported scores are called RIT scores.
When student RIT scores were available in my classroom, I was often puzzled by results which did not align with my classroom experience. Some students who had worked diligently through the school year would show little or no growth while other students who had performed poorly in class would show a years’ growth or more.
After hearing KSD Superintendent Dave Bond explain problems with the reliability of NWEA’s MAP test I began to understand these anomalies. (see http://ksdcitizens.org/2011/01/28/ksd-school-board-report-by-tom-staly-5/ “Raise the Third Grade Reading Goal…”)
This should concern Kennewick parents and students because KSD places confidence in the MAP testing program to make decisions about students and the classes they are offered.
Others propose using test scores to evaluate teacher performance.
Bond considers the state tests (MSP and HSPE) to be unreliable (see http://ksdcitizens.org/2010/09/24/school-board-report-by-tom-staly-2/ Test Results and http://ksdcitizens.org/2010/11/18/ksd-secondary-school-report/ bullet point 6) so places emphasis on the MAP which appears to provide more consistent results. Given the cost of testing and the use of results, it is important to understand the limitations of this testing program.
Are school board members and candidates who support student assessment aware of the issues related to standardized testing? Do Kennewick families understand the meaning of test scores ?
This is the first in a series of articles by Karl F. Kuhn, an engineer turned math teacher. Mr. Kuhn explores in greater detail the problems with MAP testing. These articles are published with permission.
Click on the link below to read the article:
http://conceptualmath.org/misc/testshort.htm
For other articles in this series follow the links below:
Go to this link for part 2 of this series:
http://conceptualmath.org/misc/MAPtest.htm
Go to this link for part 3 of this series:
http://conceptualmath.org/misc/testsupport.htm
Go to this link for part 4 of this series:
http://conceptualmath.org/misc/testadmin.htm
Go to this link for part 5 of this series:
http://conceptualmath.org/philo/weak_acad.htm
Go to this link for part 6 of this series:
http://conceptualmath.org/misc/testchoice.htm
Go to this link for part 7 of this series:
http://conceptualmath.org/philo/cheating.htm
Go to this link for part 8 of this series:
http://conceptualmath.org/misc/st_center.html
Go to this link for part 9 of this series:
http://conceptualmath.org/misc/ed_regs1.htm
In an email I received Tuesday, November 1, 2011, Superintendent Bond noted that this comment which I heard him speak during a school board meeting was actually a quote he heard and had attributed to Charlotte Danielson, speaker at a Teacher Principal Evaluation Project meeting attended by the TPEP Steering Committee.
Mr. Bond stated, “Ms. Danielson used the quote in the context of using students data as part of the teacher evaluation process. In any case, I do not want to be given credit for someone else’s quote.”
The failure to properly attribute this quote to Charlotte Danielson is mine.
I found a study done by the Mid-Atlantic Education Laboratory regarding the predictive validity of the MAP, “The predictive validity of selected benchmark assessments used in the Mid-Atlantic Region,” November 2007. Here is a definition of predictive validity: “The ability of one assessment tool to predict future performance either in some activity (a job, for example) or on another assessment of the same construct”. This report examines the availability and quality of predictive validity data for a selection of benchmark assessments.
And the results for MAP…drum roll please……Table 4 indicates that although the MAP scores are sufficiently precise overall and are at the cutpoints of interest, and criterion measures with adequate distributions across grade levels were used in the research studies, these studies did not provide evidence of predictive validity.
In other words, the MAP does not provide the data the district is counting on to place children in doubles or even to provide real information to guide instruction. Of course it only costs about $150,000 a year and takes a bit of time away from actual instruction, but we probably have a surplus of both. (LOL)