New KIPP Study Underestimates Attrition Effects

ksdcitizens.org will occasionally publish the findings of the Think Tank Review Project.  As major think tanks release their studies, often skewed by the ideology of the sponsoring organizations,  the Think Tank Review Project assigns a noted scholar to analyze the study and its findings.  The results are then published and made available to the public.

About the Think Tank Review Project

The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the University of Colorado at Boulder Education and the Public Interest Center (EPIC) and the ASU Education Policy Research Unit (EPRU), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.

EPIC and EPRU collaborate to produce policy briefs in addition to think tank reviews. Our goal is to promote well-informed democratic deliberation about education policy by providing academic as well as non-academic audiences with useful information and high quality analyses.

Here is such a report analyzing a study that received a great deal of media attention regarding KIPP schools.  Kipp is a popular version of charter school that has garnered praise for its success with inner city children.  One criticism of KIPP regards their policy of “skimming” only those most likely to succeed, discounting the dropouts from the program, and reporting only the successes.  This latest study praising KIPP claims the schools have the same retention rate as the public schools the children came from.  Here are the findings of the EPIC/EPRU researcher.

New KIPP Study Underestimates Attrition Effects

June 22, 2010

Comparison considers only half the equation

BOULDER, Colo. and TEMPE, Ariz. (June 22, 2010) – This morning, the Knowledge Is Power Program (KIPP) held a press conference to announce the results of a new study of KIPP middle school students. A key finding of the study is that attrition at KIPP schools is not much different from attrition at comparable conventional public schools. This finding is important because past research about KIPP suggests that selective attrition – struggling students disproportionately leaving, with more successful students staying and then scoring well on tests – may give KIPP a substantial boost.

However, an initial analysis of the report by Professor Gary Miron of Western Michigan University concludes that this initial study report misrepresents the attrition data. According to Miron, “While it may be true that attrition rates for KIPP schools and surrounding districts are similar, there is a big difference: KIPP does not generally fill empty places with the weaker students who are moving from school to school. Traditional public schools must receive all students who wish to attend, so the lower-performing students leaving KIPP schools receive a place in those schools.”

In contrast, Miron explains, “The lower performing, transient students coming from traditional public schools are not given a place in KIPP, since those schools generally only take students in during the initial intake grade, whether this be 5th or 6th grade.”

The KIPP study’s description of attrition only considers half the equation, when comparing KIPP schools to matched traditional public schools. The researchers looked at the attrition rates, which they found to be similar – in the sense of the number of students departing from schools. But they never considered the receiving or intake rate. Even though the researchers agree that the students who are mobile are lower performing, they do not take into account the reality that KIPP schools do not generally receive these students.

Professor Miron conducted his own quick analysis, using the Common Core database, and concluded that there is a 19% drop in enrollment in KIPP schools between grades 6 and 7 and a 24% drop in enrollment between grades 7 and 8. (This analysis only included KIPP schools that had enrollments in all three grades). In comparison, traditional public schools in these grades maintain the same enrollment from year to year.

While Miron’s review questions about the validity of this report’s particular findings, this is solely because of this single problem. In other ways, he found the study to be rigorous and high quality, promising to be even better in subsequent years of the evaluation. Those future reports can, and Miron hopes, will address the questions raised here and also about students retained in grade.

Importantly, Miron is also not saying that the KIPP schools do poorly. Those schools provide about 50% more instructional time and place rigorous demands on students and their families. “We have every reason to believe that KIPP likely does a great job with the low-income students of color who wish to attend and who have relatively supportive parents who can do things like drive them to Saturday school,” Miron says. But he does question whether this is a viable model for larger numbers of students, and he also wonders whether the different departure and receiving policies may make matters worse for students who are left behind or who later leave KIPP schools. How would the KIPP model work if students who cannot handle the rigorous KIPP demands could not move to conventional public schools?

Professor Miron conducted his initial analysis at the request of the Think Tank Review Project, a collaboration of the Education and Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University.

CONTACT: Gary Miron
(269) 599-7965
gary.miron@wmich.edu

Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370
kevin.welner@colorado.edu