Wednesday, March 17, 2010

"Selection Effects" and Charter Schools: A Clarification

A number of skeptics argue that one reason for some charters' success is that they skim some of the best students from traditional public schools.  Matthew Yglesias fundamentally misunderstands this argument (as, I suspect, do many others) in this post on recent research on KIPP when he writes that the authors are "able to look in a rigorous way at whether the high performance of KIPP students relative to demographically similar non-KIPP students is merely the result of some kind of selection effect".

While it's true that some charter schools do attract students who score higher, on average, than their peers, no serious education wonk is arguing that this alone is why the KIPPs of the world have higher test scores (which is not to say that nobody is making this argument).  Indeed, if we look at research on these high-flying charters -- note, I said research and not the popular press -- the statistics cited aren't usually snapshots of how many kids passed a certain test but, rather, longitudinal examinations of the growth of kids' test scores over time.  In this sense, simply having higher achieving kids from the start wouldn't help much -- and could conceivably hurt a school.

So what is meant by "selection effects" then?  Well, when skeptics argue that charters often skim off the best students, they mean best students in a more holistic sense.  If you ask a teacher to identify their best students, they wouldn't just point you to the kids with the highest test scores -- they'd point you to the kids who worked hard, cooperated, asked questions, turned in assignments on time, showed up every day, and generally did what was asked of them.  And having a school full of students in this mold would make teaching easier, hallways quieter, and a school's climate more positive -- all of which would aid student growth.

As far as I know, there hasn't been much research on whether charters do, in fact, recruit and retain kids who are "better students" in this sense (please note that I'm not saying there hasn't been any, only that I'm unaware of it -- and, actually, if you know of some I'd appreciate it if you sent it my way).  But there's plenty of reason to suspect that at least some charters' student bodies might skew in this direction.  Probably the most cited reason is that it takes extra effort for a parent to enroll their kid in a charter school -- making it quite logical to assume that more motivated parents are more likely to fill out the application (of course, maybe the parents' motivation is driven by hatred or their current school or something rather than desire for their kid to excel).  Secondly, there are various indicators that some charters are more likely to give kids the boot, or at least threaten to do so, than are traditional public schools.  For example, I watched one video in which a KIPP principal walks in the first day of school and tells a kid who's not cooperating that if this school isn't for him that he can leave -- that's not something that traditional public schools can really do.

Anyway, the point is this: when people talk about charters benefiting from "selection effects" they're talking about schools enrolling "better students" in the sense that they're more motivated and more cooperative, not that they simply enroll higher-scoring students.  I don't know whether or not charters actually have better students, but it's easy to imagine that a more enthusiastic, better behaved student body would make a school far more productive.


Attorney DC said...

I'd like to add that some of the positive effect (assuming there is one) likely comes from each student being surrounded by OTHER students who are similarly motivated and well-behaved. That is, in a classroom where 30% of the kids really want to learn but 70% of the kids are fooling around and acting disruptively (or even just not doing their homework or participating in group work) the 30% who try to stay focused will struggle. In a school where ONLY motivated, well-behaved, and hard working students attend, the 30% will be surrounded by a class of similar students, who will not impede their academic progress.

As a former teacher, this is the experience I had in teaching classes of "honors" students in a low-income junior high school: The "honors" kids hadn't taken any IQ test and were not uniformly brilliant (they may not even have been labeled "honors" in a wealthier school), but they were the students who historically got A's and B's in school. They generally came to class on time and prepared (e.g., with pencils and notebook), tried to do their homework, and paid attention in class. We were able to cover a lot of material in those classes b/c there were few or no disruptions: All the kids got to class on time, got focused, and got to work.

irasocol said...

"Selection Effects" are the highly troubling fact of the entire charter school movement, but they are magnified in the colonialist KIPP effort (a simple re-creation of the British colonial schools effort to identify good collaborators among oppressed groups).

So, KIPP with the usual charter school barriers plus parental commitment requirements, emphasizes Social Reproduction at all costs - the only children KIPP and friends will help are those predisposed to some level of "white style" parental support.

Attorney DC in his comment emphasizes this point. We are not interested in assisting the most vulnerable, only - to quote Peter Hoeg - "those who might be useful."

It a false egalitarianism which picks out winners based not on the child, but on the parents motivations. And that is the most destructive "Selection Effect" of KIPP.

(Bust as for studies - lets see, selection effect + more time in school + 90% of time devoted to test prep = SURPRISE! slightly higher test scores. Wow. we're all shocked.)

-Ira Socol

Attorney DC said...

Ira Socol: I agree with your analysis; Until people realize that charter schools select (on many different levels) the loud comparisons between traditional public schools and charter schools are disingenous.

Stuart Buck said...

Well, when skeptics argue that charters often skim off the best students, they mean best students in a more holistic sense.

What's the evidence for this? It seems, if anything, more intuitively plausible that charters, like many private schools, face negative selection effects . . . sure, there are the parents who have a brilliant kid that isn't well-served by a particular public school, but many parents might seek an alternative school precisely because their child is struggling.

Traineeship said...

They are meant to be held accountable for both academic results and fiscal practices to several groups, including the sponsor that grants them, the parents who choose them, and the public that funds them.