Monday, May 11, 2009

Sunday Commentary: Sale on Snake Oil at the BrooksStore

David Brooks wrote quite the op-ed on Friday. And I don't mean that in a good way. Having just begun reading Whatever it Takes the day before, I was excited to see him writing about the Harlem Children's Zone. I read the first couple paragraphs and started sending the link to friends I knew were interested in the topic. Then I read the rest of the piece and recoiled in horror. I like David Brooks, I find him to be the most consistently interesting of all the op-ed writers at the Times. But this is not just sloppy journalism, it's flat out irresponsible -- and I will have a hard time taking him seriously in the future if he fails to offer an apology to his readers.

Brooks writes about a recent analysis of test scores of students at the Promise Academy by Will Dobbie and Roland Fryer that found huge gains in achievement -- leading Fryer to write to Brooks that the "study has changed my life as a scientist."

Brooks concludes that the Promise Academy has succeeded in closing the Black-White achievement gap in math and that no-excuses schools can do this by themselves. He once again frames the debate as one between "reformers" who believe that schools can do great things and doubters who believe that they cannot overcome societal influences. He goes on to argue that "The Harlem Children’s Zone results suggest the reformers are right."

Brooks' analysis is so flawed that it's tough to know where to begin. Here are a few of the problems with his conclusions:

1.) The Harlem Children's Zone is an all-encompassing social services organization that provides everything from classes for expectant mothers to deliveries of healthy food for as many people as they can sign up in a 97 block area of Harlem. The idea of the project to is to combat every disadvantage that these residents face in order to help the next generation succeed. The Promise Academy is the charter school that HCZ operates. To claim that positive results that the school achieves are proof that school reform is sufficient independent of societal reform is, unacceptably sloppy if not disingenuous. Brooks apparently missed the sentence in italics on page 5 of the paper: "We cannot, however, disentangle whether communities coupled with high-quality schools drive our results, or whether the high-quality schools alone are enough to do the trick."

2.) Besides the fact that we're not sure exactly what induced the positive results, the claim that they were enough to close the black-white achievement gap is also somewhat dubious. As Aaron Pallas pointed out on Friday, the average student at the Promise Academy still scores significantly below the average White student in NYC.

3.) Even if we assume that the school itself can close the achievement gap, what, exactly, does this prove? Does anybody actually believe that simply evening the test scores of Black and White students is enough? The problem with focusing on the test-score gap as a proxy for inequality is that solving the test-score gap alone does little to solve the larger problem at hand. Sure, it would be a large step in the right direction -- but ultimately test scores aren't what matter. In the long run, what we really care about is whether students in poorer communities graduate from high school, earn college degrees, work in prestigious fields, escape poverty, avoid jail and welfare, and generally live healthy and productive lives. We cannot assume that higher test scores in elementary or middle school are enough to obtain all of these.

4.) Even if we assume that school itself can lead students to accomplish all of these things, we still have another problem: replication. For some reason that I can't quite figure out, we're all in search of the miracle cure in education. When somebody makes it home safely after drinking too much we don't rush to claim that drunk drivers who crash are making excuses or blaming their problems on alcohol. We still understand that driving drunk leads to bad things -- and that the exception doesn't disprove the rule. But every time we see a school with high test scores, a report pops up praising this as the answer to our educational problems. Even if this school walks on water, does it really mean that we've solved all of the problems of society? Each school has an inordinate number of things that make it unique -- the Promise Academy more so than most. Some of these things can be replicated, but some of them cannot. Not every school can have a well-connected and astoundingly wealthy Wall Street tycoon chairing its board of trustees. Not everybody can have Geoffrey Canada as their organization's President. Different schools succeed for different reasons, and it's not at all clear that we can replicate the reasons for success in this case. And it's certainly not clear that a so-called lack of excuses is even the main reason.

5.) While a group of haughty ideologues has successfully branded themselves as "reformers," this does absolutely nothing to prove that they are right. Brooks praises "no excuses" schools for the amazing results they've achieved. But the phrase "no excuses" has been bastardized so that it now means different things to different people. When Teach for America began training their corps members to run "no excuses" classrooms they wanted the future teachers to believe that every student can succeed, but the "no excuses" applied more to the students than to the teachers. TFA wanted teachers not to accept excuses from students -- they wanted them to be "loving hardasses" as Sherman Dorn puts it.

Those who write about "no excuses" schools usually mean that teachers and administrators do not make excuses -- that they do "whatever it takes" in order to ensure that every child succeeds, regardless of the challenges they face at home. And many who praise these schools interpret "no excuses" to mean something akin to "just do it" -- that is, that is that those in schools shut up and put up. Many point to these no excuses schools as proof not only that schools can do great things (side note: who believes they can't?) but that they can do them without wasting more resources.

In the case of the Promise Academy, however, this is certainly not the case. Beyond all of the community resources provided by HCZ, the school has pulled out all of the stops financially as well. When the middle school first opens the superintendent addresses the kids and explains that they will be given everything they need to succeed -- from up to date computers to better food. She says that "our motto is that anything that a private school can pay for, we are going to provide for free" (p. 127). In other words, even if the school does work miracles it doesn't prove that we can solve the problem by just buckling down and working harder -- it means that a school with an incredible amount of resources, a charismatic leader, a board of trustees led by a determined billionaire investor, and located in neighborhood awash in social services provided by the same organization had great success.

Conclusion
We seem to be a long way away from knowing for sure the outcomes of the Harlem Children's Zone or the Promise Academy -- many of the initiatives are only a few years old, and they're meant to effect change over the course of decades. But virtually all results indicate that something positive is taking place. So, by all means, let's celebrate those who have worked hard to achieve these results; by all means, let's take a closer look at what they're doing right; by all means, let's replicate their successes in whatever way we can. But let's avoid mischaracterizing why they've succeeded. To intimate that all schools are capable of fixing all ills of society if only those who work there would stop making excuses and focus is not only false but hinders efforts at honest reform in attempt to prove an ideological pet theory. A dedicated staff is certainly necessary for a school to succeed, but it is nowhere near sufficient. Brooks constructs a straw man to defeat, writing that the people on the wrong side of the argument believe that schools cannot do great things and that this proves they can. But none of the people with whom he's arguing believe that schools cannot do great things -- they believe that schools, given their current level of resources, cannot solve all of society's ills alone. And even if the Promise Charter school has, in fact, worked a miracle, it has done absolutely nothing to disprove this. In other words, don't buy what Brooks is selling.

12 comments:

Rachel said...

This is a bad habit David Brooks has -- take an interesting topic, toss out a few interesting ideas, and then hand wave with anecdotes. His piece a week or so ago on "genius" was similarly bad.

And my sense is that for all the interesting people he talks to, and places he visits, his hand-waving tends to get him back to his ideological preconceptions.

When my mother and I are talking politics or education and start getting fuzzy, we say, as a caveat "I'm just David-Brooksing now..."

Nancy Flanagan said...

David Brooks is the poster child for what happens when low-information columnists write about things they've only seen or experienced from afar. Even "Bobos in Paradise" was shallow and too fond of its own snarky prose to make points effectively, and I'm sure Brooks knows vastly more about the upper middle class suburban dweller than the urban school.

Brilliant post, Corey. You covered all the bases in the story convincingly, and added several new angles.

bradley stone said...

Interesting. I'd always assumed that "no excuses" referred more to the kids than to the teachers, school, etc.; that no excuses would be accepted from the kids regarding lack of work, effort, and so on. I've always thought that that type of model, even given the difficulty of creating a school environment in which kids are able to step up to that high standard, would potentially be hugely successful. I do believe that kids'll live up to expectations, but only if those expectations are school-wide, consistent, and consistently enforced. It seems that this is what schools like the one Brooks discusses are doing and having a good deal of success with. Of course there are likely other factors, but might this not be a bottom-line cause of success? Expect a lot from kids school-wide, then give 'em the time and the resources to achieve a lot. Not that Brooks is dead-on or anything, and not that there's any such thing as a magic bullet, but isn't it just common sense that this type of school would foster higher student achievement?

Stuart Buck said...

To claim that positive results that the school achieves are proof that school reform is sufficient independent of societal reform is, unacceptably sloppy if not disingenuous. Brooks apparently missed the sentence in italics on page 5 of the paper: "We cannot, however, disentangle whether communities coupled with high-quality schools drive our results, or whether the high-quality schools alone are enough to do the trick."But if you look at pages 21-22, Fryer/Dobbie make it rather clear that they don't see the community programs as important here, and it seems fair for Brooks -- in an op-ed with limited space -- to leave that point to the side.

Corey Bunje Bower said...

bradley: Yes, it makes sense that a more no-nonsense approach would work better. What I object to is Brooks deciding that this is the only reason for the success of Promise Academy when there are a million other things that are different about the school.

Stuart: Yes, the authors are guessing that it was the school that caused most of the positive results -- but the responsible thing to do is say that you can't be sure about this. The researchers made sure to do this -- in italics, before presenting their results -- Brooks did not.

Imagine that you had been accused of murder and an analysis was conducted of the crime, and the results were written up. The authors write that some limited evidence indicates that you might have been the one who committed it, but they can't be sure. And when the newspaper reports on these results it simply writes "Stuart Buck is guilty." Is that an acceptable shorthand?

Rachel said...

I thought that the whole idea behind the Harlem Children's Zone was that schools alone weren't enough...

Corey Bunje Bower said...

That's certainly how Paul Tough describes it in his book. Which is one reason I find it odd that David Brooks is citing the school's success on state tests as proof that schools can do it on their own as long as they refrain from making excuses.

Stuart Buck said...

I'm not sure I like that analogy for some reason . . . . :)

The researchers made sure to do this -- in italics, before presenting their results -- Brooks did not.True, but again, given the later discussion on pages 21-22, I don't see how Brooks is guilty of being "disingenuous" or selling "snake oil." This certainly wouldn't be the first time that an op-ed columnist with 650 words to play with discussed a new study without mentioning every single nuance. (And are you really sure you want to accuse Fryer of providing Brooks with the snake oil? Fryer emailed Brooks, remember, with high praise about the life-changing findings here.)

Corey Bunje Bower said...

He doesn't have to provide every single nuance, just avoid overstating his position. Besides, even if it was school and not the neighborhood, implying that the "no excuses" strategy of the school is the sole reason for its success is preposterous. By my calculation, the school spends an extra 5K per student above and beyond district funding, the average class size last year in 8th grade was 18 students, they have an 11 month school year and extended day, they provide extensive after-school tutoring that I'd guess (but I'm not sure) is above and beyond that extra 5K, they have extra support staff to pull kids out for one-on-one help, they provide higher-quality breakfast and lunch than do most schools, and the list goes on.

And Brooks is going to suggest that the reason for the school's success, and what should be replicated, is the attitude of the teachers there? On what grounds? That wasn't even part of the Dobbie/Fryer study. It's bad enough that he's discounting neighborhood effects (and, by the way, the results of the third graders, many of whom went through the baby college and pre-school programs, are more impressive than those of the eighth graders), but it's unacceptable that he could interpret the results to mean that all we need is school personnel who don't make excuses to close the achievement gap.

Stuart Buck said...

Brooks says this:

Reformers, on the other hand, have argued that school-based approaches can produce big results. The Harlem Children’s Zone results suggest the reformers are right. The Promise Academy does provide health and psychological services, but it helps kids who aren’t even involved in the other programs the organization offers.By my reading, that's a pretty accurate, not "disingenuous," representation of what the Fryer/Dobbie study says on that point.

mazenko said...

Corey,

This is a fantastic analysis of misreading Brooks' style or intentions. I enjoy Brooks quite a bit, though I take his comments for what they are worth in 650 words. Hopefully, he generates some discussion, and more Geoffery Canada's rise to the challenge.

Granted, the potential problem is the myopic critics of public education who do as you say, and Brooks implies, and simply conclude that all schools who are successful just aren't trying hard enough.

Though they would do that with or without Brooks. I still like that Brooks shines a spotlight, and then in discussion, society can work toward taking that discussion to appropriate depth.

Again, great analysis

Abagale said...

Hi,

We have just added your latest post "Thoughts on Education Policy: Sunday Commentary: Sale on Snake Oil at the BrooksStore" to our Directory of Pets . You can check the inclusion of the post here . We are delighted to invite you to submit all your future posts to the directory and get a huge base of visitors to your website.


Warm Regards

Petgarden.info Team

http://www.petgarden.info