Saturday, May 23, 2009

Today's Random Thoughts

-The next part of the series on what we should learn from Promise Academy should be out tomorrow night. I was out of town at a workshop most of the week and haven't had a chance to finish it yet.

-Why is there an inverse correlation between the price of a hotel room and the availability of internet? All of the budget and mid-priced hotels offer free wireless, but all of the nicer hotels charge at least $10/day.

-I'm still in shock about the fact that a number of high schools in the South still have separate proms for Black and White students (read about it here). The proms are paid for privately, but still. One White student admits that it's "awkward," but says that “It’s how it’s always been. It’s just a tradition.” Two which a Black student responds: “You’re 18 years old! You’re old enough to smoke, drive, do whatever else you want to. Why aren’t you able to step up and say, ‘I want to have my senior prom with the people I’m graduating with?’ ”

Monday, May 18, 2009

What Should We Learn from Promise Academy? Part 1

Note: In lieu of a Sunday Commentary, I will be running a multi-part discussion of the recent controversy surrounding the Promise Academy. What follows in part 1.

Last week, I wrote that David Brooks' interpretation of the results reported from Harlem's Promise Academy was "flat-out irresponsible." But I didn't discuss much exactly what we should learn from the results of Dobbie and Fryer's working paper.

Before I begin, I want to reiterate what Aaron Pallas wrote last week -- that it's too early to tell whether results from Promise Academy truly do show some sort of "miracle." Though results on the NY state test seem fantastic for 8th grade students in math, they're significantly less so (though still quite good) in reading and in 7th grade math. And the results on the Iowa Test of Basic Skills are substantially less impressive still. Are good things happening at Promise Academy? Probably? Are miracles being worked? We don't know yet.

But let's assume for a second that something very good is, indeed, happening at Promise Academy. If this is the case, it stands to reason that we might want to learn from what they're doing and maybe even replicate these results. So let's take a look at a few aspects of Promise Academy as described in Paul Tough's book and how they relate to Dobbie and Fryer's analysis and see if we can figure out what might be leading to their success. Here

high rates of spending
By my calculation, the Promise Academy spent $18,073 per pupil in 2006-07 and $15,330 per pupil in 2007-08. I e-mailed the Harlem Children's Zone to see what all this entails, but have yet to receive a response. My best guess is that this does not include the after-school and Saturday tutoring programs and possibly some other programs as well. As Ken Hirsch reported, the school has raised about an additional $5,000 per student each of the past two years.

Dobbie and Fryer write that "The schools provide free medical, dental and mental-health services (students are screened upon entry and receive regular check-ups), student incentives for achievement (money, trips to France, e.g.), high-quality, nutritious, cafeteria meals, support for parents in the form of food baskets, meals, bus fare, and so forth, and less tangible benefits such as the support of a committed staff" (p. 7).

extended school day/year
Students start arriving at 7am and many don't leave until dinner time, not to mention that many come in on Saturdays as well. The school runs 11 months of the year. I've read different estimates that students spend between 50 and 100% more time in school than does the average student.

integration with the community

The Promise Academy is not only part of a designated 97 block area known as the Harlem Children's Zone, but Geoffrey Canada's stated goal is for all of the programs in the area to lead to a type of positive contamination across the neighborhood. In this way, he wants the neighborhood to influence the school and vice-versa.

Tough writes that "the motivational strategies used by schools like KIPP's, Canada knew, often had the effect of establishing a separation between the KIPP kids and the other kids in the low-income areas where they lived . . . KIPP students often became isolated from their community . . . And this situation -- a blighted neighborhood producing a select group of high achieving kids who manage to accomplish great things and succeed beyond their peers -- was exactly the one Canada was trying to avoid when he set up the Harlem Children's Zone. If Canada's model was one of contamination, in which positive ideas and practices spread within a family and throughout a neighborhood, the KIPP model sometimes seemed by contrast to be one of quarantine, walling off the most promising kids from a sick neighborhood's contagion" (p. 162-163).

small class sizes
According to the school's report card, class sizes in 8th grade have averaged around 18 students each of the last two years. The average class size appears to be about 27 students per class for 8th grade general education classes (here's all the data on class sizes).

focus on test-prep

Tough writes that "The students who were furthest behind had been assigned to remedial classes three mornings a week, from 7:00 to 7:45 A.M. Then, for more advanced students . . . there was a separate class on Saturday mornings, from 9:00 to 11:00 A.M." (p. 137).

This actually only describes the first year of operation -- test-prep became increasingly intense with each passing year. He writes that while test-prep was intense for a couple months in the spring the first year that it was running full-speed by mid-September in the second year.

Some charter schools find ways to get rid of kids -- for example, by suggesting that they will fit in better elsewhere or threatening to hold them back if they remain. I think it's fair to ask about the expulsion tactics of any charter school before comparing its results to those of traditional public schools. In the case of Promise Academy, the record is decidedly mixed.

The book emphasizes time and time again that Canada very much wants to educate every kid in the neighborhood, especially those who are the most resistant or the lowest achieving -- indeed, he seems to be the only one at times who doesn't argue in favor of getting rid of a group of "bad apples" the principal thinks is ruining the school.

At the same time, one of these "bad apples" is expelled three weeks after the group receives a stern talking-to (p. 183). Perhaps more importantly, the notion that kids who don't want to behave can choose to go to their neighborhood instead if they don't want to put forth the effort, or will be expelled if they don't cooperate, is definitely pressed at points by the staff. At the beginning of the third year of school (fall 2006) the new principal tells the students on the first day of school that "if you find by the end of this week that you're not prepared to do what we're asking you to do, it's very important that you let someone know so we can make other arrangements for you, because we have a very long waiting list of students who would love to be here at the Promise Academy." And the new Dean follows that up by saying "If you would rather go to public school, that is really your option . . . if you decide to stay, you better recognize: This is not the place you once thought it was . . . we are going to the top this year, with or without you" (pp. 178-179). And perhaps most importantly, they essentially expel the entire 8th grade class in the spring of 2007.

While the cohort sizes dwindle noticeably (the first two go from about 100 in sixth grade to 68 and 81 by eigth grade), it's unclear how much of this attrition is voluntary. Dobbie and Fryer argue that student stability is about equal to what it is at other schools. Perhaps the bigger factor is that students who leave don't seem to be replaced by new students.

a no-nonense approach to schooling
Particularly with the installation of a new principal in year 3, the school seems to make a concerted effort to run a tight ship -- especially given that all involved seem to agree that discipline problems were the largest hindrance in the first two years. Tough describes some tough rhetoric from the Principal and Dean but doesn't go into detail regarding the extent to which this strategy permeates the daily workings of the school.

personal connections
It seems only natural that a school with dedicated staff working long hours with small groups of students could achieve deeper levels of personal connections than the average school. Consider this passage in Tough's book:

"'Why should I care about this test?' [the child] demanded. 'No one cares how I do on this test. I don't care, either.' 'But I care how you do,' [the teacher] replied. And with those words, tears sprang to the boy's eyes and started running down his face. 'Why do you care?' he asked. 'Because this is your future, and I care very deeply about you.' It was the X factor, the magic ingredient that could outweigh all the careful calculations behind Promise Academy's stratgey for success: on top of the hours and hours of cognitive training, what made the difference in many students' lives was a personal connection that was impossible to measure and difficult to replicate" (p. 186).

Coming later this week: the conclusion of this list, some analysis of the Dobbie/Fryer paper, discussion of what others have written on the topic, and some thoughts to tie everything together.

Friday, May 15, 2009

Today's Random Thoughts

-Fascinating article on self-control in the New Yorker (hat tip: Alexander Russo). I still maintain that self-control is one of two major differences I noticed between successful and unsuccessful students while I was teaching. Of course, figuring out how to teach self-control has to be a lot harder than simply figuring out that kids with more of it do better.

-I'm going to hold off on more comments on the David Brooks/Promise Academy/Harlem Children's Zone/No Excuses debate until Sunday. A lot to think through on this.

-Was the "Obama Effect" real? A new study profiled in Newsweek says it might not have been (hat tip: GothamSchools). It should be noted that the new study was done only with only 119 pre-med students taking an MCAT section, and before the November election (but after the Democratic Convention). The authors plan to re-try the experiment soon.

-At some point a couple of weeks ago or so that chapter I helped write on supplemental educational services finally got published (ch. 33). Considering that I wrote the first two drafts in the spring of 2007, it seems like that was long overdue. I'm have distinctly mixed feelings about the way the final version turned out, but I guess you have to start somewhere. Considering the book is $295, I'm not holding my breath waiting for the publisher to send me a copy.

Wednesday, May 13, 2009

Today's Random Thoughts

-Can one "bad apple" ruin an entire classroom? Any teacher will tell you that they can. Robert Pondiscio reports on a pair of economists whose findings agree with this observation. I've speculated before about the potentially positive effects of expulsion, and this does little to change my mind.

-I'm not the only person flabbergasted by Brooks' piece on Friday. Claus Van Zastrow had a similar reaction ("What?!?") to the one I did. Diane Ravitch also thinks that Brooks learned the wrong lessons from the Promise Academy's success. Robert Pondiscio thinks Brooks should take some time to read Paul Tough's book that he recommended his readers check out. And a number of readers wrote letters to the editor that weren't exactly glowing.

-Speaking of Brooks' suggested reading, let me point out that I do agree with him that both Sweating the Small Stuff (my review here) and Whatever it Takes (I'll have some more thoughts on it in the next week or so) are both worthwhile reads. It's worth nothing that Brooks could've written nearly the exact column that he did referring only the schools in the former book and he wouldn't have been too out of line -- but I still can't get over his reaction to the success of the Promise Academy. I'd love to hear the reaction of Paul Tough and Geoffrey Canada to that column as well. And, while I'm thinking of it, Fordham but a free copy of Whitman's book online earlier this year -- I've lost the link, but I have the pdf if anybody is interested.

Monday, May 11, 2009

Today's Random Thoughts

-I've had another 24 hours to think about, and I still think that Brooks' op-ed was sloppy and irresponsible. And I still think that he owes his readers an apology. Actually, the more I think about it the more problems I see with it. To imply that the only important part of the Promise Academy's program is its "no excuses" approach is simply ridiculous. I'm going to take some time to comb back through both Tough's book and Dobbie and Fryer's working paper and will have some more thoughts on this on Wednesday or Thursday.

-I'm also annoyed at the generally sloppy reactions of many to the phasing out of the D.C. voucher program. I can see an argument for maintaining it, but the notion that the program has been proven to work is not one. Everyone needs to understand that the evaluation of the program found found decidedly mixed results. The normally reliable blog on American politics run by the Economist stumbled over this. Kevin Carey, meanwhile, raises a different and interesting argument -- that the voucher program was never going to grow large enough to transform education in the district.

-I always wondered what would happen to alt cert programs if the teacher shortage in high-poverty schools ended. NYC isn't looking to hire many people this year, and it seems much more willing to discourage traditionally certified applicants than it does TFA members or NYC Teaching Fellows. Meanwhile, would people lay off the teachers in the teaching reserve? Yes, some of them shouldn't be teaching anymore but a lot of them are there through no fault of their own. Encouraging principals to hire more of these teachers is long overdue.

-I have a backlog of posts I want to write, but my yardwork is coming along nicely. With a little help (ok, a lot of help) from friends, family, and contractors I expect my yard to look quite nice by the end of the month. I'll try to find time for some interesting and thoughtful posts to find during downtime.

Sunday Commentary: Sale on Snake Oil at the BrooksStore

David Brooks wrote quite the op-ed on Friday. And I don't mean that in a good way. Having just begun reading Whatever it Takes the day before, I was excited to see him writing about the Harlem Children's Zone. I read the first couple paragraphs and started sending the link to friends I knew were interested in the topic. Then I read the rest of the piece and recoiled in horror. I like David Brooks, I find him to be the most consistently interesting of all the op-ed writers at the Times. But this is not just sloppy journalism, it's flat out irresponsible -- and I will have a hard time taking him seriously in the future if he fails to offer an apology to his readers.

Brooks writes about a recent analysis of test scores of students at the Promise Academy by Will Dobbie and Roland Fryer that found huge gains in achievement -- leading Fryer to write to Brooks that the "study has changed my life as a scientist."

Brooks concludes that the Promise Academy has succeeded in closing the Black-White achievement gap in math and that no-excuses schools can do this by themselves. He once again frames the debate as one between "reformers" who believe that schools can do great things and doubters who believe that they cannot overcome societal influences. He goes on to argue that "The Harlem Children’s Zone results suggest the reformers are right."

Brooks' analysis is so flawed that it's tough to know where to begin. Here are a few of the problems with his conclusions:

1.) The Harlem Children's Zone is an all-encompassing social services organization that provides everything from classes for expectant mothers to deliveries of healthy food for as many people as they can sign up in a 97 block area of Harlem. The idea of the project to is to combat every disadvantage that these residents face in order to help the next generation succeed. The Promise Academy is the charter school that HCZ operates. To claim that positive results that the school achieves are proof that school reform is sufficient independent of societal reform is, unacceptably sloppy if not disingenuous. Brooks apparently missed the sentence in italics on page 5 of the paper: "We cannot, however, disentangle whether communities coupled with high-quality schools drive our results, or whether the high-quality schools alone are enough to do the trick."

2.) Besides the fact that we're not sure exactly what induced the positive results, the claim that they were enough to close the black-white achievement gap is also somewhat dubious. As Aaron Pallas pointed out on Friday, the average student at the Promise Academy still scores significantly below the average White student in NYC.

3.) Even if we assume that the school itself can close the achievement gap, what, exactly, does this prove? Does anybody actually believe that simply evening the test scores of Black and White students is enough? The problem with focusing on the test-score gap as a proxy for inequality is that solving the test-score gap alone does little to solve the larger problem at hand. Sure, it would be a large step in the right direction -- but ultimately test scores aren't what matter. In the long run, what we really care about is whether students in poorer communities graduate from high school, earn college degrees, work in prestigious fields, escape poverty, avoid jail and welfare, and generally live healthy and productive lives. We cannot assume that higher test scores in elementary or middle school are enough to obtain all of these.

4.) Even if we assume that school itself can lead students to accomplish all of these things, we still have another problem: replication. For some reason that I can't quite figure out, we're all in search of the miracle cure in education. When somebody makes it home safely after drinking too much we don't rush to claim that drunk drivers who crash are making excuses or blaming their problems on alcohol. We still understand that driving drunk leads to bad things -- and that the exception doesn't disprove the rule. But every time we see a school with high test scores, a report pops up praising this as the answer to our educational problems. Even if this school walks on water, does it really mean that we've solved all of the problems of society? Each school has an inordinate number of things that make it unique -- the Promise Academy more so than most. Some of these things can be replicated, but some of them cannot. Not every school can have a well-connected and astoundingly wealthy Wall Street tycoon chairing its board of trustees. Not everybody can have Geoffrey Canada as their organization's President. Different schools succeed for different reasons, and it's not at all clear that we can replicate the reasons for success in this case. And it's certainly not clear that a so-called lack of excuses is even the main reason.

5.) While a group of haughty ideologues has successfully branded themselves as "reformers," this does absolutely nothing to prove that they are right. Brooks praises "no excuses" schools for the amazing results they've achieved. But the phrase "no excuses" has been bastardized so that it now means different things to different people. When Teach for America began training their corps members to run "no excuses" classrooms they wanted the future teachers to believe that every student can succeed, but the "no excuses" applied more to the students than to the teachers. TFA wanted teachers not to accept excuses from students -- they wanted them to be "loving hardasses" as Sherman Dorn puts it.

Those who write about "no excuses" schools usually mean that teachers and administrators do not make excuses -- that they do "whatever it takes" in order to ensure that every child succeeds, regardless of the challenges they face at home. And many who praise these schools interpret "no excuses" to mean something akin to "just do it" -- that is, that is that those in schools shut up and put up. Many point to these no excuses schools as proof not only that schools can do great things (side note: who believes they can't?) but that they can do them without wasting more resources.

In the case of the Promise Academy, however, this is certainly not the case. Beyond all of the community resources provided by HCZ, the school has pulled out all of the stops financially as well. When the middle school first opens the superintendent addresses the kids and explains that they will be given everything they need to succeed -- from up to date computers to better food. She says that "our motto is that anything that a private school can pay for, we are going to provide for free" (p. 127). In other words, even if the school does work miracles it doesn't prove that we can solve the problem by just buckling down and working harder -- it means that a school with an incredible amount of resources, a charismatic leader, a board of trustees led by a determined billionaire investor, and located in neighborhood awash in social services provided by the same organization had great success.

We seem to be a long way away from knowing for sure the outcomes of the Harlem Children's Zone or the Promise Academy -- many of the initiatives are only a few years old, and they're meant to effect change over the course of decades. But virtually all results indicate that something positive is taking place. So, by all means, let's celebrate those who have worked hard to achieve these results; by all means, let's take a closer look at what they're doing right; by all means, let's replicate their successes in whatever way we can. But let's avoid mischaracterizing why they've succeeded. To intimate that all schools are capable of fixing all ills of society if only those who work there would stop making excuses and focus is not only false but hinders efforts at honest reform in attempt to prove an ideological pet theory. A dedicated staff is certainly necessary for a school to succeed, but it is nowhere near sufficient. Brooks constructs a straw man to defeat, writing that the people on the wrong side of the argument believe that schools cannot do great things and that this proves they can. But none of the people with whom he's arguing believe that schools cannot do great things -- they believe that schools, given their current level of resources, cannot solve all of society's ills alone. And even if the Promise Charter school has, in fact, worked a miracle, it has done absolutely nothing to disprove this. In other words, don't buy what Brooks is selling.

Wednesday, May 6, 2009

on hiatus

Just wanted to let everybody know not to expect any posts from me this week. I finished my last term paper at Vanderbilt a few days ago and then decided I needed a week off to catch up on errands and yardwork. I'll be back with a vengeance next week.

Sunday, May 3, 2009

Sunday Commentary: Which Comes First, Better Schools or a Better Society?

In the debates over the merits of school reform vs. social reform – or in discussions of how to weave the two together – many make claims about the time-order sequence of reform. Which must come first, social reform or school reform? Is it the case that schools cannot meet their goals until societal inequalities are eliminated, or is it the case that societal inequalities will be eliminated by improved schooling?I propose an alternative hypothesis: that social reform and school reform have varying degrees of utility depending upon the timeframe in which one wants to achieve success.

In the very near term, certainly in cases of less than a year, it seems likely that school reforms would tend to be more efficacious. One would think that extra tutoring in reading, for example, would have a larger effect on reading scores six months from now than would moving into a new house. In the slightly longer run, however, it may be the case that social reform has more potential to reduce inequality.

In three of the four major empirical studies linking a social policy (or experiment designed to simulate a social policy) and educational performance, the authors measured the change in effects over time and in all three outcomes were significantly more positive after three or four years than they were after one. If this trend is generalizable to social policy at large, it stands to reason that social reform may produce better results a few years down the road than will school reform.

In the long-run, however, it may be the case that the achievement gap cannot be eliminated without school reform. Even if societal conditions are improved, one would think that those attending worse schools would tend to perform worse than those attending better schools. In this sense, at some point schooling will have to be equalized in order for the odds of success to also be equalized. In other words, although high-quality schools may not be sufficient for disadvantaged children to match their more advantaged counterparts today, they are likely necessary in the long run.