Thursday, November 18, 2010

Value-Added vs. Batting Averages

Stephen Sawchuck reports that a new report on value-added scores compares them to measures in other fields, including batting averages in baseball.  I think the batting average analogy is a good one.  Here's why:

When we look at a player's batting average (the number of hits divided by the number of at bats), we are provided with some information that reflects how good they are at baseball.  The value of this information increases when the player has more at bats and plays for more years.  And the larger the gap is between two players' batting averages, the more certain we can be that one player is better than the other.

But, at the same time, a player's batting average in one given year gives us only a small amount of information about that player's ability.  Averages vary year to year: players suffer injuries, become distracted, grow older, switch teams, play with different teammates, and so on.  For example:

Who's a better shortstop: Alexei Ramirez or Derek Jeter?  The former hit .282 this year while the latter hit .270.

Who's a better first baseman: Aubrey Huff or Ryan Howard?  The former hit .290; the latter .277.

Batting average doesn't do a good job of measuring power, speed, defense, leadership, or any of a million other desirable characteristics.  And for that reason, teams have moved beyond batting average when evaluating and signing players.  It seems like every year a new stat bubbles up that measures a player's ability in one facet of the game.

Adam Dunn is a career .250 hitter who plays awful defense, but will make millions of dollars next year because he'll probably hit 40 home runs yet again.

Mike Cameron, a .250 lifetime hitter, signed a 2-year, $15 million dollar contract last year as a 36 year-old.  His power (about 25 HR per season recently) didn't hurt, but the main motivation was his defense.

When we look at a more sophisticated (though still flawed) computation of how many wins a player added over an average replacement player (Wins Above Replacement, or WAR), we see that Jose Bautista, who hit .260 this year, ranks sixth.  The fifh-ranked player, Adrian Beltre, has a lifetime average of .275.

At the same time, nobody would argue that batting average is meaningless -- especially over longer periods of time.  There are plenty of hall of famers with career averages over .300, but none that I know of with averages of .220.

In other words, the batting average analogy is an excellent one for value-added scores because they have four very important things in common:

1.) Anybody who tells you that it is totally meaningless is totally wrong.
2.) When you see large differences over long periods of time between two people, you can be pretty sure that the one with the larger number is better.
3.) At the same time, that number in and of itself gives us very little information about how good somebody is, particularly over a shorter period of time.
4.) There are other skills not measured by the number that need to be taken into account when evaluating the person.

Unlike baseball, we don't have better statistics in evaluation.  We don't have something like on base percentage, yet alone change in student motivation over replacement teacher.

What does this mean for schools and value-added scores?  If we designed the perfect evaluation system, given current knowledge and tools, value-added scores would have to be included.  The only really compelling reason to leave them out is the potential for misuse by people who don't understand #'s 2-4 above.  Partly for that reason, it's important that value-added scores be only one component of how a teacher is evaluated.  And principals should be looking for large differences over long periods of time, not tiny year-to-year differences when making hiring/firing/tenure decisions.

Opponents of value-added scores would be better off arguing that they can be useful -- but only in some circumstances -- while proponents should be looking for supplemental measures to boost the meaningfulness of evaluations.  And both sides should keep in mind that the majority of teachers don't teach subjects tested by state tests anyway, so in addition to measuring a teacher's impact on students other than on standardized tests (e.g. motivation, attitude, self-control, creativity, interpersonal skills, etc.) we should also keep in mind that we need better ways to evaluate a teacher's impact on students in other subjects beyond 3rd-8th grade English/math as well.

In short, value-added scores can aid our evaluations of teachers . . . but only a little bit, and only if used properly.

Tuesday, November 16, 2010

"The Shadow Scholar"

In case you missed it, The Chronicle of Higher Education recently published an article written by "Ed Dante" (a psuedonym) about his/her experience writing papers for desperate/lazy college students.  I'd put "The Shadow Scholar" in the "must read" category for all those interested in higher education, and in the "well worth your time" category for everybody else.

The author makes decent money (on pace for 66K this year) writing papers for students and says that a student has never complained about getting caught.  The process and stories are interesting, but the author makes two main points:

1.) A lot of students are absolutely awful at writing and more should be done about this.
2.) A lot of assignments are ridiculous and drive students to look for other solutions (including purchasing papers)

I wholeheartedly agree with the first and half agree with the second.

The comments are almost as interesting as (though more tedious than) the article itself.  Between the author, the commenters, and my own head, there seem to be some competing hypotheses as to why a not insignificant number of students cheat in one way or another, particularly on written assignments.  In no particular order, they are:

1.) Professors assign meaningless/useless/stupid assignments
Why?
-large class sizes prevent individualized assignments
-laziness
-following conventions/norms
-more concerned with research than teaching

2.) Students don't know how to write
Why?
-they're non-native speakers
-they didn't learn how to write during K-12 schooling
-their college profs don't take the time to teach them how to write

3.) Students are unethical
Why?
-they don't understand what's ethical and what's not
-ethics and morals aren't emphasized enough in school
-a few bad apples make everyone look bad
-lots of pressure to get good grades; little pressure to behave ethically
-they don't value the assignments they're asked to complete or the courses they must take

4.) Unethical people make it easy to cheat
Why?
-some people care more about money than ethics
-people think they're doing others a service since they think the assignments are stupid


I don't think any of these fully explain the situation, but I do think they all shed a little bit of light on it.  In short, there are plenty of people and institutions at whom we can point fingers . . . but wouldn't it be more productive if we instead focused on fixing the problem?

Thursday, November 11, 2010

More on NYC's New Chancellor

The NY Times has a piece on Bloomberg's secretive search for the next schools chancellor.  A Times blog follows up with a mildly amusing post asking if anybody was actually interviewed during this search (and if they were, or know someone who was, to write in the comments section). 

The secrecy surrounding the search is interesting, but seems like it could be a red herring -- the ability of Ms. Black to lead the schools is ultimately more important than exactly how she was picked . . . with the caveat that if the process makes her appear illegitimate, it could undermine her authority.  Witness, for example, this online petition that a teacher friend was asking people to sign asking the state not to grant a waiver allowing her to serve as chancellor despite no educational background.

It's somewhat surprising to me that she's been greeted with this much skepticism given how many district leaders there are out there with no educational backgrounds, including her predecessor, Joel Klein.  It will be interesting to see if she's able to win over her detractors or if the next three years will be one long battle.

Wednesday, November 10, 2010

NYC's New Chancellor

The shocking news of the evening is that Joel Klein has stepped down as chancellor of NYC's schools (and accepted a job with News Corp.) and been replaced by former magazine exec Cathleen Black.  I knew nothing about Cathleen Black before a few hours ago, and have no idea whether she'll be a good leader for NYC's schools or not, so I'll leave the guessing to others.

But what I found most interesting about this is that she (a.) has no background in education and (b.) is described by the NY Times as someone who "earned a reputation in publishing as a tough-minded chief executive who never left her employees guessing what she wanted".

It's almost cliche at this point to hire a superintendent who is both an outsider and "tough".  I have no idea whether she'll still be described as "tough-minded" in years to come, so we'll have to wait and see on that one, but the absence of an educational background is worth more examination.

On the one hand, it's easy to see why somebody from the business world is attractive to a mayor or board of ed.  If your goal is to transform the school system as fast as possible, you want somebody who will come in and get the job done without much of a fuss.  And given that the vast majority of educational leaders have extensive backgrounds in education and that our system still has many problems, it would be easy to dismiss such a background as unnecessary.

And I don't think that such a view would be entirely without merit.  Ed. Schools probably catch more flak than they deserve, but it's pretty clear that many of them don't offer very high-quality or practical programs.  And there's a certain amount of value in bringing in somebody that hasn't been a part of the system -- there's an Icelandic expression that's relevant here: "the eye of the outsider is sharp".

At the same time, there's no reason that an outsider couldn't be somebody with experience outside of NYC instead of just outside of schools.  Having the negotiating skills or steely resolve necessary to stare down the union may or may not produce positive results, but it's also possible that experience as a teacher might more readily convince the union's constituents that new chief wants to work with them instead of against them.

All in all, I can't think of another field in which expertise or experience is treated with less regard than it is in education -- probably largely because virtually every resident of this country spent a good portion of their lives attending school.

Of course, if this trend continues, at some point in the future we'll read about some rebel mayor deciding to pursue the outside-the-box strategy of hiring somebody with a lot of educational experience to serve as superintendent since none of the business whizzes have yet found the magic cure-all for schools.

update: The NY Times has a forum on whether a district leader needs to have a background in education to succeed.  The arguments largely mirror those outlined above.  Clara Hemphill asks “would the mayor have named a publishing executive as head of the Department of Health?”  Richard Kahlenberg points out that a background as a teacher might make a leader more readily accepted by the rank-and-file.  And Marcus Winters and Neil McCluskey of the conservative Manhattan and Cato Institutes argue that an outsider perspective can more readily advance new ideas, especially since the old ideas that education folk have been trying have yet to cure all that ails our schools.

Friday, November 5, 2010

Today's Random Thoughts

*One college professor writes that their teacher prep program is losing many of their brightest prospective history teachers because the students are skeptical that they'll have much over the content of their courses and turned off by the battering teachers take in the press.  I tend to be skeptical of those who claim that there's a seismic shift -- with every single person who's creative abandoning teaching -- but I don't doubt that there is some serious dissatisfaction among some and, correspondingly, at least a small amount of attrition . . . I would expect it more from early-career teachers than from prospective teachers, so I find this piece both somewhat surprising and somewhat worrying.  Since we seem to be focusing a lot of time and effort lately into proposals that will attract and retain better teachers, we should probably investigate these types of claims before deciding which policies to pursue (hat tip: EP).

*A note to all the worried parents and college students out there: what the student chooses to major in doesn't really matter all that much when it comes to earnings later in life . . . in fact, it might be better to study what one is passionate about than simply picking what one deems most practical.

*Speaking of worried parents and teenagers, this NY Times/Chronicle of Higher Ed joint piece has a graphic that demonstrates fairly well the rise in the number of colleges that high schoolers have applied to over the past 20 years, though they miss out on the most explosive growth -- people applying to 10, 20, 30, or more colleges.  When I was in HS, it was typical for ambitious kids from well-to-do families to apply to about 8 or so; my understanding is that it's now typical for kids at private and upper-class public schools to apply to 20 or more schools (which the common app and online applications have made much easier to do).  In one sense, this is a savvy thing for kids to do -- with ever-declining acceptance rates, why not make sure you have every base covered?  But, at the same time, unless they're getting fee-waivers, that's a lot of money we're talking about for college applications.

*A group of researchers at Cornell write about ways to redesign lunch lines that will result in kids eating healthier food.  I always appreciate it when people find little tweaks that cost almost nothing, but get big results by nudging people in the right direction . . . but I have to wonder if simply doing these types of things wouldn't be abdicating our responsibility to teach kids about nutrition and responsibility.  If they don't take chocolate milk b/c it's behind the white milk and don't take ice cream b/c it's in a covered container, that's good for today -- but what will they do when they go to the grocery store as adults?

Thursday, November 4, 2010

I'm Still Alive

Apologies for the sudden dead air, but I've been swamped with dissertation and job-related activities. I'll try to update at least once each week over the next few months, but I can't make any promises.

Quick note while I'm here: everybody seems to be wondering if yesterday's election results will bring us compromise or gridlock for the next two years. If it's the former, education might be a logical starting point . . . I can't think of another major issue where the parties are closer to agreeing.

Friday, October 8, 2010

Small Schools and Extracurriculars

I've written this week (here and here) about the decision of Pittsburgh's superintendent to step down.  One of the reforms he's implemented over the past 5+ years is move the district away from large, comprehensive high schools and toward smaller, specialized high schools.  It's a trend anybody in NYC would be familiar with (where the average size of a middle school has shrunk by about 15% since 2004).  In the past I've referred to this as the "charterization" of schools.  As is clear in NYC and other places, charters tend to be much smaller and have more specialized names (think "Knowledge is Power Program" instead of "PS 100" or "Washington High"), and draw from beyond the local neighborhood.

In a lot of ways, I think smaller schools are a good thing (or at least have the potential to be a good thing).  I've heard one charter operator explain that he believes in schools with about 500 students because that's what all the top private schools are like -- and I think he makes a good point.  A small school makes it easier for everyone in the building to know each other and, subsequently, for schools to be more aware of the dynamics of the school environment and the needs of individual students.

But there's one major drawback to the smaller schools that doesn't seem to get much press: extracurriculars.  A lot of students define their experiences in high school (and, for that matter, college) by their experiences on athletic teams, in school plays, in the marching band, and so on.  When a friend mentions an old classmate and you can't quite put a face to the name, what do they say?  They don't say "he really liked history" . . . they say "she played the clarinet" or "he was on the swim team".  Everybody takes classes in high school, but far fewer people make pottery or play a leading role in the school play.  Accordingly, some students -- at least to some extent -- care more about their favorite extracurricular activity (or activities) than they do about classes.  Witness this excellent Sports Illustrated piece from last year about a school in Ohio that cut all sports and started hemorrhaging athletes.

So, what do we do to ensure that kids at these new smaller schools get to experience the activity (or activities) that might make them love high school?  There are a few options.

1.) We can continue to create schools that are focused around interests and activities -- arts academies and the like -- and hope that kids with similar interests enjoy similar activities.  Though I'm skeptical that athletic leagues will want to allow the new basketball academy to join its ranks.  And I'm not sure what happens when the quarterback at the football academy wants to join the glee club or the soprano at the choir academy wants to play volleyball.

2.) We can combine schools for the purposes of extracurriculars.  This is apparently Pittsburgh's plan, at least for football.  The biggest problem with that is that it robs the power of the school play or the Friday night football game to bring the community together (both the school community and the surrounding neighborhood).  If two separate schools field one football team, are they supposed to have two separate pep rallies?  And if two small towns merge their schools, in which town do they play football?  (Here's an excellent article on this phenomenon, again in the areas surrounding Pittsburgh.)

3.) Schools can boost rates of participation -- in various ways -- so that a smaller school can support more activities.  A number of private schools (I think, historically, this is something all-boys schools tended to do) mandate that every student play on a certain number of athletic teams each year.  Or schools could allow some clubs to meet during class time (I took a journalism class in jr. high in which we also wrote the school newspaper).  Or they could just emphasize clubs and activities more to students, parents, and teachers.

4.) We can go back to large, comprehensive, neighborhood high schools ("back" meaning in urban areas -- that's still still the norm in the suburbs and only somewhat possible in rural areas).  NYC might have the highest concentration of (relatively) small, specialized, non-neighborhood high schools in the country.  But it was announced this week that a new HS in Queens will be a large, comprehensive one.  “People want one large comprehensive school. You don’t want a bunch of boutique schools, a dance school, a school for lawyers,” said the DOE rep.

5.) Kids can just learn to do without.  Historically, the vast majority of schools in the United States have been quite small (which, of course, doesn't mean that the majority of students have attended small schools).  And many might deem other things (e.g. math skills) more important anyway.  Besides, there's always the possibility that local organizations (the theater, library, YMCA, little league, etc.) might sponsor activities if a school doesn't.  Of course, that would then remove one more enjoyable activity from students' school experience.

Extracurricular activities are extraordinarily important to many students, parents, schools, and communities and we do ourselves a disservice when we ignore this during our debates and discussions about education.

Thursday, October 7, 2010

More on Roosevelt's Departure

I wrote yesterday that I didn't understand why Mark Roosevelt was stepping down as Superintendent of the Pittsburgh Public Schools (and in mid-year, no less). And I still don't.

For those who are unfamiliar with Roosevelt and Pittsburgh, here's a little bit of background info: Roosevelt came to Pittsburgh a little over 5 years ago with no experience as a teacher, principal, or district administrator.  Which isn't to say was completely unqualified or without talent or relevant experience.  He had a law degree from Harvard, worked on education issues as a state legislator in Massachusetts (where he later ran for Governor), and served as a CEO and in various other leadership positions.  He then completed a fellowship with the Broad Foundation (not sure of the exact name of the program, but it was a 10 month program that dealt with leadership in urban schools) right before his appointment.

When he came to Pittsburgh, the district was most famous for its contentious (and that's being kind) board of ed meetings -- so contentious that a large number of foundations had pulled money from the district and warned that they'd better straighten up if they wanted to get it back.  At the same time, the city of Pittsburgh has fewer than half the residents that it did 60 years ago.  100 years ago it was the 8th largest city in the country and, I believe, had more corporate headquarters than any American city outside NYC (that might not be exactly correct, but the point is that it was a major center of industry).

Which means two things for the city and district: 1.) that they had far fewer students, but not far fewer schools, and 2.) that there's a lot of foundation money, per capita, in the city.  Which meant two things for Roosevelt: 1.) He needed to shrink or close schools, and 2.) he needed to make the foundations happy.

He closed 22 schools in all, and reconfigured many more.  Whether or not this was the correct move, one must admit that it took political courage and saved the district money -- and that it also upset a lot of parents living near schools that were closed or changed.  He's also been a big hit with the foundations.  The major Pittsburgh newspaper, the Post-Gazette, ran two stories on Roosevelt today (here and here), both of which are full of praise from all sorts of people.

Which is why it surprised me that he was leaving.  Yes, some parents and teachers didn't like him.  But the foundations loved him, the board of ed loved him, the mayor loved him, and the union barely put up a fight (they recently agreed on a 5-year contract).  I really have no insight into why he left -- I don't know the man personally and am unfamiliar with the day-to-day operations of his office -- so my best guess, given his previous frequent career-hopping, is that he was telling the truth when he said that he simply needed a new challenge.

The gist of what I've read and heard about him over the years is that he has many of the same strengths and weaknesses of a typical businessman.  He obviously got along well with the foundation people and the board of ed (a definite plus compared to previous regimes), but nobody would confuse him for a community organizer or former teacher.  The result was an administration that seemed to run smoothly and attracted lots of outside dollars, but didn't spend much time asking parents or teachers for advice.

Saying he ran the schools like a business would be cliche and overly simplistic, but he clearly wanted to streamline things.  He closed schools, narrowed curricula, and hired principals who would do things his way.

From the second article in the Post-Gazette: "In the classroom, Mr. Roosevelt sought to make teaching more consistent, including instituting a managed curriculum that required teachers to use certain materials and to present them at a certain pace. The recent teacher survey showed that some teachers think they have too little role in decision-making."


In the lower grades, the district closed many of the lowest-performing schools and moved the former students to K-8 "Accelerated Learning Academies" with extended hours and years.  In the upper grades, the vo-tech high school, and many other vo-tech programs, were closed in the name of academic rigor, scripted curricula were implemented, and he began opening small, specialized schools.  He leaned heavily on paid outside help to accomplish these reforms.  RAND evaluated all the schools, America's Choice provided the procedures and curricula for the ALA schools to follow, Kaplan wrote new HS curricula (and then was asked to leave so the district could re-write them themselves with some help from Pitt), and an outside firm was hired to run an alternative school for students with consistent discipline or attendance problems.  In the long run, he might be best remembered for starting the "Pittsburgh Promise," a fund that aims to pay full college tuition for qualifying district students who attend PA state colleges.  The fund got off to a rocky start, but is now much closer to its $250 million goal thanks, in part, to Roosevelt's relationship with local foundations.


In the end, he upset a number of parents and teachers.  Though the union rarely criticized him, there was a sharp divide between the attitudes of elementary and secondary teachers -- with the new contract being passed due to overwhelming support from the former and despite heavy opposition from the latter.  The lack of opposition from the union led to the election of some non-slate candidates in the last election who'd promised to be less docile.


And I'll be the last to argue that at least some of this anger was warranted.  I've written in the past that his new grading policy was poorly implemented and his newer system, though a little better, didn't work the way it was supposed to, that teacher morale was suffering under his leadership, and that some of his rhetoric was unhelpful.  But I'll be the first to admit that Pittsburgh could do a heck of a lot worse than 5+ years of more or less smooth sailing with the foundations, the board, and the union to go along with a steady stream of new ideas (some good and some bad).


But love him or hate him, one has to admit that it's somewhat irresponsible of him to step down abruptly in the middle of such turbulent change (and, personally, I think leaving mid-year reflects poorly on him -- but you're entitled to your own opinion).  This is the first year of a new $80 million dollar program (funded by Gates and the federal govt.) to re-vamp teacher pay and evaluation, numerous schools and students are slated to be reconfigured and moved next year, and he has a million other balls up in the air.  I think everybody can agree that he's done some good things and some bad things (though they certainly won't agree which things were bad and which were good), but I think everybody can also agree that he didn't finish the job.  It may or may not prove to be a good thing for the city/district that he didn't finish the job, and there may or may not be some very good reason(s) underlying the decision, but I'm somewhat disappointed in him personally for being willing to pick up and leave before the job is done.

Why?  In the midst of the effusive praise lavished on him by people of all different stripes was this quote from a leader of a local parents group: "Our group would ask that all initiatives be placed on hold until a new superintendent is found . . . We don't want all this money, expense, experimentation done if a new superintendent is going to take it in a different direction."  I don't know how many months it's going to take to find a new superintendent or how many of the current reforms will continue under his/her leadership, but the changeover seems pretty likely to result in a lot of uncertainty, hesitation, and delays.  Which can't be what Roosevelt thinks is best for the district.

Wednesday, October 6, 2010

Pittsburgh Superintendent Stepping Down

Forget the conflicting rumors over whether Ron Huberman is or is not stepping down as head of the Chicago school system, this is actually bigger news.  While Huberman is just getting his feet wet as chief of the Chicago schools, Mark Roosevelt dove in head-first a long time ago in Pittsburgh.

Apparently, he's resigning to become the head of Antioch College when it re-opens next fall -- though the college says there are a few more hurdles he has to leap before they'll be ready to make that decision.

But the timing of the move makes it both newsworthy and surprising.  And not just because he's jumping ship in the middle of the school year (December 31st, to be exact).  Few teachers would ever consider abandoning their students mid-year, so it seems somewhat tasteless for a Superintendent to do it (or maybe it just proves that teachers are more important, since the district will likely operate about the same in January as it did in December).

Roosevelt is in his sixth year leading Pittsburgh's schools (which is a fairly long time for an urban superintendent) and was, seemingly, right in the middle of his master plan for reform.  During his time he's implemented massive reform -- closing 22 schools and restructuring countless others, founding the "Pittsburgh Promise" (guaranteed college tuition for district grads that attend in-state public schools), and winning a $40 million grant from the Gates Foundation to overhaul teacher evaluation and pay last year, among many other things.  While he's upset more than a few teachers and parents, the school board is in his pocket, there are no national headlines about his brusque personality, and the mayor isn't going to be replaced anytime soon.

In other words, he pretty much had free reign to mold the district as he saw fit in the coming years (and his pay was recently upped to $240K, with which one can live like a King in eminently affordable Pittsburgh, as part of a new five year contract).  He was on top of the world.  And now he's leaving.  What?  What am I missing?

Considering that he had never worked in schools prior to his job in Pittsburgh (he was a lawyer/politician/businessman), and that he's faced so little opposition from the Board of Ed, it's hard to believe that he's burnt out (though, I suppose, not impossible).  He's originally from Massachusetts and attended Harvard, so there's not a blindingly obvious tie between him and Antioch (in Ohio).  He's never been a college administrator before, so he's not returning to his previous profession.  It's hard to believe that Antioch would be courting him if a giant scandal was about to be exposed.  So, why's he leaving?  You got me.  He's changed positions and professions quite a few times, so maybe he was just ready for a new challenge.

But considering that he was right in the middle of major reform in Pittsburgh, it seems like an awfully odd time to walk away.

It also exemplifies one of the problems with education.  Every time people start to adjust to a wave of reform, somebody new comes along and demands something else.  But people still wonder why teachers and school systems are so resistant to change.

It will be very interesting to see who gets hired as his replacement and which reforms he/she continues . . . stay tuned.

Tuesday, October 5, 2010

Today's Random Thoughts

-my latest pet peeve: people who complain about schools hiring people during a recession.  Why are we so bitter that people are finding gainful employment?

-The entire American educational system is not in crisis.  The entire system could be improved, yes, but only one portion (the same portion attended by children from low-income families) is truly in crisis.  That's essentially what Nicholas Lemann wrote in the New Yorker, but for some reason Joel Klein and others felt the need to write in and point out that a number of schools really are in crisis.  Am I the only one who fails to see an actual disagreement here?

-Not sure of the exact statistics, but Linda Perlstein writes that Macke Raymond told her that most charter school lottery "losers" actually end up at other charters, private schools, or move rather than attend the local public schools.  I'd like to see an actual statistic (e.g. is "most" 60% or 90%), but it sounds plausible.

-Albert Shanker was, apparently, full of good ideas.  I always wondered when I was teaching why a few exemplary lesson plans weren't made available for teachers on every topic so that they could get ideas and adapt them to their classrooms.  Wouldn't that make a lot more sense then asking every teacher to reinvent the wheel every time they teach?

-Another pet peeve: people who fundamentally misunderstand statistics and argue that a correlation doesn't exist because there's an exception to the rule.  Kevin Carey is far too smart to make that exact argument, but he falls into the same type of trap by pointing out that students in DC became poorer (it's unclear if this is true, but it's certainly plausible) over the past couple years at the same time that their test scores increased . . . which apparently means that income doesn't perfectly correlate with a child's test scores (?) or something like that.  Of course, we already knew that the relationship between poverty and academic performance is complex, that there's not a perfect correlation between income and test scores because a million other factors also impact test scores, and that children living in urban poverty are capable of improving their test scores.  So I'm unclear on exactly what this proves other than that people still don't understand that the exception doesn't disprove the rule and that a correlation doesn't mean that two things always move in lockstep.

-On a more positive note, I stumbled across this Freakonomics quorum on how to close the achievement gap the other day.  It's 2 1/2 years old, but it's still fairly interesting.