Tuesday, December 14, 2010

The Times' Tenure Math Problem

The NY Times today profiles some fairly reasonable, and minor, changes to the way tenure is granted to teachers in NYC (principals a.) have to describe why they're giving a teacher tenure and b.) are able to select from four ratings that can award, delay, or deny teachers tenure).

On the surface, those make sense, and the article seems to do a pretty good job describing them.  But I have a problem with the math on which the article is premised.  The second sentence of the article states that "virtually every new teacher earns" tenure, and then later cites a statistic that "6,400 teachers who were eligible for tenure, 234 teachers were denied it, or roughly 3.7 percent".

The problem here is that a "new teacher" and a teacher "eligible for tenure" are two very different things.  Writing that 96% of those eligible are awarded tenure is very different from writing that "virtually every new teacher earns" tenure.  Why?  To start with, NYC has a high attrition rate -- particularly among new teachers.  Nationwide, around 50% of teachers in urban schools leave within their first three years.  Let's say that that statistic is precisely correct for the teachers that entered NYC schools in 2007; the correct calculation would then be that 46% of new teachers who began teaching in 2007 were awarded tenure in 2010 -- which looks like very different.  In other words, the statement that "virtually every new teacher earns" tenure is demonstrably false.

Now, the 46% statistic wouldn't be all that helpful in judging the rigor of the tenure process because new teachers leave for all sorts of reasons.  Some leave because their principals threaten to fire them if they don't; some leave because they simply prefer a different district or profession; some leave because they're simply unhappy; and some leave for other reasons or a combination of reasons (e.g. family relocation, pregnancy, health, etc.).

So, leaving aside the silly -- but important -- syntax error by the Times, what would actually be the actual best way to compute how many teachers earn tenure?  There are a few different ways depending on exactly what the question is that one wants to answer, but I think you'd have to start by defining a population of teachers who want to earn tenure at some point during their teaching career (in NYC that would mean they want to teach in the city for more than 3 years).  Exactly who counts in that group is up for debate: for example, would we include a teacher who wants to make it a career during year 1 but finds that he/she just doesn't like teaching during year 2?  There's no clear answer.  I would certainly include, however, any teachers who leave explicitly because they believe they will receive a negative rating if they don't -- something that happens all over the place but never seems to get mentioned in the press and only rarely in academic work (I've heard economists reference "the dance of the lemons," to describe teachers who transfer to other schools to avoid being fired).  The estimate would shed some light on how many unsuccessful teachers -- teachers who might not receive tenure if they went through the process -- are leaving prior to starting their 4th year.

Whatever parameters one set, the answer to how many teachers tried to get tenure and didn't would be somewhere between the 46% estimate above and the 96% statistic cited by the NY Times.  The more practical questions to answer, however, would really be in estimating how many people would be valuable as tenured teachers but don't end up teaching more than 3 years for whatever reason and how many are not particularly valuable as tenured teachers but earn tenure anyway for any number of reasons (my guess is that the former group is larger than the latter, though there are also probably quite a few in the former who wouldn't stay in teaching almost no matter what), and then figuring out how to get more in the former group to stay and more in the latter group to leave.

At any rate, I don't think there's much disagreement that tenure is too easy to earn in NYC (personally I'd argue for a more meaningful process that's awarded a little later in one's career, and matched by some added distinction that's meaningful to teachers), so I don't want readers to think that this critique of the Times' math problem is really a veiled attack on any who would dare insinuate that all teachers aren't perfect.  But the Times should know better than to make such silly mistakes.  And I hope that readers know better than to accept such superficial calculations as gospel.

Do "virtually all new teacher earn" tenure?  No.  Do almost all teachers who go up for tenure review earn tenure?  Yes.  But those are two very different statistics -- and neither really answers the more important questions of how many excellent teachers don't receive tenure and how many poor teachers do.

No comments: