Tuesday, December 14, 2010

The Times' Tenure Math Problem

The NY Times today profiles some fairly reasonable, and minor, changes to the way tenure is granted to teachers in NYC (principals a.) have to describe why they're giving a teacher tenure and b.) are able to select from four ratings that can award, delay, or deny teachers tenure).

On the surface, those make sense, and the article seems to do a pretty good job describing them.  But I have a problem with the math on which the article is premised.  The second sentence of the article states that "virtually every new teacher earns" tenure, and then later cites a statistic that "6,400 teachers who were eligible for tenure, 234 teachers were denied it, or roughly 3.7 percent".

The problem here is that a "new teacher" and a teacher "eligible for tenure" are two very different things.  Writing that 96% of those eligible are awarded tenure is very different from writing that "virtually every new teacher earns" tenure.  Why?  To start with, NYC has a high attrition rate -- particularly among new teachers.  Nationwide, around 50% of teachers in urban schools leave within their first three years.  Let's say that that statistic is precisely correct for the teachers that entered NYC schools in 2007; the correct calculation would then be that 46% of new teachers who began teaching in 2007 were awarded tenure in 2010 -- which looks like very different.  In other words, the statement that "virtually every new teacher earns" tenure is demonstrably false.

Now, the 46% statistic wouldn't be all that helpful in judging the rigor of the tenure process because new teachers leave for all sorts of reasons.  Some leave because their principals threaten to fire them if they don't; some leave because they simply prefer a different district or profession; some leave because they're simply unhappy; and some leave for other reasons or a combination of reasons (e.g. family relocation, pregnancy, health, etc.).

So, leaving aside the silly -- but important -- syntax error by the Times, what would actually be the actual best way to compute how many teachers earn tenure?  There are a few different ways depending on exactly what the question is that one wants to answer, but I think you'd have to start by defining a population of teachers who want to earn tenure at some point during their teaching career (in NYC that would mean they want to teach in the city for more than 3 years).  Exactly who counts in that group is up for debate: for example, would we include a teacher who wants to make it a career during year 1 but finds that he/she just doesn't like teaching during year 2?  There's no clear answer.  I would certainly include, however, any teachers who leave explicitly because they believe they will receive a negative rating if they don't -- something that happens all over the place but never seems to get mentioned in the press and only rarely in academic work (I've heard economists reference "the dance of the lemons," to describe teachers who transfer to other schools to avoid being fired).  The estimate would shed some light on how many unsuccessful teachers -- teachers who might not receive tenure if they went through the process -- are leaving prior to starting their 4th year.

Whatever parameters one set, the answer to how many teachers tried to get tenure and didn't would be somewhere between the 46% estimate above and the 96% statistic cited by the NY Times.  The more practical questions to answer, however, would really be in estimating how many people would be valuable as tenured teachers but don't end up teaching more than 3 years for whatever reason and how many are not particularly valuable as tenured teachers but earn tenure anyway for any number of reasons (my guess is that the former group is larger than the latter, though there are also probably quite a few in the former who wouldn't stay in teaching almost no matter what), and then figuring out how to get more in the former group to stay and more in the latter group to leave.

At any rate, I don't think there's much disagreement that tenure is too easy to earn in NYC (personally I'd argue for a more meaningful process that's awarded a little later in one's career, and matched by some added distinction that's meaningful to teachers), so I don't want readers to think that this critique of the Times' math problem is really a veiled attack on any who would dare insinuate that all teachers aren't perfect.  But the Times should know better than to make such silly mistakes.  And I hope that readers know better than to accept such superficial calculations as gospel.

Do "virtually all new teacher earn" tenure?  No.  Do almost all teachers who go up for tenure review earn tenure?  Yes.  But those are two very different statistics -- and neither really answers the more important questions of how many excellent teachers don't receive tenure and how many poor teachers do.

Thursday, December 9, 2010

Remember: Not Everyone Prioritizes Achievement

The commentariat seems to agree on two things regarding American schools: that they're not good enough and that students need to be learning more.  Read a random blog or watch a random talk show and you'll likely see or hear something about more rigorous curricula, longer hours, longer school years, or some sort of argument that our students need to be pushed harder.

And you're twice as likely to see or hear these things after the results of an international assessment are released (e.g. the latest PISA results this week) and the U.S. again ranks behind a lot of countries.

But we in the education world can sometimes forget that there are other things that matter too.   It would be wise for those driving this commentary to remember that not everybody shares their priorities and keep in mind that many people in our country have no interest in spending more time in school.  I'd divide this constituency into three different groups:

1.) People who absolutely detest school and can't wait for the bell to ring.  Witness this cartoon comparing school to jail . . . with the only difference being that the disgusting food offered in jails is always free.  For kids who believe school is tantamount to jail or torture, it's unclear how exposing them to more misery will improve their lives.  In other words, it may be more important to find a type of schooling that they only hate a little bit than to simply pile on more.

2.) Families with students enrolled in elite, high-pressure schools (usually private or suburban).  Witness the number of parents excited by the new movie "Race to Nowhere" that chronicles the downside of such schools.  Indeed, the most obvious sign of the difference between classes when it comes to schooling may be the broad push for more rigorous schooling for low-income students while high-income parents increasingly worry about the amount of stress their children endure while running five clubs, taking 4 AP classes, and waking up early for SAT prep class on Saturday before going out to volunteer at the local animal shelter.

3.) Those whose priorities lie elsewhere -- e.g. sports.  Witness the saga of Urban Meyer, the University of Florida's highly successful football coach, who stepped down yesterday -- at the age of 46 -- because he wanted to spend more time with his family.  Did the press release say that he wanted to help them with their homework or volunteer in their classrooms?  No.  It said that "after spending more than two decades motivating and celebrating the young men I've been so proud to coach, I relish the opportunity to cheer for my three terrific kids as they compete in their own respective sports".  For other families it's music, art, drama, religion, or any of a million other activities that evoke more passion.

 There's some overlap, of course, between these groups.  But an awful lot of students/parents/families fall into at least one of these groups.  This certainly doesn't mean that we shouldn't examine longer days/years or such reforms to boost our national achievement levels.  But it does mean that we shouldn't assume that these types of changes would automatically be popular or successful.  And it illustrates some of the trade-offs of these types of reforms.

In a number of the highest scoring countries, many kids spend countless hours in "cram schools" after school and on weekends -- essentially doing test prep.  American kids, on the other hand, are more likely to go to football practice, piano lessons, or their part-time job after school.  There's no objective answer as to which practice is better, but time is finite and kids can't do everything.  And many, many parents would prefer that their kids choose from the latter list.

So, by all means, we should be examining ways in which we can boost our national achievement.  But boosting these test scores isn't the only thing that the citizens of our country care about.  As such, any reform designed to boost these scores will not automatically be accepted by all.  In short, a more productive discussion would weigh the costs and benefits of emphasizing different areas rather than simply assuming that everybody wants higher test scores at whatever cost.

Are Superintendant Searches a Thing of the Past?

After NYC hired an outsider (Joel Klein) to run their schools, Pittsburgh decided to hire an outsider as well (Mark Roosevelt).  Both leaders were controversial, but can claim some successes.  And both were generally liked more by policy wonks than by teachers or parents.

When NYC decided to replace Klein this year, the process was done behind closed doors and -- as far as anyone can tell -- exactly one candidate was interviewed (update: apparently, at least one other person may have been offered the job).  The end result is a prospective new chancellor (Cathleen Black) that may be even more strongly disliked by parents and teachers.  When Pittsburgh needed to replace their superintendent this year, meetings were held behind closed doors and exactly one candidate was interviewed.  The announcement was made today that the board will hire Deputy Superintendant Linda Lane as the next leader of Pittsburgh Public Schools on Monday.  It remains to be seen how parents and teachers will react to the news, but it seems unlikely that the move was made to assuage their concerns.

There, are, however, three major differences:

1.) Lane has spent almost 40 years working as a teacher and administrator in public schools, while it was unclear whether Black had even ever set foot in a public school when her hiring was announced.

2.) While Bloomberg was evasive and seemed to insinuate that others were interviewed for the position, the Pittsburgh Board of Education is being upfront about it.  Said the board president: "What would be the point of spending the money to do the search when what we wanted was right here?"

3.) Many in the press and public turned on Bloomberg after his announcement.  Judging by the glowing article in the Pittsburgh Post-Gazette, however, Lane is apparently already beloved by the press.  It will be interesting to see if the public feels the same way.  Despite the glowing review, it does seem somewhat unlikely that a board of ed that didn't even feel the need to pretend that they were conducting a thorough search would emerge completely unscathed.

Thursday, November 18, 2010

Value-Added vs. Batting Averages

Stephen Sawchuck reports that a new report on value-added scores compares them to measures in other fields, including batting averages in baseball.  I think the batting average analogy is a good one.  Here's why:

When we look at a player's batting average (the number of hits divided by the number of at bats), we are provided with some information that reflects how good they are at baseball.  The value of this information increases when the player has more at bats and plays for more years.  And the larger the gap is between two players' batting averages, the more certain we can be that one player is better than the other.

But, at the same time, a player's batting average in one given year gives us only a small amount of information about that player's ability.  Averages vary year to year: players suffer injuries, become distracted, grow older, switch teams, play with different teammates, and so on.  For example:

Who's a better shortstop: Alexei Ramirez or Derek Jeter?  The former hit .282 this year while the latter hit .270.

Who's a better first baseman: Aubrey Huff or Ryan Howard?  The former hit .290; the latter .277.

Batting average doesn't do a good job of measuring power, speed, defense, leadership, or any of a million other desirable characteristics.  And for that reason, teams have moved beyond batting average when evaluating and signing players.  It seems like every year a new stat bubbles up that measures a player's ability in one facet of the game.

Adam Dunn is a career .250 hitter who plays awful defense, but will make millions of dollars next year because he'll probably hit 40 home runs yet again.

Mike Cameron, a .250 lifetime hitter, signed a 2-year, $15 million dollar contract last year as a 36 year-old.  His power (about 25 HR per season recently) didn't hurt, but the main motivation was his defense.

When we look at a more sophisticated (though still flawed) computation of how many wins a player added over an average replacement player (Wins Above Replacement, or WAR), we see that Jose Bautista, who hit .260 this year, ranks sixth.  The fifh-ranked player, Adrian Beltre, has a lifetime average of .275.

At the same time, nobody would argue that batting average is meaningless -- especially over longer periods of time.  There are plenty of hall of famers with career averages over .300, but none that I know of with averages of .220.

In other words, the batting average analogy is an excellent one for value-added scores because they have four very important things in common:

1.) Anybody who tells you that it is totally meaningless is totally wrong.
2.) When you see large differences over long periods of time between two people, you can be pretty sure that the one with the larger number is better.
3.) At the same time, that number in and of itself gives us very little information about how good somebody is, particularly over a shorter period of time.
4.) There are other skills not measured by the number that need to be taken into account when evaluating the person.

Unlike baseball, we don't have better statistics in evaluation.  We don't have something like on base percentage, yet alone change in student motivation over replacement teacher.

What does this mean for schools and value-added scores?  If we designed the perfect evaluation system, given current knowledge and tools, value-added scores would have to be included.  The only really compelling reason to leave them out is the potential for misuse by people who don't understand #'s 2-4 above.  Partly for that reason, it's important that value-added scores be only one component of how a teacher is evaluated.  And principals should be looking for large differences over long periods of time, not tiny year-to-year differences when making hiring/firing/tenure decisions.

Opponents of value-added scores would be better off arguing that they can be useful -- but only in some circumstances -- while proponents should be looking for supplemental measures to boost the meaningfulness of evaluations.  And both sides should keep in mind that the majority of teachers don't teach subjects tested by state tests anyway, so in addition to measuring a teacher's impact on students other than on standardized tests (e.g. motivation, attitude, self-control, creativity, interpersonal skills, etc.) we should also keep in mind that we need better ways to evaluate a teacher's impact on students in other subjects beyond 3rd-8th grade English/math as well.

In short, value-added scores can aid our evaluations of teachers . . . but only a little bit, and only if used properly.

Tuesday, November 16, 2010

"The Shadow Scholar"

In case you missed it, The Chronicle of Higher Education recently published an article written by "Ed Dante" (a psuedonym) about his/her experience writing papers for desperate/lazy college students.  I'd put "The Shadow Scholar" in the "must read" category for all those interested in higher education, and in the "well worth your time" category for everybody else.

The author makes decent money (on pace for 66K this year) writing papers for students and says that a student has never complained about getting caught.  The process and stories are interesting, but the author makes two main points:

1.) A lot of students are absolutely awful at writing and more should be done about this.
2.) A lot of assignments are ridiculous and drive students to look for other solutions (including purchasing papers)

I wholeheartedly agree with the first and half agree with the second.

The comments are almost as interesting as (though more tedious than) the article itself.  Between the author, the commenters, and my own head, there seem to be some competing hypotheses as to why a not insignificant number of students cheat in one way or another, particularly on written assignments.  In no particular order, they are:

1.) Professors assign meaningless/useless/stupid assignments
Why?
-large class sizes prevent individualized assignments
-laziness
-following conventions/norms
-more concerned with research than teaching

2.) Students don't know how to write
Why?
-they're non-native speakers
-they didn't learn how to write during K-12 schooling
-their college profs don't take the time to teach them how to write

3.) Students are unethical
Why?
-they don't understand what's ethical and what's not
-ethics and morals aren't emphasized enough in school
-a few bad apples make everyone look bad
-lots of pressure to get good grades; little pressure to behave ethically
-they don't value the assignments they're asked to complete or the courses they must take

4.) Unethical people make it easy to cheat
Why?
-some people care more about money than ethics
-people think they're doing others a service since they think the assignments are stupid


I don't think any of these fully explain the situation, but I do think they all shed a little bit of light on it.  In short, there are plenty of people and institutions at whom we can point fingers . . . but wouldn't it be more productive if we instead focused on fixing the problem?

Thursday, November 11, 2010

More on NYC's New Chancellor

The NY Times has a piece on Bloomberg's secretive search for the next schools chancellor.  A Times blog follows up with a mildly amusing post asking if anybody was actually interviewed during this search (and if they were, or know someone who was, to write in the comments section). 

The secrecy surrounding the search is interesting, but seems like it could be a red herring -- the ability of Ms. Black to lead the schools is ultimately more important than exactly how she was picked . . . with the caveat that if the process makes her appear illegitimate, it could undermine her authority.  Witness, for example, this online petition that a teacher friend was asking people to sign asking the state not to grant a waiver allowing her to serve as chancellor despite no educational background.

It's somewhat surprising to me that she's been greeted with this much skepticism given how many district leaders there are out there with no educational backgrounds, including her predecessor, Joel Klein.  It will be interesting to see if she's able to win over her detractors or if the next three years will be one long battle.

Wednesday, November 10, 2010

NYC's New Chancellor

The shocking news of the evening is that Joel Klein has stepped down as chancellor of NYC's schools (and accepted a job with News Corp.) and been replaced by former magazine exec Cathleen Black.  I knew nothing about Cathleen Black before a few hours ago, and have no idea whether she'll be a good leader for NYC's schools or not, so I'll leave the guessing to others.

But what I found most interesting about this is that she (a.) has no background in education and (b.) is described by the NY Times as someone who "earned a reputation in publishing as a tough-minded chief executive who never left her employees guessing what she wanted".

It's almost cliche at this point to hire a superintendent who is both an outsider and "tough".  I have no idea whether she'll still be described as "tough-minded" in years to come, so we'll have to wait and see on that one, but the absence of an educational background is worth more examination.

On the one hand, it's easy to see why somebody from the business world is attractive to a mayor or board of ed.  If your goal is to transform the school system as fast as possible, you want somebody who will come in and get the job done without much of a fuss.  And given that the vast majority of educational leaders have extensive backgrounds in education and that our system still has many problems, it would be easy to dismiss such a background as unnecessary.

And I don't think that such a view would be entirely without merit.  Ed. Schools probably catch more flak than they deserve, but it's pretty clear that many of them don't offer very high-quality or practical programs.  And there's a certain amount of value in bringing in somebody that hasn't been a part of the system -- there's an Icelandic expression that's relevant here: "the eye of the outsider is sharp".

At the same time, there's no reason that an outsider couldn't be somebody with experience outside of NYC instead of just outside of schools.  Having the negotiating skills or steely resolve necessary to stare down the union may or may not produce positive results, but it's also possible that experience as a teacher might more readily convince the union's constituents that new chief wants to work with them instead of against them.

All in all, I can't think of another field in which expertise or experience is treated with less regard than it is in education -- probably largely because virtually every resident of this country spent a good portion of their lives attending school.

Of course, if this trend continues, at some point in the future we'll read about some rebel mayor deciding to pursue the outside-the-box strategy of hiring somebody with a lot of educational experience to serve as superintendent since none of the business whizzes have yet found the magic cure-all for schools.

update: The NY Times has a forum on whether a district leader needs to have a background in education to succeed.  The arguments largely mirror those outlined above.  Clara Hemphill asks “would the mayor have named a publishing executive as head of the Department of Health?”  Richard Kahlenberg points out that a background as a teacher might make a leader more readily accepted by the rank-and-file.  And Marcus Winters and Neil McCluskey of the conservative Manhattan and Cato Institutes argue that an outsider perspective can more readily advance new ideas, especially since the old ideas that education folk have been trying have yet to cure all that ails our schools.

Friday, November 5, 2010

Today's Random Thoughts

*One college professor writes that their teacher prep program is losing many of their brightest prospective history teachers because the students are skeptical that they'll have much over the content of their courses and turned off by the battering teachers take in the press.  I tend to be skeptical of those who claim that there's a seismic shift -- with every single person who's creative abandoning teaching -- but I don't doubt that there is some serious dissatisfaction among some and, correspondingly, at least a small amount of attrition . . . I would expect it more from early-career teachers than from prospective teachers, so I find this piece both somewhat surprising and somewhat worrying.  Since we seem to be focusing a lot of time and effort lately into proposals that will attract and retain better teachers, we should probably investigate these types of claims before deciding which policies to pursue (hat tip: EP).

*A note to all the worried parents and college students out there: what the student chooses to major in doesn't really matter all that much when it comes to earnings later in life . . . in fact, it might be better to study what one is passionate about than simply picking what one deems most practical.

*Speaking of worried parents and teenagers, this NY Times/Chronicle of Higher Ed joint piece has a graphic that demonstrates fairly well the rise in the number of colleges that high schoolers have applied to over the past 20 years, though they miss out on the most explosive growth -- people applying to 10, 20, 30, or more colleges.  When I was in HS, it was typical for ambitious kids from well-to-do families to apply to about 8 or so; my understanding is that it's now typical for kids at private and upper-class public schools to apply to 20 or more schools (which the common app and online applications have made much easier to do).  In one sense, this is a savvy thing for kids to do -- with ever-declining acceptance rates, why not make sure you have every base covered?  But, at the same time, unless they're getting fee-waivers, that's a lot of money we're talking about for college applications.

*A group of researchers at Cornell write about ways to redesign lunch lines that will result in kids eating healthier food.  I always appreciate it when people find little tweaks that cost almost nothing, but get big results by nudging people in the right direction . . . but I have to wonder if simply doing these types of things wouldn't be abdicating our responsibility to teach kids about nutrition and responsibility.  If they don't take chocolate milk b/c it's behind the white milk and don't take ice cream b/c it's in a covered container, that's good for today -- but what will they do when they go to the grocery store as adults?

Thursday, November 4, 2010

I'm Still Alive

Apologies for the sudden dead air, but I've been swamped with dissertation and job-related activities. I'll try to update at least once each week over the next few months, but I can't make any promises.

Quick note while I'm here: everybody seems to be wondering if yesterday's election results will bring us compromise or gridlock for the next two years. If it's the former, education might be a logical starting point . . . I can't think of another major issue where the parties are closer to agreeing.

Friday, October 8, 2010

Small Schools and Extracurriculars

I've written this week (here and here) about the decision of Pittsburgh's superintendent to step down.  One of the reforms he's implemented over the past 5+ years is move the district away from large, comprehensive high schools and toward smaller, specialized high schools.  It's a trend anybody in NYC would be familiar with (where the average size of a middle school has shrunk by about 15% since 2004).  In the past I've referred to this as the "charterization" of schools.  As is clear in NYC and other places, charters tend to be much smaller and have more specialized names (think "Knowledge is Power Program" instead of "PS 100" or "Washington High"), and draw from beyond the local neighborhood.

In a lot of ways, I think smaller schools are a good thing (or at least have the potential to be a good thing).  I've heard one charter operator explain that he believes in schools with about 500 students because that's what all the top private schools are like -- and I think he makes a good point.  A small school makes it easier for everyone in the building to know each other and, subsequently, for schools to be more aware of the dynamics of the school environment and the needs of individual students.

But there's one major drawback to the smaller schools that doesn't seem to get much press: extracurriculars.  A lot of students define their experiences in high school (and, for that matter, college) by their experiences on athletic teams, in school plays, in the marching band, and so on.  When a friend mentions an old classmate and you can't quite put a face to the name, what do they say?  They don't say "he really liked history" . . . they say "she played the clarinet" or "he was on the swim team".  Everybody takes classes in high school, but far fewer people make pottery or play a leading role in the school play.  Accordingly, some students -- at least to some extent -- care more about their favorite extracurricular activity (or activities) than they do about classes.  Witness this excellent Sports Illustrated piece from last year about a school in Ohio that cut all sports and started hemorrhaging athletes.

So, what do we do to ensure that kids at these new smaller schools get to experience the activity (or activities) that might make them love high school?  There are a few options.

1.) We can continue to create schools that are focused around interests and activities -- arts academies and the like -- and hope that kids with similar interests enjoy similar activities.  Though I'm skeptical that athletic leagues will want to allow the new basketball academy to join its ranks.  And I'm not sure what happens when the quarterback at the football academy wants to join the glee club or the soprano at the choir academy wants to play volleyball.

2.) We can combine schools for the purposes of extracurriculars.  This is apparently Pittsburgh's plan, at least for football.  The biggest problem with that is that it robs the power of the school play or the Friday night football game to bring the community together (both the school community and the surrounding neighborhood).  If two separate schools field one football team, are they supposed to have two separate pep rallies?  And if two small towns merge their schools, in which town do they play football?  (Here's an excellent article on this phenomenon, again in the areas surrounding Pittsburgh.)

3.) Schools can boost rates of participation -- in various ways -- so that a smaller school can support more activities.  A number of private schools (I think, historically, this is something all-boys schools tended to do) mandate that every student play on a certain number of athletic teams each year.  Or schools could allow some clubs to meet during class time (I took a journalism class in jr. high in which we also wrote the school newspaper).  Or they could just emphasize clubs and activities more to students, parents, and teachers.

4.) We can go back to large, comprehensive, neighborhood high schools ("back" meaning in urban areas -- that's still still the norm in the suburbs and only somewhat possible in rural areas).  NYC might have the highest concentration of (relatively) small, specialized, non-neighborhood high schools in the country.  But it was announced this week that a new HS in Queens will be a large, comprehensive one.  “People want one large comprehensive school. You don’t want a bunch of boutique schools, a dance school, a school for lawyers,” said the DOE rep.

5.) Kids can just learn to do without.  Historically, the vast majority of schools in the United States have been quite small (which, of course, doesn't mean that the majority of students have attended small schools).  And many might deem other things (e.g. math skills) more important anyway.  Besides, there's always the possibility that local organizations (the theater, library, YMCA, little league, etc.) might sponsor activities if a school doesn't.  Of course, that would then remove one more enjoyable activity from students' school experience.

Extracurricular activities are extraordinarily important to many students, parents, schools, and communities and we do ourselves a disservice when we ignore this during our debates and discussions about education.