Saturday, April 12, 2008

Standardized Tests and Tenure

Earlier this week, New York passed a law banning the use of standardized test results when making decisions on teacher tenure. The little afterthought of a law that was stuck in a big budget bill has set the education blogosphere aflame. It contains a piece of almost all of the major issues of the day: unions, standardized tests, teacher quality, and the intervention of politicians.

Here's a recap of how various people reacted to the decision:

Joel Klein: The sky is falling, and it's all the union's fault
Randi Weingarten: We don't need no stinkin' standardized tests
The NY Times: It's not a good idea, and it wasted valuable time
The Quick and The Ed: It's bad . . . and anyone who argues it's not is stupid
Education Notes: It's good . . . and nobody is honestly refuting my argument
The Socratic Method: It's both unnecessary and stupid
Eduwonkette: People are overreacting
Sherman Dorn: Let's not forget that it's a moratorium, not a ban

My guess is that the way people view this law is largely the same as the way the view unions. If they don't like unions, they don't like the law; and vice-versa. And that's really a shame, because it's the sort of polarizing pseudo-argument that overly-partisan politicians in Washington use to paralyze our country.

The reality, in this situation, (as it always seems to be) is somewhere in between. The idea that unions are either purely good or purely bad is pure nonsense. In this case, unions went out of their way to push a law that may or may not have been unnecessary, but probably won't really hurt anyone in the short-run, though it could prove harmful if it became permanent. In the meantime, it's possible that the state government would have been more productive if they'd been doing something else. What? Exactly. Let's break this down:

The Means: Unions clearly used the back door route to pressure politicians to tuck what they wanted in an unrelated large bill. I can't fault anybody who doesn't like the way they did it -- they're not exactly setting a stellar example here.

The Ends: Districts aren't allowed to use data from state tests to make decisions on teacher tenure for the next two years. This might be a bit extreme, but I see little evidence that the data would have radically transformed decision-making. The last time I checked, the state English test was given in Mid-January and the state Math test was given in Mid-March (approximately 55% and 75% of the way through the school year). Furthermore, I'm not sure that the tests are designed to be used as value-added assessments (meaning that you can compare the results from one year to those from the previous year to see how much a student learned) as they are in TN and some other states. Throw in the fact that the majority of teachers do not teach 3rd-8th grade Math or English, and you have a set of information that is far from perfect. Maybe two years from now they'll have a better system, but right now I don't see any compelling reason to believe that a district or principal gains much from using the data or loses much from not using the data. Meanwhile, the possibility that inaccurate data is used to decide whether a teacher stays or goes is eliminated.

The Context: How much of an effect the law has over the next two years largely depends on how many principals and districts were planning on using the data from state tests to make decisions on tenure over the next two years. If nobody was planning on doing it, then the law was a gigantic waste of time for everybody involved. If a lot of people were moving in that direction, then the state just intervened in a heavy-handed way that might result in fairer tenure decisions over the next two years.

So, in the end, these are really the questions we should be answering:
  1. Did the ends justify the means?
  2. How can we obtain better information on teacher performance than is currently available?
  3. Is it possible for all of us to say the word "union" and remain rational?
update: I just read the comment that Sherman Dorn left on Eduwonkette's post, and he raises a very good point. The two-year moratorium means that NYC won't be able to use this data while Bloomberg (and, likely, Klein) are in office. That's gotta sting for them.


ed notes online said...

The point I was trying to make was all this on the part of Bloomberg/Klein and the union is about politics, not education. Klein has been in charge for 6 years and you mean to say almost all principals are giving teachers tenure and he has to get them to use test scores when they have all the weapons they need already - the untenured have just about 0 union protection. Many teachers see the UFT selling them out and this "victory" shows off their influence. This is a defeat for Bloomberg/Klein - their goal was to declare a victory over the UFT by adding a meaningless measure soley for bragging rights. Thus, the games that are played with children and teachers.

Brett Pawlowski said...

First things first - I have a favor to ask. You left a note on the EduWonk blog stating that you had seen a presentation on a survey of teachers based on the degree to which they "own" student learning. Can you tell me where I could find more information? It ties in directly to a project I'm working on involving value-added data in Tennessee. If you have anything you can send, please email me at

I also wnated to comment this particular post by asking, at what point do we decide that the data is good enough for us to use in making decisions, and who decides? As I mentioned, I'm familiar with TVAAS, and know that it is being used across the state to make instructional decisions, including staffing decisions. But others, notably eduwonkette, have argued that value-added isn't ready for prime time (and set up a near-impossible set of conditions that any assessment system would need to clear before she'd be comfortable).

So even though NY doesn't have value-added, do we not use performance data in any way until they do? Or is value-added not even sufficient? And what do we do in the meantime - eschew any independent data, and rely only on teachers' subjective assessments of progress and performance, knowing full well the conflicts of interests they represent?

Corey Bunje Bower said...

ENO: I remain skeptical that principals are as powerless to remove teachers as people make them out to be. You may be right, this might be pure politics, but both sides also have a genuine education issue on their side -- ensuring fair evaluation for the union and ensuring in-depth evaluation for administrators.

Brett: It wasn't really a measure of how much teachers "own" student learning, it was a measure of teacher efficacy combined with a measure of the influence they think home environment has on children. The two struck me as an odd grouping -- there seems to be an assumption that they're mutually exclusive.

I'm not an expert on the subject but, as far as I know, value-added isn't "ready for prime-time." I think some states are closer than others. I'm unconvinced that there's any value whatsoever in using the NYC data. You'd end up with a very fuzzy measure of how much students had learned in English between January of grade x and January of grade y. Beyond the fact that the comparison of the two scores is less than perfect, I see no way (and others have made this argument as well) to determine how much a teacher influenced the score differential. For example, little Johnny gains 15 points in reading between the two tests: Did Mrs. Smith cause 10 points of this gain, 0, 20? How much did Mrs. Jones cause? How much influence did the summer have? The law seems overbearing and was implemented in a secretive way, but I'm not convinced that it's depriving anybody of extremely useful information.