Earlier, I wrote a post pointing out that Teach For America members can do a lot of good after their two-year commitment is up even if they're not in the classroom. In that post, I wrote that "it’s a fact that most do not teach long, if at all, past their two year commitment".
A commenter over at Eduwonk takes issue with this statement by pointing to this post, which says that one study found that 61% of TFA teachers teach beyond their 3rd year and then continues to say that a TFA alumni survey found that 32% of current alums are teachers right now.
Even though TFA is 20 years old, substantial recent growth means that there are far more graduates from the last few years than from the first 10 or so. Plus, it seems plausible that earlier grads might be harder to track down and/or less likely to respond (though this can be addressed by proper weighting of the results, I don't have the survey report so I'm not sure whether or not this was done). So let's say that the average TFA alum in this survey is 5 years out and 32% of those 5 years out are still teaching. That would be mean that over 2/3 of TFA members teach fewer than 5 years after their two year commitment. And that few alums stick around very long in the classroom -- just as I wrote before. Perhaps more to the point, the vast majority of TFA members do not become career teachers.
But my guess is that the criticism of my statement wasn't really an attack on my methodology or facts. My guess is that it was a defensive reaction by somebody who supports TFA. Indeed, I've seen TFA employees become very defensive when people ask about the attrition numbers. And I don't think they need to be. Because TFA can do a lot of good regardless of whether or not all their teachers immediately bolt the classroom. We're once again confusing two separate issues:
1.) Does TFA transform the teaching profession?
2.) Does TFA positively impact high-poverty schools?
I've long argued that the most significant effect of TFA in the long-run is going to be the large number of people in power who have taught in high-poverty schools and care about the issues confronting them -- not the formation of teaching force that accounts for fewer than 1% all of teachers in America. I think it's more than possible for 2/3 of TFA alums to quickly move out of the classroom and the program to have a significant net positive effect.
In other words, I don't see the exodus of TFA alums from the teaching profession as something that's necessarily, or at least entirely, bad. TFA can still do a lot of good even if every single teacher leaves after their second year.
So, I'd argue that the answer to question #1 is mostly "no" and the answer to question #2 is mostly "yes" -- and that that's perfectly ok. TFA employees and supporters should have no problem with this, and TFA critics should acknowledge that TFA can do a lot of good even if it doesn't provide a long-term, comprehensive solution to the shortage of high-quality teachers in high-poverty schools.
Here's how I'd frame it if I were in charge of PA for the organization: TFA is providing an emergency stop-gap intervention in the short run, with the long-term goal of transforming the system so that we no longer need emergency interventions. Critics allege that TFA is nothing but a band-aid solution, but band-aids can do a world of good if they're then followed up with more extensive care.
The band-aid that TFA provides is reasonably good teachers in schools that, for the most part, wouldn't otherwise be able to hire people of comparable success. Studies of TFA tend to find that TFA teachers are as good as, or a little better than, other teachers in the district. If anything, it's likely that these studies actually understate the relative quality of TFA teachers. I say this because they compare TFA teachers to other teachers currently teaching in the district. But the most relevant comparison group are the teachers who didn't get hired because a TFA teacher was hired instead. In some cases, a perfectly competent teacher lost out to a TFA member, but in many cases a TFA member was hired instead of somebody that was unqualified or even instead of the position remaining vacant for the year. On average, it's likely that the teachers not hired because of TFA are a little worse than the average teacher in the district, or even the average new or provisionally certified teacher in the district. All this is to say that it's likely that most schools that hire TFA members are better off in the short-run with a TFA teacher or three in place than they would have been had they not had the option of hiring TFA members (now, there may be exceptions -- schools were principals hire TFA teachers that will actually be worse than the alternative -- but, in the aggregate, there is little doubt that most schools that do so are better off, at least in the short run).
The more extensive follow-up care that TFA provides is through the engagement of thousands of the country's most ambitious and talented young adults in the issues facing high-poverty schools. Countless TFA alums who might otherwise be doing something else are now teaching in high-poverty schools, leading high-poverty schools, or advocating for high-poverty schools. In the decades to come, I think we'll see steady growth in the number of TFA alums leading schools and districts, publishing research on schools and districts, managing educational non-profits, serving in legislative bodies, and influencing the situation on other ways. I'm not sure what, exactly, the end results of these efforts will be, but I have little doubt that they have the potential to do more for America's education system as a whole than do those TFA members currently serving in the classroom. This is not to diminish the importance of those currently in the classroom -- to the students in their class(es), they might be the most important figure in their life -- but, rather, to say that since TFA teachers make up well under 1% of the current teaching force, there are practical limits as to what they can do for the entire system.
So, TFA really has no need to adopt a defensive posture and hide behind the misleading statistic that 61% of people teach at some point after their second year. When TFA recruits and trains their members, they actively encourage them to spend a few years in the classroom and then go to pursue other careers (the job market for TFA alums is pretty robust, as are the job placement services provided by TFA). But, when people transition from their time as TFA members to their time as TFA alums they are encouraged to stay engaged in the issues surrounding high-poverty schools. At the current rate, within a couple decades there will be hundreds of thousands of TFA alums in high-ranking positions throughout the country -- so, those who focus on the few thousand young teachers in the classroom may be missing out on the big picture. No, we're never going to fully staff all of our high-poverty schools with TFA teachers, but that shouldn't be the measure of success for TFA.
So, to answer the question in the title, yes: TFA can hemorrhage teachers and still have a net positive impact on our society and our schools. And TFA advocates and critics alike should keep that in mind.
*full disclosure: as it says on the front page of my blog, I am an alum of the NYC Teaching Fellows (a sister program of TFA) who no longer teaches in a high-poverty school, but remains engaged in the issues confronting them. While I am not a TFA alum, it's possible that I have a soft spot for former teachers who believe they can still make a difference.
Thursday, January 27, 2011
TFA Members, Year 3+
Eduwonk writes that the number of people signed up for Teach for America's upcoming summit "should put to rest the idea that corps members just do two years and move on".
But I think he's confusing two separate questions here:
1.) Do TFA corps members remain in the classroom?
2.) Do TFA corps members remain engaged in education?
It's a fact that most do not teach long, if at all, past their two year commitment. That's not necessarily a bad thing, though it does make it impossible for TFA in its current form to be a large-scale solution for the teacher quality problem.
But I've always argued that the second question is more important. There's a bevy of anecdotal evidence that TFA members move on to other careers (both within and outside of education) that impact our educational system. We're already seeing TFA alums serving as principals and superintendents, and in the next decade I think we'll see a growing number of TFA alums taking leadership roles in academia, government, and other fields as well. To me, this will be the largest impact of TFA in the long run.
But I think he's confusing two separate questions here:
1.) Do TFA corps members remain in the classroom?
2.) Do TFA corps members remain engaged in education?
It's a fact that most do not teach long, if at all, past their two year commitment. That's not necessarily a bad thing, though it does make it impossible for TFA in its current form to be a large-scale solution for the teacher quality problem.
But I've always argued that the second question is more important. There's a bevy of anecdotal evidence that TFA members move on to other careers (both within and outside of education) that impact our educational system. We're already seeing TFA alums serving as principals and superintendents, and in the next decade I think we'll see a growing number of TFA alums taking leadership roles in academia, government, and other fields as well. To me, this will be the largest impact of TFA in the long run.
So, regarding his post, I don't think a large turnout would say anything about the first debate, but it would say an awful lot about the second.
Tuesday, January 25, 2011
How the US Health Care System is like the US Education System
I was struck by this NY Times piece discussing a recent report by the National Academy of Sciences on longevity in the US and other nations.. I was struck because one could substitute an analogous remark about education for virtually every mention of health care and the report would still ring true. For example:
Over the last 25 years, life expectancy at age 50 in the U.S. has been rising, but at a slower pace than in many other high-income countries, such as Japan and Australia. This difference is particularly notable given that the U.S. spends more on health care than any other nation
could become
Over the last 25 years, educational achievement in the U.S. has been rising, but at a slower pace than in many other high-income countries . . . This difference is particularly notable given that the U.S. spends more on education than any other nation
Here's the best graph I could find of this trend (the red dots represent the US):
and another example:
With respect to income inequality, it is widely believed that such inequalities are higher in the United States than in other high-income countries . . . Poverty rates also appear to be higher in the United States than in most of the other countries . . .
This combination of factors could result in higher mortality rates among people in the lower socioeconomic brackets in the United States than in other countries, pulling down US life expectancy levels in general
could become
This combination of factors could result in lower achievement rates among people in the lower socioeconomic brackets in the United States than in other countries, pulling down US achievement levels in general
Additionally, we could draw parallels between the success of the highly sophisticated and specialized branches of both health care (e.g. cancer research) that the report lauds and our nation's elite college and university system.
So, what does this all mean? I'd argue that the parallels illustrate two truisms about American education:
1.) Society matters more than any single institutions. While the quality of our doctors and teachers influence how healthy and educated people are, other factors are, cumulatively, far more important. Both are affected by parenting, neighborhood context, diet, exercise, and so on. But while we rush to blame teachers for the failures of American students, I hear very few rushing to blame doctors for the American populace's unhealthy lifestyle.
2.) We excel when we strive for excellence, but place little emphasis on providing excellence for all -- one concoct examples from numerous other fields (e.g. transportation or housing). Whether this is good or bad is largely a value judgment, but it's a fact that the US is more individualist and less collectivist than maybe any nation on Earth.
The end result of these two factors is that our health and education systems are quite similar in many ways (which should surprise no one) and, as such, are likely similarly difficult to reform.
Over the last 25 years, life expectancy at age 50 in the U.S. has been rising, but at a slower pace than in many other high-income countries, such as Japan and Australia. This difference is particularly notable given that the U.S. spends more on health care than any other nation
could become
Over the last 25 years, educational achievement in the U.S. has been rising, but at a slower pace than in many other high-income countries . . . This difference is particularly notable given that the U.S. spends more on education than any other nation
Here's the best graph I could find of this trend (the red dots represent the US):
and another example:
With respect to income inequality, it is widely believed that such inequalities are higher in the United States than in other high-income countries . . . Poverty rates also appear to be higher in the United States than in most of the other countries . . .
This combination of factors could result in higher mortality rates among people in the lower socioeconomic brackets in the United States than in other countries, pulling down US life expectancy levels in general
could become
This combination of factors could result in lower achievement rates among people in the lower socioeconomic brackets in the United States than in other countries, pulling down US achievement levels in general
Additionally, we could draw parallels between the success of the highly sophisticated and specialized branches of both health care (e.g. cancer research) that the report lauds and our nation's elite college and university system.
So, what does this all mean? I'd argue that the parallels illustrate two truisms about American education:
1.) Society matters more than any single institutions. While the quality of our doctors and teachers influence how healthy and educated people are, other factors are, cumulatively, far more important. Both are affected by parenting, neighborhood context, diet, exercise, and so on. But while we rush to blame teachers for the failures of American students, I hear very few rushing to blame doctors for the American populace's unhealthy lifestyle.
2.) We excel when we strive for excellence, but place little emphasis on providing excellence for all -- one concoct examples from numerous other fields (e.g. transportation or housing). Whether this is good or bad is largely a value judgment, but it's a fact that the US is more individualist and less collectivist than maybe any nation on Earth.
The end result of these two factors is that our health and education systems are quite similar in many ways (which should surprise no one) and, as such, are likely similarly difficult to reform.
How Many Bad Teachers Are There?
With all the discussion surrounding the need to fire "bad teachers" these days, I wish we could take a step back and examine how many teachers out there are truly worth firing.
A working paper by Thomas Kane and colleagues sheds some light on the issue. In the paper, they investigate the validity of teacher observations conducted by outsiders and find them to be fairly strongly correlated with both teacher value-added scores and levels of teacher experience. This leads to the conclusion that the observations are fairly good measures of teacher effectiveness.
Included in the paper is this chart showing the spread of teachers who averaged ratings on a 1-4 scale.
What sticks out to me is that the distribution is skewed far to the right -- well over half of the teachers earn at least a 3 on the scale, and there are only a few that score really low. This gibes with TNTP's report on San Francisco, in which fewer than 1% of teachers are rated unsatisfactory, and 94% of principals agree that teachers in their school who are underperforming are rated as such.
In other words, maybe the vast majority of teachers are doing a pretty good job. Which, of course, is not to say that teachers who are failing their students should remain as is forever. Clearly, as in any profession, there are teachers who need to receive more training, work harder, and/or move to another job. But if this is the case (and I'm not going to claim it is based on only one study), then we should be a little more careful with our rhetoric. If most teachers are doing a pretty good job, then we should probably spend a little more time praising them, a little less time decrying the fate of the teaching profession, and refer to bad teachers as the exception rather than the rule.
A working paper by Thomas Kane and colleagues sheds some light on the issue. In the paper, they investigate the validity of teacher observations conducted by outsiders and find them to be fairly strongly correlated with both teacher value-added scores and levels of teacher experience. This leads to the conclusion that the observations are fairly good measures of teacher effectiveness.
Included in the paper is this chart showing the spread of teachers who averaged ratings on a 1-4 scale.
What sticks out to me is that the distribution is skewed far to the right -- well over half of the teachers earn at least a 3 on the scale, and there are only a few that score really low. This gibes with TNTP's report on San Francisco, in which fewer than 1% of teachers are rated unsatisfactory, and 94% of principals agree that teachers in their school who are underperforming are rated as such.
In other words, maybe the vast majority of teachers are doing a pretty good job. Which, of course, is not to say that teachers who are failing their students should remain as is forever. Clearly, as in any profession, there are teachers who need to receive more training, work harder, and/or move to another job. But if this is the case (and I'm not going to claim it is based on only one study), then we should be a little more careful with our rhetoric. If most teachers are doing a pretty good job, then we should probably spend a little more time praising them, a little less time decrying the fate of the teaching profession, and refer to bad teachers as the exception rather than the rule.
Monday, January 17, 2011
3.6 Inches of Snow = 10 Days Without School
Last week, we had 3.6 inches of snow in Nashville (2.5 inches on Monday morning, .6 inches on Tuesday, .4 inches in Wednesday, and .1 inches on Thursday according to the NOAA). The result was that Nashville's school system -- and almost all others in the mid-state region -- canceled school for the entire week. Though only 5 days of school were canceled, the day off for MLK's birthday means that there will be 10 consecutive days without school in the Metro Nashville Public Schools.
As a Northern transplant, I've often cynically joked that Nashville cancels one day of school for each inch of snowfall. Since I moved here, this the first time that the rule hasn't held.
Why would 3.6 inches of snow shut down a school system for an entire week? You might think it's because of the lack of snow clearing equipment in Southern cities, and you'd be partially right. But all the main roads had been plowed and salted by the end of the day Monday, and the vast majority of side roads were clear by Tuesday afternoon. So what gives? Well, a small fraction of the side roads -- including a number that school buses regularly travel -- remained covered with ice the entire week. This was the result of two things: 1.) temperatures that remained below freezing the entire week; and 2.) the fact that side roads, apparently, do not get salted or plowed in this city.
The continued cancellation of school day after day must've gotten somebody's attention, because the district felt compelled to post this explanation of how they make the decision following this announcement of a fifth consecutive day of school closures which says, in part, that:
We had serious concerns about safe bus travel. We did not want children to be left standing at a bus stop in 13-degree weather because a bus was unable to safely travel on an icy road. Additionally, we did not want to risk having a bus try to travel on an icy road and getting into an accident. We simply could not guarantee safe transportation for EVERY student in MNPS.
The explanation isn't completely unreasonable. The pictures posted on the website make it quite apparent that there were, in fact, still some icy roads on Friday. But this doesn't mean the city and school district didn't drop the ball. There are at least two easy solutions that could -- and should -- have been implemented.
1.) Since the main roads were clear on Monday, that left most, if not all, of the snow clearing equipment free for the rest of the week. It, of course, costs money to salt other roads -- money that undoubtedly wasn't part of an already tight budget. But at some point in the next four days, somebody has to have thought to themselves that it would be worth saving 3 or 4 days of school to go salt a couple dozen troublesome spots. Since bus safety was the main issue, trucks could have simply driven the school bus routes and salted problem areas.
2.) MNPS officials should be aware after a similar episode last year that it can take a number of days for the snow/ice to melt from all the roads in the county. Given this knowledge, it would seem prudent to have alternate/emergency bus routes set up for days where only a few hilly, shaded roads are dangerous. For a couple of days each year, kids can be dropped off at bus stops slightly farther from home and still make it to school safely. Many public transportation systems (including Nashville's) have such alternate routes for their buses.
The end result is that it's only mid-January and Nashville has already used 7 snow days. They only have 4 built in, so they'll be attending school on President's Day and will tack on two extra days at the end of the year (after the state testing). More importantly, students will have attended exactly four days of school between December 17th and January 17th. I don't envy the teachers who have to re-establish classroom norms and get students back up to speed this Tuesday and for the rest of the week and month.
Now, this is not to say that it's always a bad idea to cancel school. Overly zealous decisions to keep school in session regardless of weather conditions can be harmful or, at best, a waste of time. During my two years in NYC, we had three days (including one where 19 inches of snow fell on Sunday) where virtually all of the surrounding suburban districts called off of school but NYC did not. On all three days, the majority of teachers and majority of students did not show up. The SOP for those days was for all present students and teachers to report directly to the cafeteria in the morning. The administrators then surveyed the scene and assigned students to teachers (on one day I had about a half-dozen kids from three different classes) and a day of babysitting, videos, and hangman ensued. Chancellor Klein undoubtedly thought he was doing the students right by preventing students from missing valuable instruction time, but those on the ground saw a different story unfold (though, to be fair, the kids who did show got free childcare and meals even if they didn't learn much of anything).
In the end, the Nashville situation is yet another example of ways that circumstances outside of the control of teachers and administrators can influence the achievement of students. Beyond that, it's an example of the myriad ways in which school districts and cities can influence the achievement of students beyond the standard examinations of pedagogy, curriculum, teacher quality, etc. When designing models to explain the achievement level of students, in addition to accounting for teacher, school, neighborhood, and family effects we may also need to include the ability of local governments to mitigate weather problems.
As a Northern transplant, I've often cynically joked that Nashville cancels one day of school for each inch of snowfall. Since I moved here, this the first time that the rule hasn't held.
Why would 3.6 inches of snow shut down a school system for an entire week? You might think it's because of the lack of snow clearing equipment in Southern cities, and you'd be partially right. But all the main roads had been plowed and salted by the end of the day Monday, and the vast majority of side roads were clear by Tuesday afternoon. So what gives? Well, a small fraction of the side roads -- including a number that school buses regularly travel -- remained covered with ice the entire week. This was the result of two things: 1.) temperatures that remained below freezing the entire week; and 2.) the fact that side roads, apparently, do not get salted or plowed in this city.
The continued cancellation of school day after day must've gotten somebody's attention, because the district felt compelled to post this explanation of how they make the decision following this announcement of a fifth consecutive day of school closures which says, in part, that:
We had serious concerns about safe bus travel. We did not want children to be left standing at a bus stop in 13-degree weather because a bus was unable to safely travel on an icy road. Additionally, we did not want to risk having a bus try to travel on an icy road and getting into an accident. We simply could not guarantee safe transportation for EVERY student in MNPS.
The explanation isn't completely unreasonable. The pictures posted on the website make it quite apparent that there were, in fact, still some icy roads on Friday. But this doesn't mean the city and school district didn't drop the ball. There are at least two easy solutions that could -- and should -- have been implemented.
1.) Since the main roads were clear on Monday, that left most, if not all, of the snow clearing equipment free for the rest of the week. It, of course, costs money to salt other roads -- money that undoubtedly wasn't part of an already tight budget. But at some point in the next four days, somebody has to have thought to themselves that it would be worth saving 3 or 4 days of school to go salt a couple dozen troublesome spots. Since bus safety was the main issue, trucks could have simply driven the school bus routes and salted problem areas.
2.) MNPS officials should be aware after a similar episode last year that it can take a number of days for the snow/ice to melt from all the roads in the county. Given this knowledge, it would seem prudent to have alternate/emergency bus routes set up for days where only a few hilly, shaded roads are dangerous. For a couple of days each year, kids can be dropped off at bus stops slightly farther from home and still make it to school safely. Many public transportation systems (including Nashville's) have such alternate routes for their buses.
The end result is that it's only mid-January and Nashville has already used 7 snow days. They only have 4 built in, so they'll be attending school on President's Day and will tack on two extra days at the end of the year (after the state testing). More importantly, students will have attended exactly four days of school between December 17th and January 17th. I don't envy the teachers who have to re-establish classroom norms and get students back up to speed this Tuesday and for the rest of the week and month.
Now, this is not to say that it's always a bad idea to cancel school. Overly zealous decisions to keep school in session regardless of weather conditions can be harmful or, at best, a waste of time. During my two years in NYC, we had three days (including one where 19 inches of snow fell on Sunday) where virtually all of the surrounding suburban districts called off of school but NYC did not. On all three days, the majority of teachers and majority of students did not show up. The SOP for those days was for all present students and teachers to report directly to the cafeteria in the morning. The administrators then surveyed the scene and assigned students to teachers (on one day I had about a half-dozen kids from three different classes) and a day of babysitting, videos, and hangman ensued. Chancellor Klein undoubtedly thought he was doing the students right by preventing students from missing valuable instruction time, but those on the ground saw a different story unfold (though, to be fair, the kids who did show got free childcare and meals even if they didn't learn much of anything).
In the end, the Nashville situation is yet another example of ways that circumstances outside of the control of teachers and administrators can influence the achievement of students. Beyond that, it's an example of the myriad ways in which school districts and cities can influence the achievement of students beyond the standard examinations of pedagogy, curriculum, teacher quality, etc. When designing models to explain the achievement level of students, in addition to accounting for teacher, school, neighborhood, and family effects we may also need to include the ability of local governments to mitigate weather problems.
Tuesday, December 14, 2010
The Times' Tenure Math Problem
The NY Times today profiles some fairly reasonable, and minor, changes to the way tenure is granted to teachers in NYC (principals a.) have to describe why they're giving a teacher tenure and b.) are able to select from four ratings that can award, delay, or deny teachers tenure).
On the surface, those make sense, and the article seems to do a pretty good job describing them. But I have a problem with the math on which the article is premised. The second sentence of the article states that "virtually every new teacher earns" tenure, and then later cites a statistic that "6,400 teachers who were eligible for tenure, 234 teachers were denied it, or roughly 3.7 percent".
The problem here is that a "new teacher" and a teacher "eligible for tenure" are two very different things. Writing that 96% of those eligible are awarded tenure is very different from writing that "virtually every new teacher earns" tenure. Why? To start with, NYC has a high attrition rate -- particularly among new teachers. Nationwide, around 50% of teachers in urban schools leave within their first three years. Let's say that that statistic is precisely correct for the teachers that entered NYC schools in 2007; the correct calculation would then be that 46% of new teachers who began teaching in 2007 were awarded tenure in 2010 -- which looks like very different. In other words, the statement that "virtually every new teacher earns" tenure is demonstrably false.
Now, the 46% statistic wouldn't be all that helpful in judging the rigor of the tenure process because new teachers leave for all sorts of reasons. Some leave because their principals threaten to fire them if they don't; some leave because they simply prefer a different district or profession; some leave because they're simply unhappy; and some leave for other reasons or a combination of reasons (e.g. family relocation, pregnancy, health, etc.).
So, leaving aside the silly -- but important -- syntax error by the Times, what would actually be the actual best way to compute how many teachers earn tenure? There are a few different ways depending on exactly what the question is that one wants to answer, but I think you'd have to start by defining a population of teachers who want to earn tenure at some point during their teaching career (in NYC that would mean they want to teach in the city for more than 3 years). Exactly who counts in that group is up for debate: for example, would we include a teacher who wants to make it a career during year 1 but finds that he/she just doesn't like teaching during year 2? There's no clear answer. I would certainly include, however, any teachers who leave explicitly because they believe they will receive a negative rating if they don't -- something that happens all over the place but never seems to get mentioned in the press and only rarely in academic work (I've heard economists reference "the dance of the lemons," to describe teachers who transfer to other schools to avoid being fired). The estimate would shed some light on how many unsuccessful teachers -- teachers who might not receive tenure if they went through the process -- are leaving prior to starting their 4th year.
Whatever parameters one set, the answer to how many teachers tried to get tenure and didn't would be somewhere between the 46% estimate above and the 96% statistic cited by the NY Times. The more practical questions to answer, however, would really be in estimating how many people would be valuable as tenured teachers but don't end up teaching more than 3 years for whatever reason and how many are not particularly valuable as tenured teachers but earn tenure anyway for any number of reasons (my guess is that the former group is larger than the latter, though there are also probably quite a few in the former who wouldn't stay in teaching almost no matter what), and then figuring out how to get more in the former group to stay and more in the latter group to leave.
At any rate, I don't think there's much disagreement that tenure is too easy to earn in NYC (personally I'd argue for a more meaningful process that's awarded a little later in one's career, and matched by some added distinction that's meaningful to teachers), so I don't want readers to think that this critique of the Times' math problem is really a veiled attack on any who would dare insinuate that all teachers aren't perfect. But the Times should know better than to make such silly mistakes. And I hope that readers know better than to accept such superficial calculations as gospel.
Do "virtually all new teacher earn" tenure? No. Do almost all teachers who go up for tenure review earn tenure? Yes. But those are two very different statistics -- and neither really answers the more important questions of how many excellent teachers don't receive tenure and how many poor teachers do.
On the surface, those make sense, and the article seems to do a pretty good job describing them. But I have a problem with the math on which the article is premised. The second sentence of the article states that "virtually every new teacher earns" tenure, and then later cites a statistic that "6,400 teachers who were eligible for tenure, 234 teachers were denied it, or roughly 3.7 percent".
The problem here is that a "new teacher" and a teacher "eligible for tenure" are two very different things. Writing that 96% of those eligible are awarded tenure is very different from writing that "virtually every new teacher earns" tenure. Why? To start with, NYC has a high attrition rate -- particularly among new teachers. Nationwide, around 50% of teachers in urban schools leave within their first three years. Let's say that that statistic is precisely correct for the teachers that entered NYC schools in 2007; the correct calculation would then be that 46% of new teachers who began teaching in 2007 were awarded tenure in 2010 -- which looks like very different. In other words, the statement that "virtually every new teacher earns" tenure is demonstrably false.
Now, the 46% statistic wouldn't be all that helpful in judging the rigor of the tenure process because new teachers leave for all sorts of reasons. Some leave because their principals threaten to fire them if they don't; some leave because they simply prefer a different district or profession; some leave because they're simply unhappy; and some leave for other reasons or a combination of reasons (e.g. family relocation, pregnancy, health, etc.).
So, leaving aside the silly -- but important -- syntax error by the Times, what would actually be the actual best way to compute how many teachers earn tenure? There are a few different ways depending on exactly what the question is that one wants to answer, but I think you'd have to start by defining a population of teachers who want to earn tenure at some point during their teaching career (in NYC that would mean they want to teach in the city for more than 3 years). Exactly who counts in that group is up for debate: for example, would we include a teacher who wants to make it a career during year 1 but finds that he/she just doesn't like teaching during year 2? There's no clear answer. I would certainly include, however, any teachers who leave explicitly because they believe they will receive a negative rating if they don't -- something that happens all over the place but never seems to get mentioned in the press and only rarely in academic work (I've heard economists reference "the dance of the lemons," to describe teachers who transfer to other schools to avoid being fired). The estimate would shed some light on how many unsuccessful teachers -- teachers who might not receive tenure if they went through the process -- are leaving prior to starting their 4th year.
Whatever parameters one set, the answer to how many teachers tried to get tenure and didn't would be somewhere between the 46% estimate above and the 96% statistic cited by the NY Times. The more practical questions to answer, however, would really be in estimating how many people would be valuable as tenured teachers but don't end up teaching more than 3 years for whatever reason and how many are not particularly valuable as tenured teachers but earn tenure anyway for any number of reasons (my guess is that the former group is larger than the latter, though there are also probably quite a few in the former who wouldn't stay in teaching almost no matter what), and then figuring out how to get more in the former group to stay and more in the latter group to leave.
At any rate, I don't think there's much disagreement that tenure is too easy to earn in NYC (personally I'd argue for a more meaningful process that's awarded a little later in one's career, and matched by some added distinction that's meaningful to teachers), so I don't want readers to think that this critique of the Times' math problem is really a veiled attack on any who would dare insinuate that all teachers aren't perfect. But the Times should know better than to make such silly mistakes. And I hope that readers know better than to accept such superficial calculations as gospel.
Do "virtually all new teacher earn" tenure? No. Do almost all teachers who go up for tenure review earn tenure? Yes. But those are two very different statistics -- and neither really answers the more important questions of how many excellent teachers don't receive tenure and how many poor teachers do.
Thursday, December 9, 2010
Remember: Not Everyone Prioritizes Achievement
The commentariat seems to agree on two things regarding American schools: that they're not good enough and that students need to be learning more. Read a random blog or watch a random talk show and you'll likely see or hear something about more rigorous curricula, longer hours, longer school years, or some sort of argument that our students need to be pushed harder.
And you're twice as likely to see or hear these things after the results of an international assessment are released (e.g. the latest PISA results this week) and the U.S. again ranks behind a lot of countries.
But we in the education world can sometimes forget that there are other things that matter too. It would be wise for those driving this commentary to remember that not everybody shares their priorities and keep in mind that many people in our country have no interest in spending more time in school. I'd divide this constituency into three different groups:
1.) People who absolutely detest school and can't wait for the bell to ring. Witness this cartoon comparing school to jail . . . with the only difference being that the disgusting food offered in jails is always free. For kids who believe school is tantamount to jail or torture, it's unclear how exposing them to more misery will improve their lives. In other words, it may be more important to find a type of schooling that they only hate a little bit than to simply pile on more.
2.) Families with students enrolled in elite, high-pressure schools (usually private or suburban). Witness the number of parents excited by the new movie "Race to Nowhere" that chronicles the downside of such schools. Indeed, the most obvious sign of the difference between classes when it comes to schooling may be the broad push for more rigorous schooling for low-income students while high-income parents increasingly worry about the amount of stress their children endure while running five clubs, taking 4 AP classes, and waking up early for SAT prep class on Saturday before going out to volunteer at the local animal shelter.
3.) Those whose priorities lie elsewhere -- e.g. sports. Witness the saga of Urban Meyer, the University of Florida's highly successful football coach, who stepped down yesterday -- at the age of 46 -- because he wanted to spend more time with his family. Did the press release say that he wanted to help them with their homework or volunteer in their classrooms? No. It said that "after spending more than two decades motivating and celebrating the young men I've been so proud to coach, I relish the opportunity to cheer for my three terrific kids as they compete in their own respective sports". For other families it's music, art, drama, religion, or any of a million other activities that evoke more passion.
There's some overlap, of course, between these groups. But an awful lot of students/parents/families fall into at least one of these groups. This certainly doesn't mean that we shouldn't examine longer days/years or such reforms to boost our national achievement levels. But it does mean that we shouldn't assume that these types of changes would automatically be popular or successful. And it illustrates some of the trade-offs of these types of reforms.
In a number of the highest scoring countries, many kids spend countless hours in "cram schools" after school and on weekends -- essentially doing test prep. American kids, on the other hand, are more likely to go to football practice, piano lessons, or their part-time job after school. There's no objective answer as to which practice is better, but time is finite and kids can't do everything. And many, many parents would prefer that their kids choose from the latter list.
So, by all means, we should be examining ways in which we can boost our national achievement. But boosting these test scores isn't the only thing that the citizens of our country care about. As such, any reform designed to boost these scores will not automatically be accepted by all. In short, a more productive discussion would weigh the costs and benefits of emphasizing different areas rather than simply assuming that everybody wants higher test scores at whatever cost.
And you're twice as likely to see or hear these things after the results of an international assessment are released (e.g. the latest PISA results this week) and the U.S. again ranks behind a lot of countries.
But we in the education world can sometimes forget that there are other things that matter too. It would be wise for those driving this commentary to remember that not everybody shares their priorities and keep in mind that many people in our country have no interest in spending more time in school. I'd divide this constituency into three different groups:
1.) People who absolutely detest school and can't wait for the bell to ring. Witness this cartoon comparing school to jail . . . with the only difference being that the disgusting food offered in jails is always free. For kids who believe school is tantamount to jail or torture, it's unclear how exposing them to more misery will improve their lives. In other words, it may be more important to find a type of schooling that they only hate a little bit than to simply pile on more.
2.) Families with students enrolled in elite, high-pressure schools (usually private or suburban). Witness the number of parents excited by the new movie "Race to Nowhere" that chronicles the downside of such schools. Indeed, the most obvious sign of the difference between classes when it comes to schooling may be the broad push for more rigorous schooling for low-income students while high-income parents increasingly worry about the amount of stress their children endure while running five clubs, taking 4 AP classes, and waking up early for SAT prep class on Saturday before going out to volunteer at the local animal shelter.
3.) Those whose priorities lie elsewhere -- e.g. sports. Witness the saga of Urban Meyer, the University of Florida's highly successful football coach, who stepped down yesterday -- at the age of 46 -- because he wanted to spend more time with his family. Did the press release say that he wanted to help them with their homework or volunteer in their classrooms? No. It said that "after spending more than two decades motivating and celebrating the young men I've been so proud to coach, I relish the opportunity to cheer for my three terrific kids as they compete in their own respective sports". For other families it's music, art, drama, religion, or any of a million other activities that evoke more passion.
There's some overlap, of course, between these groups. But an awful lot of students/parents/families fall into at least one of these groups. This certainly doesn't mean that we shouldn't examine longer days/years or such reforms to boost our national achievement levels. But it does mean that we shouldn't assume that these types of changes would automatically be popular or successful. And it illustrates some of the trade-offs of these types of reforms.
In a number of the highest scoring countries, many kids spend countless hours in "cram schools" after school and on weekends -- essentially doing test prep. American kids, on the other hand, are more likely to go to football practice, piano lessons, or their part-time job after school. There's no objective answer as to which practice is better, but time is finite and kids can't do everything. And many, many parents would prefer that their kids choose from the latter list.
So, by all means, we should be examining ways in which we can boost our national achievement. But boosting these test scores isn't the only thing that the citizens of our country care about. As such, any reform designed to boost these scores will not automatically be accepted by all. In short, a more productive discussion would weigh the costs and benefits of emphasizing different areas rather than simply assuming that everybody wants higher test scores at whatever cost.
Are Superintendant Searches a Thing of the Past?
After NYC hired an outsider (Joel Klein) to run their schools, Pittsburgh decided to hire an outsider as well (Mark Roosevelt). Both leaders were controversial, but can claim some successes. And both were generally liked more by policy wonks than by teachers or parents.
When NYC decided to replace Klein this year, the process was done behind closed doors and -- as far as anyone can tell -- exactly one candidate was interviewed (update: apparently, at least one other person may have been offered the job). The end result is a prospective new chancellor (Cathleen Black) that may be even more strongly disliked by parents and teachers. When Pittsburgh needed to replace their superintendent this year, meetings were held behind closed doors and exactly one candidate was interviewed. The announcement was made today that the board will hire Deputy Superintendant Linda Lane as the next leader of Pittsburgh Public Schools on Monday. It remains to be seen how parents and teachers will react to the news, but it seems unlikely that the move was made to assuage their concerns.
There, are, however, three major differences:
1.) Lane has spent almost 40 years working as a teacher and administrator in public schools, while it was unclear whether Black had even ever set foot in a public school when her hiring was announced.
2.) While Bloomberg was evasive and seemed to insinuate that others were interviewed for the position, the Pittsburgh Board of Education is being upfront about it. Said the board president: "What would be the point of spending the money to do the search when what we wanted was right here?"
3.) Many in the press and public turned on Bloomberg after his announcement. Judging by the glowing article in the Pittsburgh Post-Gazette, however, Lane is apparently already beloved by the press. It will be interesting to see if the public feels the same way. Despite the glowing review, it does seem somewhat unlikely that a board of ed that didn't even feel the need to pretend that they were conducting a thorough search would emerge completely unscathed.
When NYC decided to replace Klein this year, the process was done behind closed doors and -- as far as anyone can tell -- exactly one candidate was interviewed (update: apparently, at least one other person may have been offered the job). The end result is a prospective new chancellor (Cathleen Black) that may be even more strongly disliked by parents and teachers. When Pittsburgh needed to replace their superintendent this year, meetings were held behind closed doors and exactly one candidate was interviewed. The announcement was made today that the board will hire Deputy Superintendant Linda Lane as the next leader of Pittsburgh Public Schools on Monday. It remains to be seen how parents and teachers will react to the news, but it seems unlikely that the move was made to assuage their concerns.
There, are, however, three major differences:
1.) Lane has spent almost 40 years working as a teacher and administrator in public schools, while it was unclear whether Black had even ever set foot in a public school when her hiring was announced.
2.) While Bloomberg was evasive and seemed to insinuate that others were interviewed for the position, the Pittsburgh Board of Education is being upfront about it. Said the board president: "What would be the point of spending the money to do the search when what we wanted was right here?"
3.) Many in the press and public turned on Bloomberg after his announcement. Judging by the glowing article in the Pittsburgh Post-Gazette, however, Lane is apparently already beloved by the press. It will be interesting to see if the public feels the same way. Despite the glowing review, it does seem somewhat unlikely that a board of ed that didn't even feel the need to pretend that they were conducting a thorough search would emerge completely unscathed.
Thursday, November 18, 2010
Value-Added vs. Batting Averages
Stephen Sawchuck reports that a new report on value-added scores compares them to measures in other fields, including batting averages in baseball. I think the batting average analogy is a good one. Here's why:
When we look at a player's batting average (the number of hits divided by the number of at bats), we are provided with some information that reflects how good they are at baseball. The value of this information increases when the player has more at bats and plays for more years. And the larger the gap is between two players' batting averages, the more certain we can be that one player is better than the other.
But, at the same time, a player's batting average in one given year gives us only a small amount of information about that player's ability. Averages vary year to year: players suffer injuries, become distracted, grow older, switch teams, play with different teammates, and so on. For example:
Who's a better shortstop: Alexei Ramirez or Derek Jeter? The former hit .282 this year while the latter hit .270.
Who's a better first baseman: Aubrey Huff or Ryan Howard? The former hit .290; the latter .277.
Batting average doesn't do a good job of measuring power, speed, defense, leadership, or any of a million other desirable characteristics. And for that reason, teams have moved beyond batting average when evaluating and signing players. It seems like every year a new stat bubbles up that measures a player's ability in one facet of the game.
Adam Dunn is a career .250 hitter who plays awful defense, but will make millions of dollars next year because he'll probably hit 40 home runs yet again.
Mike Cameron, a .250 lifetime hitter, signed a 2-year, $15 million dollar contract last year as a 36 year-old. His power (about 25 HR per season recently) didn't hurt, but the main motivation was his defense.
When we look at a more sophisticated (though still flawed) computation of how many wins a player added over an average replacement player (Wins Above Replacement, or WAR), we see that Jose Bautista, who hit .260 this year, ranks sixth. The fifh-ranked player, Adrian Beltre, has a lifetime average of .275.
At the same time, nobody would argue that batting average is meaningless -- especially over longer periods of time. There are plenty of hall of famers with career averages over .300, but none that I know of with averages of .220.
In other words, the batting average analogy is an excellent one for value-added scores because they have four very important things in common:
1.) Anybody who tells you that it is totally meaningless is totally wrong.
2.) When you see large differences over long periods of time between two people, you can be pretty sure that the one with the larger number is better.
3.) At the same time, that number in and of itself gives us very little information about how good somebody is, particularly over a shorter period of time.
4.) There are other skills not measured by the number that need to be taken into account when evaluating the person.
Unlike baseball, we don't have better statistics in evaluation. We don't have something like on base percentage, yet alone change in student motivation over replacement teacher.
What does this mean for schools and value-added scores? If we designed the perfect evaluation system, given current knowledge and tools, value-added scores would have to be included. The only really compelling reason to leave them out is the potential for misuse by people who don't understand #'s 2-4 above. Partly for that reason, it's important that value-added scores be only one component of how a teacher is evaluated. And principals should be looking for large differences over long periods of time, not tiny year-to-year differences when making hiring/firing/tenure decisions.
Opponents of value-added scores would be better off arguing that they can be useful -- but only in some circumstances -- while proponents should be looking for supplemental measures to boost the meaningfulness of evaluations. And both sides should keep in mind that the majority of teachers don't teach subjects tested by state tests anyway, so in addition to measuring a teacher's impact on students other than on standardized tests (e.g. motivation, attitude, self-control, creativity, interpersonal skills, etc.) we should also keep in mind that we need better ways to evaluate a teacher's impact on students in other subjects beyond 3rd-8th grade English/math as well.
In short, value-added scores can aid our evaluations of teachers . . . but only a little bit, and only if used properly.
When we look at a player's batting average (the number of hits divided by the number of at bats), we are provided with some information that reflects how good they are at baseball. The value of this information increases when the player has more at bats and plays for more years. And the larger the gap is between two players' batting averages, the more certain we can be that one player is better than the other.
But, at the same time, a player's batting average in one given year gives us only a small amount of information about that player's ability. Averages vary year to year: players suffer injuries, become distracted, grow older, switch teams, play with different teammates, and so on. For example:
Who's a better shortstop: Alexei Ramirez or Derek Jeter? The former hit .282 this year while the latter hit .270.
Who's a better first baseman: Aubrey Huff or Ryan Howard? The former hit .290; the latter .277.
Batting average doesn't do a good job of measuring power, speed, defense, leadership, or any of a million other desirable characteristics. And for that reason, teams have moved beyond batting average when evaluating and signing players. It seems like every year a new stat bubbles up that measures a player's ability in one facet of the game.
Adam Dunn is a career .250 hitter who plays awful defense, but will make millions of dollars next year because he'll probably hit 40 home runs yet again.
Mike Cameron, a .250 lifetime hitter, signed a 2-year, $15 million dollar contract last year as a 36 year-old. His power (about 25 HR per season recently) didn't hurt, but the main motivation was his defense.
When we look at a more sophisticated (though still flawed) computation of how many wins a player added over an average replacement player (Wins Above Replacement, or WAR), we see that Jose Bautista, who hit .260 this year, ranks sixth. The fifh-ranked player, Adrian Beltre, has a lifetime average of .275.
At the same time, nobody would argue that batting average is meaningless -- especially over longer periods of time. There are plenty of hall of famers with career averages over .300, but none that I know of with averages of .220.
In other words, the batting average analogy is an excellent one for value-added scores because they have four very important things in common:
1.) Anybody who tells you that it is totally meaningless is totally wrong.
2.) When you see large differences over long periods of time between two people, you can be pretty sure that the one with the larger number is better.
3.) At the same time, that number in and of itself gives us very little information about how good somebody is, particularly over a shorter period of time.
4.) There are other skills not measured by the number that need to be taken into account when evaluating the person.
Unlike baseball, we don't have better statistics in evaluation. We don't have something like on base percentage, yet alone change in student motivation over replacement teacher.
What does this mean for schools and value-added scores? If we designed the perfect evaluation system, given current knowledge and tools, value-added scores would have to be included. The only really compelling reason to leave them out is the potential for misuse by people who don't understand #'s 2-4 above. Partly for that reason, it's important that value-added scores be only one component of how a teacher is evaluated. And principals should be looking for large differences over long periods of time, not tiny year-to-year differences when making hiring/firing/tenure decisions.
Opponents of value-added scores would be better off arguing that they can be useful -- but only in some circumstances -- while proponents should be looking for supplemental measures to boost the meaningfulness of evaluations. And both sides should keep in mind that the majority of teachers don't teach subjects tested by state tests anyway, so in addition to measuring a teacher's impact on students other than on standardized tests (e.g. motivation, attitude, self-control, creativity, interpersonal skills, etc.) we should also keep in mind that we need better ways to evaluate a teacher's impact on students in other subjects beyond 3rd-8th grade English/math as well.
In short, value-added scores can aid our evaluations of teachers . . . but only a little bit, and only if used properly.
Tuesday, November 16, 2010
"The Shadow Scholar"
In case you missed it, The Chronicle of Higher Education recently published an article written by "Ed Dante" (a psuedonym) about his/her experience writing papers for desperate/lazy college students. I'd put "The Shadow Scholar" in the "must read" category for all those interested in higher education, and in the "well worth your time" category for everybody else.
The author makes decent money (on pace for 66K this year) writing papers for students and says that a student has never complained about getting caught. The process and stories are interesting, but the author makes two main points:
1.) A lot of students are absolutely awful at writing and more should be done about this.
2.) A lot of assignments are ridiculous and drive students to look for other solutions (including purchasing papers)
I wholeheartedly agree with the first and half agree with the second.
The comments are almost as interesting as (though more tedious than) the article itself. Between the author, the commenters, and my own head, there seem to be some competing hypotheses as to why a not insignificant number of students cheat in one way or another, particularly on written assignments. In no particular order, they are:
1.) Professors assign meaningless/useless/stupid assignments
Why?
-large class sizes prevent individualized assignments
-laziness
-following conventions/norms
-more concerned with research than teaching
2.) Students don't know how to write
Why?
-they're non-native speakers
-they didn't learn how to write during K-12 schooling
-their college profs don't take the time to teach them how to write
3.) Students are unethical
Why?
-they don't understand what's ethical and what's not
-ethics and morals aren't emphasized enough in school
-a few bad apples make everyone look bad
-lots of pressure to get good grades; little pressure to behave ethically
-they don't value the assignments they're asked to complete or the courses they must take
4.) Unethical people make it easy to cheat
Why?
-some people care more about money than ethics
-people think they're doing others a service since they think the assignments are stupid
I don't think any of these fully explain the situation, but I do think they all shed a little bit of light on it. In short, there are plenty of people and institutions at whom we can point fingers . . . but wouldn't it be more productive if we instead focused on fixing the problem?
The author makes decent money (on pace for 66K this year) writing papers for students and says that a student has never complained about getting caught. The process and stories are interesting, but the author makes two main points:
1.) A lot of students are absolutely awful at writing and more should be done about this.
2.) A lot of assignments are ridiculous and drive students to look for other solutions (including purchasing papers)
I wholeheartedly agree with the first and half agree with the second.
The comments are almost as interesting as (though more tedious than) the article itself. Between the author, the commenters, and my own head, there seem to be some competing hypotheses as to why a not insignificant number of students cheat in one way or another, particularly on written assignments. In no particular order, they are:
1.) Professors assign meaningless/useless/stupid assignments
Why?
-large class sizes prevent individualized assignments
-laziness
-following conventions/norms
-more concerned with research than teaching
2.) Students don't know how to write
Why?
-they're non-native speakers
-they didn't learn how to write during K-12 schooling
-their college profs don't take the time to teach them how to write
3.) Students are unethical
Why?
-they don't understand what's ethical and what's not
-ethics and morals aren't emphasized enough in school
-a few bad apples make everyone look bad
-lots of pressure to get good grades; little pressure to behave ethically
-they don't value the assignments they're asked to complete or the courses they must take
4.) Unethical people make it easy to cheat
Why?
-some people care more about money than ethics
-people think they're doing others a service since they think the assignments are stupid
I don't think any of these fully explain the situation, but I do think they all shed a little bit of light on it. In short, there are plenty of people and institutions at whom we can point fingers . . . but wouldn't it be more productive if we instead focused on fixing the problem?
Subscribe to:
Posts (Atom)