*Just* A Kindergarten Teacher

Tonight's episode of How I Met Your Mother features an argument between Marshall and Lily about her career that speaks volumes about the current state of America's educational system.  (In case you don't watch the show, Marshall and Lily are college sweethearts who are now married thirty-somethings with a baby -- Marshall is a lawyer and Lily is a kindergarten teacher).

Lily is upset because she's told by an acquaintance that she's "just a kindergarten teacher" and the following exchange ensues (video at the bottom):

Marshall: Oh my God! Lily! What is the big deal?  Ok, so what? So he said you were just a kindergarten teacher.  Why do you let that bother you?
Lily: Because he was right; I am just a kindergarten teacher.  And, yes, I have a degree in art history, and I was meant to do something with it -- but I didn't. Somewhere along the line I forgot to pursue my dream and, and now I'm old, and I'm a Mom, and it's just too late for me.

At this point -- particularly knowing the cutesy relationship they have and how much Marshall adores Lily -- I expected Marshall to respond by saying something like "Lily, that's one of my favorite things about you: few people are more important or incredible than kindergarten teachers"

Marshall instead responds by emphatically saying "No, it's not too late. You're going to quit your job, tomorrow, and you're gonna go back and pick up right where you left off with that art stuff . . ."

Maybe I'm overreacting to a few moments in a sitcom, but this seems indicative of one of the largest problems with our efforts to improve our educational system.  Virtually all the reform efforts of the past few years have focused on teacher quality because everybody agrees it's so important; but nobody's willing to actually treat teachers like they're important.

After all, who's going to want to be just a teacher?  Certainly not the best and the brightest.  And what teacher is going to be empowered or respected enough to change the system if teachers are viewed as second-class citizens?  If we want to recruit, retain, and develop the best teaching corps in the world (like we say we do), we can't keep demeaning and demoralizing them.  If we're going to justify every new pet policy (which always seem to place teachers under even more scrutiny) by talking of teachers' vast importance, we can't then act like they aren't worthy of our attention.

Our teachers deserve better.  Our kids deserve better.  And our country deserves better.  And we won't get it if quitting is the only way for teachers to reach their potential.


Cross-Posted at Blog of the Century

Here's the clip, but the quality is poor -- you'll probably have to turn up your volume to hear anything.

UPDATE: The video has been blocked by FOX, who made a copyright claim. A 36-second clip sure seems like fair use to me, so this distresses me (and I'm not sure why FOX would claim a show that aired on CBS). YouTube says I get a "copyright strike" if I challenge and lose, so I'm going to let this one go unless somebody knows a copyright lawyer who can advise me whether I'm in the right or not.

Is Teacher Quality Really Causing the Achievement Gap?

Yesterday, the NY Times released the value-added scores of thousands of teachers over the past five years.  Before and immediately after the release, people seemed to mostly argue the merit of the decision to release the data.  But I have a substantive question about the data.

What caught my eye was that, according to the NYT analysis of this particular set of scores, good teachers are evenly distributed between high-poverty and low-poverty schools.  From the article:

there was no relationship between a school’s demographics and its number of high- or low-performing teachers: 26 percent of math teachers serving the poorest of students had high scores, as did 27 percent of teachers of the wealthiest.

The LA Times reported a roughly a similar situation in LA when they released teachers' scores a couple years ago.  Which is really quite shocking in a number of ways.  Most notably, researchers and practitioners have long assumed that lower-poverty schools had worse teachers than higher-poverty schools -- past studies have repeatedly found that teachers in high-poverty schools are less experienced, turn over at a much higher rate, score lower on achievement tests, attend less selective colleges, etc.  Accordingly, at least part of the theory of action behind the teacher quality movement has been that giving low-income students teachers who are as good as or better than those in higher-income schools would significantly narrow the achievement gap.

But these two measures of teacher quality indicate there may be no major differences between low- and high-poverty schools, while we know that large gaps in achievement still exist between low-income and high-income students.  Which means at least one of two things.

1.) Differences in teacher quality are not a major driver of the achievement gap.

2.) These value-added scores are not a good measure of teacher quality.

I don't think anybody seriously doubts -- or at least that anybody serious doubts -- that some teachers are much better than others and that the best teachers can make a large difference.  But if quality teachers, according to these value-added measure, are roughly evenly distributed between high- and low-poverty schools in LA at the same time that we see differences between high- and low-income students growing, then  improving the quality of teachers (again,as indicated by these value-added measures) in high-poverty schools seems unlikely to close the achievement gap.  Either other factors influence achievement far more, the effects of quality teachers on students are much less direct than many assume, or what we're measuring isn't what matters.

In short, these data indicate that we need to broaden our focus beyond teacher quality and/or re-evaluate the way we're currently measuring teacher quality.

The Logistics of "Thinning Out" Bad Teachers

Nick Kristof recently wrote another column calling for more high-quality teachers based on the latest paper on value-added measures of teacher quality.  There's a whole lot to discuss about both the column and the research paper, but let me focus for a minute on one small part of it.

Near the end of the column, Kristof writes that "If we want to recruit and retain the best teachers, we simply have to pay more — while also more aggressively thinning out those who don’t succeed. It’s worth it."  Recruiting, retaining, paying (and training, which is left out of this sentence) are all complex endeavors, but the "thinning out" part of the equation is often taken for granted.

Here's my question for Kristof: even if (and that's a big if) we can find a fair, accurate, and agreeable way to identify and dismiss the worst teachers, how many teachers are we actually going to dismiss in such a scenario?

The first question would obviously be whether we need to fire the bottom 5%, 10%, 25% or some other number.  That's up for discussion.

But the logistical question then, is how many teachers among the bottom X% 1.) can be readily identified and 2.) are planning on teaching again next year.  This will differ greatly by school and district, but in some places, this is going to be a very small number.Why?  Let's take a look at what the research says.

First, research consistently finds that it takes 3-5 years for a teacher to reach their potential.  So a good number of the lowest-performing teachers are simply going to be novices who will be better teachers next year.  We don't want to fire a first-year teacher who was in the bottom X% if we have reason to believe they'll be a really good teacher in a couple of years.  That would be incredibly counterproductive.

Second, research has consistently found that value-added measures of teacher effectiveness bounce around considerably from year to year -- particularly for teachers who teach a small number of students (e.g. a 4th grade reading teacher with 18 students versus an 8th grade math teacher with 150 students).  At least one paper has found that averaging scores over three years provides a much better, and more stable, estimate of teacher performance than does any single-year estimate.

Third, a number of recent papers have found that, at least in the first few years, many of the least successful teachers exit teaching.  This makes sense -- if you start a new career and find yourself completely overwhelmed, you're not likely stay very long.

Fourth, teacher attrition is exceedingly high in many high-poverty schools.  The general consensus is that about half of urban teachers leave the field within their first 3 years.

So, what does this mean?  We probably don't want to fire a whole lot of teachers in the first 3-5 years of their career because a.) they're still learning and improving; b.) we can't be that sure who the worst teachers are anyway; and c.) a good portion of the catastrophically bad teachers are self-selecting out of the field anyway.  If we give discount the first two years, when teachers are still learning their craft, and then take three more years to compute accurate value-added scores, it would only be teachers who'd taught for 5+ years who would really be ripe for firing due to low value-added scores.

Which means that the main herd we're trying to thin is the teachers who've made it through those first few years, reached their potential, and for whom we have accurate value-added estimates.  But how many teachers is that?  When I looked at high-poverty NYC middle schools a few years, I found that in the average school, only 1/3 of teachers had 5 or more years of experience.

Let's say that we're very confident in our ability to recruit and retain teachers who are better than our current teaching force and so we decide to fire all below average teachers (a full 50%) -- which would be a far more aggressive plan than any I've seen proposed.  First, the majority of these below average teachers are novices who are still improving and for whom we don't have particularly good estimates of ability.  Given that the majority of struggling beginning teachers either improve or self-select out of the profession, let's estimate that 2/3 of all teachers in their first 5 years are identified as below average teachers.  This would mean that only 1/6 of all teachers in their sixth year and beyond are below average teachers.  And since only 1/3 of teachers are in their sixth year or beyond, this would mean that only 1/18 of all teachers would both have 5+ years of experience and be rated below average.  This is a little under 6% of all teachers.

The average school in my sample had 72 teachers.  So, that's the equivalent of firing four teachers.  And that's under an extremely aggressive scenario.  Besides, now that you've rid your school of the chaff, who, exactly, do you want to fire next year?  And if you want to argue that we could be more aggressive and fire some of the novice teachers, that would mean there'd be fewer low-performing experienced teachers (since teachers tend to be roughly equally effective pre- and post-tenure).  So, for now, let's stick with the assumption that, under an aggressive plan, we'd fire four teachers this year in the average school.

Now, other districts have far more experienced teachers.  And it might make more of a dent there.  But a good number of our poorest-performing schools and districts are quickly churning through teachers too fast for firing low performers to make much of an impact.  Certainly, we should make every effort to rid our schools of the worst teachers (by increasing the performance of, and/or dismissing the lowest performers) -- I don't think anybody seriously disputes that notion.  Or, at the very least, I don't think anybody serious disputes that notion.  But will firing the lowest performing 6% of teachers in high-poverty NYC schools make a difference?  It's possible.  But let's be reasonable -- it's not going to make much of a difference.

So, yes, let's work harder to rid our schools of the worst teachers.  But let's not pretend it will be easy to do.  And, perhaps more importantly, let's not hold our breath while we wait to see if that bullet is actually silver.  In most places, other problems loom far larger.

Education Roundtable: Causes of Teaching Quality

Corey: Last time, we discussed the effects of under-performing principals versus under-performing teachers in schools, including some ways in which the former can result in the latter. Which brings me to something that’s been rattling around in my head lately: the causes of teaching quality. The dialogue on teacher/teaching quality often seems to treat it as something that’s both quantifiable and immutable. We talk about the need for principals to hire "good teachers" and fire "bad teachers" as though people are "good" or "bad" teachers much like they’re short or tall. While some certainly have more natural talent for teaching, my guess is that the actual teaching quality to which students are exposed to in a classroom has more to do with the context of the classroom and the school than it does with the natural abilities of the instructor.

I’ve included a rough list of some of these factors below, including teacher characteristics (observable traits), classroom context (which may influenced by differing degrees by teachers and principals in different schools), and school characteristics (which are typically more heavily influenced by principals than teachers). Most notable, to me, is that all of the items I listed under “teacher characteristics” are malleable and can be influenced by principals (and, of course, teacher training programs). Hence, my belief that principals have as much do to with teaching quality as do teachers -- and their ability to hire and fire teachers is only a small part of that. My intent isn’t to shift the discussion from blaming teachers to blaming principals but, rather, to better think through how to expose more students to high-quality teaching.

Teacher Characteristics:
Knowledge
Interpersonal Skills/Charisma
Motivation/Effort

Classroom Context:
Curriculum
Textbooks
Other Materials
Subject Matter
Student Characteristics

School Characteristics:
School Climate
School Discipline System
Teacher Colleagues
Professional Development
Student Demographics
Planning Time

ClassroomView: I don’t think I have anything to add to your factors, Corey. However, I would certainly reverse the order of importance. I would place school characteristics first, teacher characteristics second, and classroom context third. As someone who has taught for a while, I cannot emphasize enough the role that overall school culture plays in effective instruction. Yes, quality teaching is important, but the whole building needs to reflect a high level of both caring and competence for all students.

CEP: I would add teacher autonomy in there, whether at the classroom context level or school characteristic level or both, I’m not sure. When I was in the classroom, on the one hand, it was my characteristics and the classroom context (including a fairly high level of autonomy, at least as far as no one paid attention to what I was doing as long as I had good test scores) that made me an effective teacher, it was ultimately the school characteristics that drove me out of the classroom as they were not working together to make me a more effective instructor.

Corey: I'm not sure I have much more to add other than to remark on my surprise at the lack of objections to my postulation among those in this group.  I suspect that others may take more issue with the idea that teaching quality is influenced more by contextual factors than a teacher's underlying abilities.

Education Roundtable: Increasing Teaching Quality

Corey: Last week, we discussed how to deal with disaffected students in the classroom. One theme that emerged in the discussion was the potential influence of teachers on such students.

Here’s a timely, and relevant, piece on the subject that was just published. In the piece, Robert Pondiscio reviews the book Teach Like A Champion by Doug Lemov and, I think, does an excellent job. While I don’t agree with everything that Pondiscio writes, I think he makes a number of important and insightful points.

One of these is in this excerpt:

His focused, obsessively practical study of what makes teachers effective could—and should—shift the terms of our increasingly vitriolic national debate from “teacher quality” to “quality teaching.” This is no mere semantic distinction. The difference is not who is in the front of the room. The difference is what that person does. Lemov’s achievement is to examine teaching at the molecular level. By doing so, he may have rescued education reform from its implicit dependence on classroom saints and superheroes. It is an indispensable shift. If teaching effectively is something for the best and the brightest, rather than the merely dedicated and diligent, education reform is finished, now and forever.

I’m not sure that teacher quality and teaching quality are quite that distinct from one another -- a smart, hardworking, teacher will be more likely to do something worthwhile in front of a class -- but, nonetheless, I think it’s an important distinction.

We could take this discussion any of a million different ways, but let’s try to focus on the issue of “quality teaching” that Pondiscio identifies. I think he’s correct that it would be easier to train every teacher to teach well than it would be to recruit millions of superstar individuals into the classroom (and keep them there), but that doesn’t make it an easy task. Lemov, Pondiscio and others argue that we need more practical lessons that focus on day-to-day details in our teacher training programs. I agree that my program would have been more helpful to me had it done that, though I can see the other side as well -- that if we only focus on classroom management techniques, our teachers will ultimately lack the vision they need to be truly transformational leaders.

What do others think? Is Pondiscio on to something here? Is “quality teaching” something that can be accomplished in the foreseeable future? Do we know how to accomplish this? If so, how?

CEP: I agree with Corey’s assessment that teacher quality and quality teaching are not that distinct from one another. In many ways, who a person is determines what that person does or is capable of doing. To say otherwise implies that anyone can teach and all that is required is an effective, scripted curriculum that is “teacher-proof.” The current trend in education towards stricter certification requirements for teachers, a shift towards the professionalization of teaching, indicates that teaching should require some discretionary activity, rather than the constraint of a teacher-proof curriculum.

That aside, quality teaching is something that can be accomplished. Modes of teacher education and professional development are the place to start. The field knows little about what actually goes on in teacher preparation programs, be they traditional or alternative, much less how the discrete elements of preparation programs may lead to effective teaching. Conversely, we do know something about what makes professional development effective: content focus, active learning, coherence, sufficient duration, and collective participation. In spite of a growing amount of evidence to support the effectiveness of professional development when these characteristics are present, the vast majority of professional development continues to be one-shot workshops, disconnected from teachers’ perceived needs. Understanding what goes on in teacher preparation programs, with the goal of restructuring them and improving professional development to focus on quality teaching could go a long way to improved teaching practices.

ClassroomView: I like Lemov’s ideas a lot, and I agree that good teaching is done minute by minute, rather than just created as a large symphony, if you will, by a magical teacher. I will say, though, that some people are clearly better suited to teaching than others. While great teachers are made, they have to be cut from the right cloth. I think it would do teacher education programs well to follow the lead of the Teaching Fellows by requiring incoming students to audition in, if you will, to programs. I’ve seen way too many teachers simply sign up for a degree in education with absolutely no performance element required, and as a result a lot of completely uninteresting people end up in front of children. I think this is a mistake, and that we should assess raw teaching ability before we start to apply the step by step guidelines that Lemov so convincingly suggests.

Corey: I think the causes of quality teaching are more complex than we’re willing to recognize. Yes, raw ability matters. Yes, training can influence teaching. And, yes, we have a lot more to learn about how to identify both. But I think the causes go well beyond identifying and training talent. My personal theory on the former is that there are a few people who will succeed in almost set of any circumstances, and a few that will fail, but that most will perform very differently in different environments. How they’re trained and mentored probably matters, but so does the curriculum that they’re (not) given, the administrator(s) who evaluate and manage them, the climate and context of the school in which they teach, and the subjects and students they’re assigned to teach.

What is taught (and how) in our classrooms is hugely important -- nobody disputes that. Recruiting, training, and retaining good teachers is part of that equation. But it’s not everything, and I’m not even sure it’s most of the puzzle.

CEP: Are you suggesting that yet unmentioned policy-driven solutions specific to education are part of the equation or are you thinking more along the lines of the need for better understandings of cognition/ how people learn, in order to better inform what drives good pedagogy? Or, are you driving at something outside the parameters of schooling, along the lines of social policy?

Corey: Unless I misunderstand you, I don’t think I’m suggesting any of those. What I’m suggesting is that the quality of teaching to which children are exposed relies on a large number of factors. We tend to focus on teaching ability defined in two different ways when we talk about teacher quality: the first is what I’ll call natural ability (smart, hard working, charismatic, etc.) and the second is what I’ll pedagogical ability (structuring lessons, passing out papers, managing a class and many of the other things Lemov talks about). I’m suggesting that the context of the school and the class in which a teacher teaches also greatly influence the quality of the teaching to which students are exposed. Curriculum is certainly a large part of this, but even beyond that I think the effectiveness of individual teachers will vary widely depending on the particular subjects and children they’re asked to teach, the climate of the school, the support (or lack thereof) from administrators, and any number of other factors. In other words, I’d suggest that the quality of teaching students experience may have as much to do with the context of the school and classroom as it does any particular aspects of the teacher in the front of the classroom.

If I’m right, the policy ramifications would be that we should be hiring administrators who can facilitate teacher development, help create a positive school climate, assign teachers to their areas of strength, and generally design schedules and implement policies that will help teachers succeed.

How Many Bad Teachers Are There?

With all the discussion surrounding the need to fire "bad teachers" these days, I wish we could take a step back and examine how many teachers out there are truly worth firing.

A working paper by Thomas Kane and colleagues sheds some light on the issue.  In the paper, they investigate the validity of teacher observations conducted by outsiders and find them to be fairly strongly correlated with both teacher value-added scores and levels of teacher experience.  This leads to the conclusion that the observations are fairly good measures of teacher effectiveness.

Included in the paper is this chart showing the spread of teachers who averaged ratings on a 1-4 scale.

What sticks out to me is that the distribution is skewed far to the right -- well over half of the teachers earn at least a 3 on the scale, and there are only a few that score really low.  This gibes with TNTP's report on San Francisco, in which fewer than 1% of teachers are rated unsatisfactory, and 94% of principals agree that teachers in their school who are underperforming are rated as such.

In other words, maybe the vast majority of teachers are doing a pretty good job.  Which, of course, is not to say that teachers who are failing their students should remain as is forever.  Clearly, as in any profession, there are teachers who need to receive more training, work harder, and/or move to another job.  But if this is the case (and I'm not going to claim it is based on only one study), then we should be a little more careful with our rhetoric.  If most teachers are doing a pretty good job, then we should probably spend a little more time praising them, a little less time decrying the fate of the teaching profession, and refer to bad teachers as the exception rather than the rule.

Has Kristof Gone Krazy?

Nicholas Kristof is at it again.  He's written another misinformed op-ed about education in the NY Times.

Yes, I'm annoyed that he seems to imply that in-school factors are more important than non-school factors when we know the opposite is true.  And, yes, I'm annoyed that he reverentially references Stephen Brill's hatchet job.  And, yes, I'm annoyed that he cites anecdotal evidence to argue that the exception disproves the rule.  And, yes, I'm annoyed that he assumes enacting certain reforms is moving in the right direction despite very little evidence that these reforms will improve things.

But what bothers me most is his statement that "A study found that if black students had four straight years of teachers from the top 25 percent of most effective teachers, the black-white testing gap would vanish in four years."  Please -- everybody -- please stop saying this.  It's, quite simply, not true

First of all, the study being referenced didn't "find" this -- it was speculation (or, in economist-speak, a back of the envelope calculation).  Second of all, the speculation was based on an erroneous assumption -- that teacher effects were additive.  In other words, that if a really good teacher could help kids close 1/4 of the achievement gap in one year, that four really good teachers could help kids completely close the gap in four years.  But life doesn't work that way.  For at least three reasons:

1.) The effect that a teacher has on students wanes over time as those kids go off and play over the summer and move on to different classes with different teachers.

2.) The large gain brought about by one teacher is, in part, due to the inferior teachers those students had in previous years.  Now that they've had a world-beater, it will be harder for their teacher their next year to help them make as much progress.

3.) That large gain might not have even occurred.  Measurement of teacher effects is notoriously imprecise.  In a number of studies, the correlation between a teacher's effect one year and the next have been surprisingly inconsistent and have only low correlations.

note: Somehow an earlier, different, draft of this post ended up here after firefox crashed.  This is the correct version.

update: really, seriously, his claim isn't true