Tuesday, July 29, 2008

In Need of Alteration

I saw an opinion piece in the latest issue of Newsweek yesterday that I caught my attention. In it Johnathon Alter offers some thoughts on what's wrong with and should be done about education in the United States.

I'm always glad when education gets more attention in the mainstream media, and Alter raises some valid points, but the piece leaves an awful lot to be desired. If you haven't yet read it, it's short and it's worth a glance. To those of you who read the education blogs regularly, I don't think you'll see anything particularly new.

To paraphrase, he essentially argues that unions are ruining our schools, that we need to hold teachers more accountable, make schools more like KIPP schools, and that Obama should go against the unions and find ways to bring and keep better teachers in our school.

Unfortunately, Alter makes a few errors. Perhaps most grievous, at least to those of us with research training, is his false statement about how students are selected for KIPP and other charter schools. He says that students are "randomly selected" to attend. Random selection is a technical term meaning that the sample (in this case the students attending KIPP) was selected completely at random and, therefore, no person should be more likely to be included than any other person. This is simply not how KIPP works. Parents and students have to apply in order to enroll. Yes, students are admitted from the pool of applicants by lottery -- but this does not constitute random selection because they were randomly chosen from a non-random group.

Alter acknowledges that KIPP schools aren't fully replicable, but not necessarily for the right reason. He argues that there simply aren't enough effective teachers to go around. That may well be the case, but I hardly think that KIPP has a monopoly on effective teachers or that "effectiveness" is necessarily what sets KIPP teachers apart from teachers at other schools. As far as I can tell, a large number of teachers at KIPP and other similar schools are young and without families and, therefore, willing to essentially devote their life to the school for a few years before moving on. What we really lack is enough people who are willing and able to do this in order to run all schools like KIPP.

I must also take issue with his overly strong statement that "we know what works to close the achievement gap." Some people appear to have done it, and we have some good ideas -- but that doesn't mean the solution is just sitting out there waiting for everybody to latch on.

I'll give Alter credit for arguing that we need to devise ways to measure how well teachers teach rather than simply arguing that we need to measure them and then hold them accountable.

Perhaps the most ridiculous thing that Alter writes -- and the statement that gives away the ideological underpinnings of his argument if anybody wasn't already aware -- is that unions "still believe that protecting incompetents is more important than educating children." Unions are far from perfect, and this is far from the most inflammatory rhetoric that I've read about them, but it's still sheer and utter nonsense. Alter disagrees with unions, so he essentially resorts to exaggerating their misdeeds and calling them names. Though more polite, it's the intellectual equivalent of calling somebody with whom you disagree a Nazi or a terrorist.

If I were a union leader, however, I would mull over Alter's final point. He argues that unions should "change their focus from job security and the protection of ineffective teachers to higher pay and true accountability for performance." The statement is a bit over the top, but the general idea that unions could view submitting their members to more scrutiny in exchange for higher pay is something on which both sides might find some common ground.

Thursday, July 17, 2008

The Theory behind Merit Pay

Liam Julian posted a couple quick blurbs on merit pay today, including one partially in response to a comment I left on the first. Merit pay is, as I'll explain, fine and dandy on paper -- but I still worry about its feasibility in real life.

By "merit pay," I mean plans to pay better teachers more money -- which has taken all sorts of forms in different schools and districts.

The basic theories behind merit pay, as far as I can tell, are that:
-Teachers will work harder if they know that better teaching will result in more money
-It's more fair to pay teachers based on how good they are than on seniority or education
-Successful teachers are more likely to stay in the profession if their success is rewarded
-Brighter and more driven people are more likely to enter the profession if they know that their success will be rewarded
-Less successful teachers are more likely to leave voluntarily if their pay isn't advanced

All of these are eminently reasonable assumptions. On paper, they make a lot of sense. It wouldn't surprise me at all if any or all of these were borne out by experiments with merit pay.

That said, I also refuse to assume that all of these will be borne out in real life. Economists like to imagine that everybody is a "rational actor" that acts in their own best interest. That theory is, generally speaking, usually true. But there are exceptions -- large exceptions. Not to mention practical hurdles. And these mean that there are a lot of unanswered questions about merit pay, including:

-How much harder are teachers able/willing to work for more money?
-Would teachers motivated by money behave differently from teachers motivated intrinsically?
-How many more people would consider teaching if merit pay were common?
-How large of a role do salaries play when teachers leave the field?
-What types of people would enter teaching if pay were different?
-How well can we measure how "good" a teacher is?
-Will teachers buy-in to any measure of their success?
-Can a rewards system that is both fair and easy to understand be created?
-What type of behavior should be rewarded?
-What types of challenges or additional responsibilities should be rewarded?
-Do teachers know how to alter their behavior so that they will be more successful? (i.e. if teachers work harder, will they necessarily be better?)

I see multiple meritorious answers to all of these questions. If you'll bear with me, I have two quick anecdotal pieces of evidence that point in opposite directions. When I was teaching, I lived, ate, slept, breathed teaching. I gave it my all. If somebody had offered me $1 million if my kids improved, it wouldn't have changed my behavior any b/c I was already doing everything I could. A friend of mine, meanwhile, taught in a similar situation. He came in and out-performed all of the other math teachers in his middle school (as measured by student test scores) and then left, in part b/c he wouldn't be paid any more the next year as a result of his success -- he found it quite depressing that he would continue to earn only his 3% (or whatever) salary step bumps and make less than the older teachers in the department no matter what he did.

In short, I see arguments on both sides of the coin. I, for one, refuse to assume that merit pay either will or will not work. In order to answer that question (prepare for the most hated four words in education research) more research is needed.

Wednesday, July 16, 2008

Last-Second Changes

I haven't come across much on this in the research literature, but I'm willing to stick my neck out and assert that one thing that absolutely drives people in schools nuts are the number of last-minute changes from above. People are told that they will be teaching different subjects or grades days or hours before the school year begins, for example. I was hired by the school at which I worked two days before the start of the school year because the previous 6th grade teacher had just been moved across the hall to be a bilingual teacher.

This is far from the most egregious example, but the state just shuffled around an awful lot of administrators in Nashville. A month before the start of the school year is a lot better than a day, but the former principals have been hiring staff and otherwise preparing their schools for the fall for two months now.

Imagine Mr. Smith hiring a new teacher in June and then Mr. Jones being installed as the new principal in July. What if Mr. Jones doesn't like the new teacher or the new teacher doesn't like Mr. Jones? It's going to take at least a year to sort these kinds of things out, and many of them could have been avoided had the state simply acted two months earlier.

Monday, July 14, 2008

"Rational" and "Stupid" are not Mutually Exclusive

Back when I was studying economics, we learned all about the rational behavior in which individuals and organizations engage in response to various incentives and disincentives. "Home Economicus" is the generic rational individual that we assume all people to be -- actions are based on a rational evaluation of what actions would most benefit that individual.

Though all sorts of exceptions exist, the theory generally holds in most situations. We can predict that a person will be more likely to buy a product when the price is lowered, for example. In the same vein, we can predict that a teacher will change their behavior if offered a bonus when he/she reaches certain benchmarks. Such behavior would be considered rational.

Here's the problem that we sometimes overlook: what is rational to economists is not always in the best interest of society. Here's the latest example: Flypaper blogs about a plan in Los Angeles to allow certain students to attend school for only two hours per week. The plan is labeled "unbelievably stupid" and I largely agree.

But the plan is anything but irrational. The main goal of the program seems to be to encourage students to enroll in this new school rather than dropping out. They may not learn much by attending school for only two hours per week, but the fact that they're enrolled means that the school district continues to receive money from the state for each student -- money they wouldn't receive if the student dropped out. The plan is, in short, in the best interest of the school district. Those who came up with the idea were acting in their own self-interest.

Self-interest is a powerful motivator. But it's not always the solution.

Wednesday, July 9, 2008

Why Do Teachers Get so Defensive?

I read Liam Julian's post "The Martyrs" right before the holiday (yes, I'm a little behind). Though needlessly flippant and condescending, he asks an interesting question: why are teachers so defensive when criticized?

He, specifically, is upset at teachers who dismiss the opinions of others by pointing out that they've never taught (as some teachers recently did to him) and argues that the idea that only teachers can judge other teachers is a logical fallacy. He makes a valid point; it is possible to recognize excellent or poor teaching (and, for that matter, good and bad ideas for a school) without having ever taught.

That said, I'm not sure he really gets to the heart of the issue. Simply saying that a teacher is wrong when they make that argument is at least as unproductive as a teacher making that argument in the first place.

In other words, let's try to find an answer to the question "Why do teachers get so defensive?" other than "they're wrong, so who cares."

I was quite critical of teachers before I started teaching. But now, with the exception of those few who are both truly awful and don't care, I hesitate a bit before I lash out. And I wince about when I hear others lash out.

Why?

In my experience, teaching is a very personal pursuit. It requires a lot of an individual. Many people who go into teaching devote more than just time and energy into their teaching -- it's more than just a job to them. It's also much more difficult and nuanced than it looks. I was pretty cocky before I started teaching -- I was convinced that I could sweep in and change the world. And then I discovered about a million things that I never would have anticipated.

I think these two things make teachers quite jumpy when they are criticized. I used to deliver newspapers when I was younger. If somebody had told me I was a bad newspaper boy, I wouldn't have been particularly happy; but it wouldn't have been the end of the world. I simply placed newspapers inside of doors after school and then went back home -- it was something that I did to earn some cash to buy candy and other cool (and worthwhile) stuff rather than a part of my identity.

For many teachers, however, their job is personal. And criticism of the way they do their jobs is seen as criticism of them as people. And nobody takes kindly to that.

Perhaps more important is that anybody who has taught knows that there's more to it than it looks like. When people from the outside criticize teachers, I think it's quite logical to point out that that they don't know the full story. It's a heck of a lot easier to point out what a teacher is doing incorrectly than to actually do it correctly. That doesn't mean that somebody who has never taught can't recognize good teaching and bad teaching, but that their understanding of why a teacher is doing something or what they should do differently lacks nuance.

Lastly, I think there is a mismatch in expectations and rewards for teachers. I'm not sure if the members of any other occupation have such high expectations placed upon them by the public. Your dentist doesn't make the newspaper if they post a picture of themselves in a bikini on their myspace page. Your accountant isn't shunned on the local news if they have a sex-change operation. Teachers are expected to be role models and to conduct themselves with dignity at all times. And yet, teachers may also be criticized more than any other sector of society. In other words, teachers are expected to be perfect, but are treated as second-class citizens in a number of respects.

When somebody pours their heart and sole into a difficult job that commands little respect it shouldn't surprise us when they bristle at criticism.

Tuesday, July 1, 2008

How to Fix Testing?

Maybe it's b/c my brain is still only half-functioning, but I feel like writing something today about which I know little and that will probably result in a lot of people calling me stupid.

Testing is a huge part of American education, but it's riddled with all sorts of problems. One problem is that most tests are simply not a valid measurement of student knowledge/aptitude/performance (or whatever it is that the test strives to measure). Allow me one example. My first year, all my students took the English exam at the end of the year and their performance was compared to the previous year. This makes sense on paper. But, in reality, the exam was 50 minutes long and contained 50 multiple choice questions. The internal validity, in short, was lacking. Economists would say there was "a lot of noise" in the test results -- the scores of the students varied widely from their true ability based on all sorts of random reasons. This is one flaw with the value-added formula -- if we're not really sure how much the kid knew last year, then we can't really tell how much they learned this year.

Many have also complained that students spend all day on "test prep" in the weeks or months (I was told to stop teaching social studies after Christmas and do practice English tests) leading up to the exam -- in effect, teachers are trying to teach kids to game tests. This not only robs kids of other learning opportunities but, if done correctly, artificially raises scores.

Anyway, skoolboy posted a tongue-in-cheek call for more testing on eduwonkette's blog today. While I was teaching I thought up a somewhat similar proposal to his, but my tongue was not in my cheek. Here's one way I think testing could be done that would solve a lot of the problems:

Every class in the school takes one test approximately every two weeks. The time, day, and subject matter of the test are random. The teacher and student find out that they will be taking a test a certain period the following day at the end of the previous day. They are not informed of the subject matter until they arrive in the testing room (a testing room equipped with computers is necessary to ensure that all these tests can be graded quickly). By the end of the year, each student has taken about 20 tests spread across all subjects. This provides a much broader base on which to judge students' knowledge. Since testing runs all year and one never knows on what subject they'll be tested next, test prep is impossible to do effectively or to sustain throughout the pre-test time period. The system will be a bit of a pain in the butt and will generate some complaints from teachers. A well-organized testing coordinator and a well-run testing room would have to present in each school. A wide variety of high-quality tests that can be graded largely by computer will have to be available -- national testing may be the only affordable way to do this. But, if done correctly it would solve a lot of problems.

In short, I'm not convinced that more testing isn't the answer.

Friday, June 27, 2008

Does Teacher Opinion Matter?

Sorry for my sudden disappearance -- I've been off trying to deal with more important issues.

The NY Times ran an article this morning about the survey the UFT recently conducted of NYC teachers (UFT press release here, full results here). In short, it seems that teachers don't particularly like Chancellor Klein.

Methodological concerns aside, I wonder if these results matter.

I think a strong argument can be made that teachers' feelings about a particular reform will greatly influence how they implement said reform, but I wonder about their feelings toward things that are more removed from their immediate situation. Does a teacher's love or hatred for Joel Klein affect how they go about their daily business? Does it make them less satisfied with their jobs? Does it make them less likely to implement reforms he pushes? Or is "Chancellor Klein" too much of an abstraction to matter much when teaching a roomful of kids?

Wednesday, June 18, 2008

Home vs. School Influence

I previously asserted that the largest reason for the failures at my school was not anything in the school but, rather, what took place at home.

Ken DeRosa, over at D-Ed Reckoning, took the time to thoughtfully respond to a lot of what I had to say (he disagrees with me) and I'd like to respond to some of his comments.

*The first point he makes is that my assignment of causality to home factors over school factors is suspect. He has a point. I can't be 100% sure about that. I didn't rigorously evaluate student attitudes toward school, survey parents, observe home environments, run regressions, etc. I did not use rigorous research methodology to determine what caused what and, as such, I won't be publishing it in an academic journal anytime soon. But what I did do is spend all day in the school for two years straight. I taught about 70 different kids in my two classes, and probably at least a thousand others while covering other classes. I held conferences with parents. I got to know students. I talked to teachers. I observed other classrooms. In short, I knew the school inside and out. I can't empirically prove that home factors matter more than school ones, but I have plenty of good reasons to believe it. There's certainly no way to disprove my theory either.

His second point that he makes surrounding validity is that the students' behavior was confounded by previous academic experience. All the kids should have attended school for at least 5 years prior to coming to our school (NYC doesn't mandate kindergarten), and I know very little about those experiences or how they affected learning and behavior. This is true. If I were publishing this in a journal, it would be a major weakness if not a fatal flaw. Again, I can't be 100% sure -- but given my in-depth knowledge of the situation, I'm fairly confident in my position.

*DeRosa says that behavioral problems weren't caused by poor home environments but, rather, by poor management and instructional strategies. Given that DeRosa has never, to my knowledge, stepped foot in my school it's hard for him to know exactly what was happening and why. That said, I agree to some extent. My classroom management skills were lacking, as were most of the other new teachers'. My training was not enough, and I (and other teachers) certainly could have used better techniques. But he'll have to trust me when I say that I've managed plenty of kids in plenty of situations in my lifetime and that the kids in this school were different. They were angrier. They were more defiant. They had less self-control. Of course the adults in the building failed to do enough to create a positive and structured environment, but doing so was much, much harder than it should have been because of the social norms to which the kids were exposed outside of school.

In summary of what went wrong at my school, let me be clear: myself and the other adults in the school failed the kids in many, many ways. The school was poorly run. I lacked adequate training. We could have done any number of things better (especially around discipline) and it would have helped the situation. But I stand by my assertion that the largest cause of problems at the school was the home life of the children. And I apologize for the obscene length of this post, but I'm going to explain why.

1.) When I started teaching I was convinced that the reason why schools such as mine were failing was because they were underfunded and that the teachers didn't work hard enough. I was at least 100% confident that I could turn around the life of every student I encountered. I believed I could overcome every problem they've faced simply by being inspiring, demanding, and generally awesome. But I slowly came to realize that this was not the case.

I saw many problems in my school, including: incompetent administrators (not all of them, mind you), high teacher turnover (and, therefore, inexperienced teachers), few resources and supplies, a dilapidated building, etc. I wanted to believe that with the right policies that we could be as good as any other school, but here's what I realized (and feel free to disagree on this point). If you were to lift a school from, say, Scarsdale and plop it down where our school was and then plop our school down where theirs was, you would have an interesting experiment. Our school, with the lack of resources, inexperienced teachers, etc. would now be populated by wealthy suburban kids. The Scarsdale school, with much higher expenditures, more experienced teachers, nicer facilities, smaller classes, etc. would be populated by our kids. Which school would be better?

Before I started teaching, I would've thought that the former Scarsdale school would be better. No longer. In my mind, our school would instantly become superior once it was populated by kids who were well-fed, well-adjusted, well-behaved, and had involved parents. In other words, all the of the structural factors matter less than the population of the school.

2.) My final summer in the Bronx, I taught at a private school that ran a special summer program for public school kids who had fallen behind. I taught Math, which I'd never taught before. The students came from the same neighborhoods (there was an income cap to get into the program). Despite this, the program was an unmitigated success. Students came to school on-time and ready to learn. Students smiled. Students listened when I talked. Students completed homework. Students learned. I had fun.

What was different? Certainly the program was well-run. Classes were capped at 15. I had two or three high school mentors as assistants in each class. The facilities were shiny (they even washed the board for me at night!). The teachers were good. All these things made a difference, but the largest difference was the home environment from which these kids came. The students had parents who went out of their way to apply to a selective program, come in for face-to-face interviews, make sure their kids attended so that they would be allowed to remain, and attended parent day even though they had to take a day off work.

To be fair, the program was selective and had no qualms about removing students who didn't show up or didn't cooperate. As a result, we were left with kids that came from stable families but that lived in poor neighborhoods.

Again, I have no empirical evidence -- but my experiences have convinced me that the home lives of these students contributed more to the success of the program than did the competency of the adults involved. I'm not arguing that the other stuff didn't matter, just that it mattered less than what happened at home.

3.) DeRosa argues that "The problems Corey sees may start at home, but there is no reason to believe that they cannot be solved and compensated for by schools." I disagree. I see no reason why they can't be assuaged, but schools with current levels of resources will never fully overcome what happens at home.

I'm sure just about everybody reading this could point out a school or ten that have overcome this. First of all, an exception doesn't disprove a rule. Secondly, many of the schools that have overcome home disadvantages go far beyond what we normally expect schools to do. Take KIPP schools for example, students not only self-select but are asked to leave (or simply told they will be held back if they stay) when they don't cooperate. The schools run, I believe, about 9-10 hours per day. Millions of dollars in grants are brought in to provide all sorts of extra opportunities. I think this is great. I would encourage many of the kids that I taught to go to a KIPP school if they could, but it doesn't accurately reflect what we'd normally expect schools to do.

Take, as another example, the SEED school in D.C. By all accounts, it has been tremendously successful. But it's a boarding school. If that's what it takes to overcome disadvantages stemming from students' home lives, then I'm all for it -- but we can't reasonably expect all inner-city schools to become boarding schools.

As a final example, let me provide one more anecdote. The leader of YES! Prep in Houston was on campus to speak in the fall. He opened with a question that read something to the effect of "can a school educate poor students as well as wealthy ones when they have the same resources?" And their answer was "YES!" As far as I could tell, the school did a phenomenal job. They had a dedicated staff that went the extra mile to make sure students succeeded. But they didn't provide the same resources as a suburban school, they provided the same resources as both a suburban school and a suburban family. They had extended school hours and all sorts of social activities. They raised money to provide their kids with college scholarships. They raised money to take their kids on college visits and on weekend trips to museums, plays, etc.

In short, I'm not arguing that it's not possible for a school to overcome a great deal of what happens at home, but rather that it's not possible given our current level of school resources. If somebody can figure out a way to fund a doubling of school hours, cultural trips, college visits, etc. then I'll be more open to considering schools as a full remedy for social ills rather than a partial one. Let's not forget that once a student is old enough to attend school they only spend about 22% of their waking hours inside a school building. The other 78% of the time, not to mention the first 5 or 6 years of their lives, have an awful lot of influence ((180*7/365*16) = .22).

4.) Last point, I promise. Let me get away from personal anecdotes and move on to some research. The Coleman Report was commissioned in the 1960's to prove that poor kids (especially African-Americans in segregated schools) were failing because their schools didn't have enough resources. Instead, it found that home life mattered more than what happened in school. It's now been about 42 years since the study was released, and countless follow-up studies have confirmed this finding. If there's any certainty in education policy research, it's that homes influence academic achievement more than schools.

The only exception of which I'm aware is what's known as the "Heyneman-Loxley Effect" based on a paper written by Stephen Heyneman and William Loxley in 1983. The pair studied 29 countries and examined the influence of home vs. school. They found a strong correlation (-.72) between the GNP of a country and the percentage influence of home vs. that of school. In other words, schools mattered more in poor countries and homes mattered more in rich countries.

In Conclusion: I stand by my assertion that the largest problem affecting my school was the home lives of our students. This does not mean that I made excuses rather than working hard. If anything, this motivated me to work harder. This does not mean that schools can't make a world of difference. I wouldn't be studying education if I didn't think it could make a difference in students' lives. What this means is that problems exist both inside and outside of schools -- and that schools can't fix everything.

Narrowing Gap

A new report out today from the Fordham Institute apparently describes how the bottom 10 percent of students have made more progress this century than the top 10 percent of students.

Given that the low-performing students in the U.S. lag behind low-performers in other countries while high-performing students hold their own against other high-performers (previous post), it's hard for me to see this as anything but a good thing.

Tuesday, June 17, 2008

Why TFA?

The Wall St. Journal has an editorial today extolling the virtues of Teach For America. I think TFA does far more good than harm, but I can't help but notice at least two glaring errors.

1.) Near the end they cite the problematic Urban Institute study as iron-clad proof that TFA teachers are better than regular teachers. Citing one study as definitive proof of anything will continue to annoy me, no matter what the circumstances. A handful of studies with larger sample sizes have been conducted, and they should have mentioned them.

2.) They seem to imply that the only reason people apply to TFA is because they can cut through the red tape. Here's the part of the editorial I'm talking about so that you can judge for yourself:

Unions keep saying the best people won't go into teaching unless we pay them what doctors and lawyers and CEOs make. Not only are Teach for America salaries significantly lower than what J.P. Morgan might offer, but these individuals go to some very rough classrooms. What's going on?

It seems that Teach for America offers smart young people something even better than money – the chance to avoid the vast education bureaucracy.

The chance to be put right in a classroom is certainly part of the allure of TFA and similar programs (without that opportunity I never would have applied to any of these programs), but it seems somewhat disingenuous to suggest that it's the only reason. I'm not sure if any studies have been done on the topic (please let me know if you've seen any), but I'm willing to bet that TFA applicants apply for all of the following reasons:

-the chance to "make the world a better place"
-the chance to move directly to the classroom without going through certification
-it looks darn good on a resume
-it's a good transition from college to the working world
-gives people two years to decide what they really want to do
-not a long-term commitment
-it's the cool thing to do

Additionally, TFA puts a lot of effort into recruiting people.

In conclusion: I hardly think that the fact that TFA attracts talented people proves that schools would have no problem attracting talented people if certification didn't exist.