Sunday, March 30, 2008

Complicated Statistics and Education Policy

I've encountered a number of surprises after switching from teaching to researching, but perhaps none have been more notable than the role of complicated statistical methods in education research. It seems that, historically, education research has been denigrated as non-rigorous but that, currently, the trend is toward more quantitative methods -- much of which is done by economists. Pick up virtually any journal and you'll find an article with statistical tables and strings of Greek letters that 99% of the population won't understand. In some cases this is a good thing, but in some cases I'm not so sure that it's not hurting the field. And the question still remains: why have increasingly complicated statistical models become so commonplace in educational research?

Let me start with an anecdote (a decidedly non-rigorous research method):

When I presented my paper on teacher retention at a conference earlier this month, there were apparently a number of audience members who had not been trained in statistics (I used some fairly basic ones for my analysis). I didn't really receive any criticism about the presentation, and the feedback seemed to indicate that most people didn't really understand the tables I'd presented. Afterwards, one guy walked up to me and said "you lost me on the statistics, so I'll take your word for it."

At the time, I chuckled and continued with the conversation. But, in retrospect, this troubles me. Why should he take my word for it? Just because I used some statistics that he didn't understand? And it got me thinking. Does this happen on a larger scale as well? Are people scared to argue with the methods in these papers because they don't understand them? Do some people take economists and others at their word when they do complicated statistical analyses b/c they simply assume that, since they don't understand them, they must be thorough and correct?

Probably the other most notable thing I've learned is that there is no such thing as a perfect research study -- every single one has significant flaws (at least in the social sciences, I claim ignorance on physicists observing quarks). Part of the motivation for this post was the conversation in which I found myself embroiled on another website (here) . The blog post and comments seem (at least to me) to assume that the results of the study provide a definitive answer -- in part b/c it uses incredibly complicated statistics. The paper is better than most, so I hesitate to use it as an example, but, nonetheless, it has flaws and limitations -- just like any other. And I wonder if the methods were more accessible and understandable if people would be so willing to accept the findings without further discussion.

Thursday, March 27, 2008

How Does Teacher Retention Affect Schools?

I've been on the road for two weeks now and all the conference presentations are starting to muddle together in my head. As such, I'm just going to talk briefly about the paper I presented yesterday. It's one of the few things that I can keep straight in my head right now.

Here's the premise of the paper:

Lots of people have investigated why teachers leave schools, but I haven't seen much on how schools are affected by teachers leaving. The simplest version of this question would be "does retaining more teachers improve student performance?"

As I've discussed before, the nature of teacher retention is very different in different schools. Since retention rates are notably low in high-poverty urban schools, I chose to focus solely on them. I had some data and was able to compile a dataset of 43 NYC middle schools that had large percentages of poor students (not including magnet, K-8, and some other schools with unreliable statistics). Across these schools, about 40% of teachers were in their first or second year of teaching at their current schools and less than half had a total of 5 or more years of teaching experience.

There was also a moderately strong (r=.44, p<.05) relationship between the average student score on the 8th grade state math test and the percent of teachers who had been teaching for at least two years at that school. In other words, schools with higher rates of teacher retention also had higher student achievement.

Using regression analysis and controlling for race and attendance rate, teacher retention was still significantly related to student achievement. Teacher retention had a fairly sizable effect (effect size of .20) and the model did a pretty good job of explaining the variance in test scores (R-squared of .70).

So, in short, among high-poverty middle schools in NYC, those with higher rates of teacher retention also had higher test scores, even when controlling for other things that influence student test scores. The question that I can't answer with the data is whether better schools make teachers want to stay there more or if more teachers staying in a school improves the school and boosts student achievement (or a little of both). I hope to come closer to answering this question in the future.

Wednesday, March 26, 2008

Discipline: The "Dirty Little Secret?"

I often say that I spent two years trying to teach in the Bronx. Why trying? Because discipline problems in my school were so severe that it was sometimes hard to believe I was teaching much at all. I have no evidence on exactly how typical my school was, but it was abundantly clear that discipline issues were, far and away, the biggest problem.

I spent most of the day in a training session on how to use international datasets rather than attending sessions, so I was planning to go to sleep and not bore anybody with the details, but I just happened to notice Diane Ravitch's latest post right before turning out the lights.

Ravitch basically writes that discipline problems are a major problem in the United States b/c kids don't respect adults and that other countries don't have the same problem (she, of course, says this more eloquently). This immediately stirred a number of thoughts:

1. Based on my experiences, I couldn't agree more. Discipline is a major hurdle for a number of schools. It is much more important than standards or curriculum in these schools, but seems to be researched far less frequently. Is discipline not viewed as a serious problem by academics b/c it's limited to only some schools, b/c it's seen as more of an issue for the popular press, or something else?

2. Based on my research, I cannot uphold her claims. I've found zero evidence that the U.S. is an outlier in terms of discipline problems. There are, however, at least two major problems with my research so far: 1.) The international data on discipline is far from conclusive and 2.) I just found out today that the TIMSS international assessment does not use a nationally representative sample of teachers and, therefore, saying x% of teachers say that behavior is a major problem is not a valid statistic (I have to look at how many students have teachers who said this instead).

3. I wonder how widespread this problem is and how different it is from previous generations. The first question is somewhat easily answered while I'm not sure that it's possible to obtain an answer to the second. The U.S. did stand out in one way from other countries in my research: there was a stronger relationship between problems a principal reported in a school and the SES of the students (schools with poorer students reported both more frequent and more severe problems) than in any other country. I know discipline was the major issue in my school. I know discipline is the major issue in other schools where friends have taught. But I'm unsure how much of an issue discipline is across the country. Do adults always think that kids are less respectful than they should be, or do we have a real problem across the country?

p.s. I stole "dirty little secret" from a commenter on the Ravitch blog entry. I've used similar terminology in the past, but am too tired to think of exactly what it was. I may have more to say on this topic tomorrow.

Tuesday, March 25, 2008

What Does Alternative Certification Say About the Value of Traditional Certification?

In my last post I mentioned debate over alt cert programs. Allow me to elaborate on one strand of the debate.

It is my impression that alt cert programs were originally started as a way to fill vacancies in hard to staff schools. To generalize, the problem was that not enough certified teachers were willing to teach in certain places, but prospective (uncertified) teachers were -- but, at the same time, were not willing to go through the whole certification process. "Alternative Certification," then, allowed competent but uncertified individuals a different, more acceptable, route to certification and also enabled districts and schools to fill vacancies. It's a rational compromise.

Both academia and the press frequently discuss the pluses and minuses of these programs. One of the most frequent arguments against (at least some of) them is that, though they are better than the status quo, they are not the ideal solution -- with some saying they're really more of a band-aid. On the opposite side, many argue that alternatively teachers are just as effective as regularly certified teachers (research has usually found either small or no benefits to certification depending on how it's defined) and sometimes take the opportunity to impugn certification programs and schools of education.

A number of students in my program (including myself) taught through alt cert programs, so they often come up in conversation. One issue that's raised is the effect that alt cert programs have on regular certification programs and public perception of them.

I am wholly unsure to what degree the people who run alt cert programs (and I'm sure there is a lot of difference on this) are out to prove that regular certification is useless. I get the feeling that most programs do not explicitly make this case. It seems to me that the programs are more focused on attracting qualified individuals than making political statements. But whether or not they explicitly criticize schools of education, the mere presence of these programs ultimately implies that certification is not important. In other words, they might not be trying to offend teachers who went through regular certification, but it's understandable if these teachers are offended.

For example, the NYC Teaching Fellows (my former program) regularly blankets subway cars with idealistic ads in order to attract applicants. The ads imply (whether or not they are intended to) that anybody who is smart and works hard can be a good teacher no matter their background and, therefore, that work ethic and intelligence matter more than training.

I'm not saying that there isn't some merit to the statement, but imagine if there was an alternative med school program and anybody who was smart and willing to work hard could be in the operating room after one summer of training. Regardless of whether the program fails or succeeds, its mere presence implies that it's not hard to gain the same expertise as doctors who went to regular medical school.

I'm not saying that this is necessarily good or bad (like most things, it's probably some of each), but I've noticed that this part often escapes people and I find it interesting.

An Urban Teaching Corps?

I was all set to go to bed when I noticed this post on the eduwonkette blog. Given that I was less than satisfied with my last two posts (I was too busy to do much more than regurgitate what happened in those sessions) I felt pressured to write something a little better for anybody clicking here for the first time. So I clicked on the other link provided in that posting and found an interesting idea that Jennifer Steinberger Pease wrote about in EdWeek.

She writes about an acquaintance who wants to apply to Teach For America (TFA), but will already be certified and have a master's degree and, therefore, is not really TFA material (I checked TFA's website to see if they would accept such a candidate, but their description didn't answer the question in either direction). She feels that this teacher-to-be would benefit enormously from, and be ideal for, TFA, but sees no other comparable option available for certified teachers. She, therefore, proposes an urban teaching corps similar to TFA -- but for certified teachers.

I've thought (and debated) a lot about the pluses and minuses of TFA and other alternative certification programs but, honestly, such a scenario had never occurred to me (nor had such a solution). Jennifer does seem to have found quite the gap in the current system. I've had many discussions about whether TFA and other programs imply that teacher certification is meaningless, but never really thought about the TFA-quality people who are already certified. I'm not sure that there are quite as many people who want to teach in high-poverty schools but are unable to as she seems to believe, but I don't doubt that more would teach in these schools if they could enter a TFA-like program.

I can see only two major holes in her plan:

1. TFA may, in fact, take certified teachers -- I simply don't know. And if not them, many other similar programs might (The New Teacher Project runs a number of sister programs). I taught with the NYC Teaching Fellows (one of those sister programs) and I know that other people in the program had previous teaching experience (though I'm unsure if any were certified).

2. High-Poverty schools are not only in urban areas. Accordingly, TFA is not only in urban areas (I know two former TFA members that taught in rural areas in the Mississippi Delta and along the Texas border). So I would propose either changing "urban" to "high-poverty" or adding a "rural" teaching corps as well.

Normally new ideas for education have to be taken with a large grain of salt -- it seems that there's always a strong reason that they'll never happen and/or a large downside. I see neither of these with this idea. The easiest solution (but not necessarily the best), of course, would be for alt cert programs to simply start accepting certified teachers. Either way, this idea makes sense to me.

Monday, March 24, 2008

More on Alternative Schools

The final session that I attended was a symposium of researchers from CA that investigated alternative schools (previous post), which Milbrey McLaughlin said were essentially "invisible."

Alternative schools in CA started around 1917 when the Smith-Hughes Act established federal funding for continuation high schools which, at that time, were designed to provide vocational training to people with other jobs. They eventually morphed into places for "over-aged, under-credited" students to go in a last ditch attempt to avoid dropping out. Today, the 520 continuation high schools in CA vary widely in every way; from pedagogy to the students they enroll.

Among the largest challenges these schools face (other than that most people are unaware of them) are that about half of students enroll for less than 90 days, that they often serve as dumping grounds for both unwanted students and teachers, and that they are often last in line for funding. Indeed, one superintendent told the researchers that it was acceptable for 10% of the students to fail and end up at the local alternative school b/c nobody would notice.

It is my perception that alternative schools are usually started to remove "problem" kids from classrooms in order to better facilitate learning for the rest and/or to "fix" these problem kids. According to the panel, the most successful schools were ones who tried to fix the school environment to suit the kids instead of fixing the kids so that they could fit in a typical school environment.

McLaughlin said that the system was basically "Balkanized" -- there was little communication with other county services, everybody seemed to envision different missions for the schools, etc. What they did find, however, was that the most successful schools were led by "supermen/women" who treated their job as a calling and worked tirelessly to make their school the best place possible.

The work of some of those individuals impresses me, but I do wonder if that is really replicable or scalable -- in other words, if there are enough people willing to sacrifice their lives to these schools in order to run all alternative schools in the country. It's the same question confronting KIPP and other successful schools. Many of them are built on the backs of supermen/women who work tirelessly for their schools and essentially sacrifice years of their life to make their school successful. I am in awe of what these people do, but is it really a solution for the nation? Can we find enough qualified people to treat schools as missionary work to make all schools uber-successful?

The Costs and Benefits of Schooling

The first session I attended at AERA was an overview of a recent book, The Price We Pay. The book was published by a team of noted economists as an attempt to quantify the benefits of schooling to society and conduct a cost-benefit analysis of some education reforms with proven track records.

They concluded that a person who drops out of high school pays about $100,000 less in taxes (in today's dollars) over the course of their lifetime than does a person who completes high school (only high school, not college) and that, overall, each high school dropout costs the country about $209,000 in lost taxes, welfare, court costs, etc.

Given this figure, they look at how much various interventions cost per additional student that graduates from high school and find that most cost about 1/3 of the amount that society benefits from their graduation. I didn't read the book -- I only heard a 90 minute summary of it -- so I'm not sure exactly what they take into account, but two things that would bias these estimates jumped into my head.

1.) They only look at the additional benefits from the students who graduate from high school that otherwise would not have. What about the students who attended college or graduated from college that otherwise would not have? Taking this into account might mean that their estimates were biased downward -- that there are more benefits than they thought to the interventions.

2.) A shift of a couple percentage points probably wouldn't affect the benefits of a high school education much, but a large increase likely would. If, say, half as many students dropped out each year then it's likely that the benefits of graduating from high school would be smaller. This would mean that their estimates were biased upward -- that the benefits of the interventions were not as large as they thought.

Henry Levin assured a questioner who raised a similar issue that they had taken a lot of things into account and that their estimates were very complex, so it's quite likely that these concerns are addressed in the book. Especially without having read the book I hesitate to believe that these estimates can possibly be very precise, but I think there is some value to them nonetheless. I think a financial analysis of the costs and benefits of school interventions is valuable information to have before making a decision. That said, Henry Levin made sure to emphasize at the very beginning that the primary reason for helping the poorest children is a moral one rather than a financial one.

Also: perhaps the most notable tidbit I gathered from the session was the projection that in 2020 the workforce will be less educated than the workforce in 2000. I had always assumed that the number of people attending college was and will continue to steadily increase, but I had noticed in previous research that the % of 25-29 year olds with a bachelor's degree in 2006 is actually lower than it was in 2000. Given the changing demographics of the country, Marta Tienda concludes that this trend will continue into the future.

The Calm Before the Storm

Noon today marks the first session of the 89th annual conference of the American Educational Research Association (AERA). Close to 20,000 educational researchers (and people interested in education research) will descend on New York City for this year's conference. Through Friday there will be around 120 concurrent sessions every 90 minutes or so. That means that most days there is time to attend 4 or 5 sessions out of about 500 or so.

As with any conference this size there will be sessions that are phenomenal and sessions that are horrible. The trick is to pick the right ones. Given the large selection, there are usually at least a half-dozen sessions centered on topics that interest me and involve somebody who I've read, so I'm hoping I don't pick too many duds this year (I had pretty good luck last year).

I'll be posting anything that I find extraordinarily important/interesting, but I don't know when or how since the conference is hosted by a number of hotels -- all of which likely feel the need to charge extra for internet access despite charging exorbitantly just to stay there.

Thursday, March 20, 2008

More on Inequality and National Achievement

Douglas Willms gave the keynote address last night here at CIES and spoke about increasing both equity and achievement ("raising and leveling the learning bar"). For every country that took part in two international assessments (PISA and PIRLS) he constructed a "learning bar" that showed the relationship between socio-economic status (SES) and achievement. SES was a significant predictor of achievement in every single country, but the slope of the line is different across nations.

For example, Canadians, on average, outscored Americans. But the wealthiest Americans outscored the wealthiest Canadians by a bit (meaning the poorest Canadians outscored the poorest Americans by a lot). This means that the American "learning bar" has a steeper slope -- that SES matters more in the U.S. than it does in Canada. The report of most of the results he discussed can be found here.

He makes a distinction between "raising" this learning bar -- meaning that everybody scores higher -- and "leveling" the learning bar -- meaning that low-SES students improve more than high-SES students, the line becomes flatter, and there is less of a gap between rich and poor. Here's what I found most interesting: his findings dovetail nicely with mine -- countries that scored higher overall also had lower gaps between rich and poor. Additionally, higher scoring countries have distributions that are less negatively skewed -- meaning that there are fewer very low scoring students. In other words, there doesn't seem to be a tradeoff between "raising" and "leveling" the "learning bar" -- or between aiming for high achievement and high levels of equality.

Wednesday, March 19, 2008

National Achievement and Inequality

I'm in New York City right now attending the meetings of the Comparative and International Education Society (CIES). I've been mostly hiding away in my room desperately trying to prepare my presentation for this afternoon (Tuesday). As such, I do not yet have much of interest to report other than about my paper.

My paper did not live up to my expectations for it, but I think my presentation made sense and I found one interesting thing. The basic motivation for the paper was to find out why some countries have a small spread in achievement and some countries have a large spread in achievement. In the end, I couldn't really find much that looked like compelling predictors for whether achievement in countries would be more or less spread out.

But I did find one thing that surprised me. The last thing that I checked on, mostly out of curiosity since it was really only tangentially related to my topic, was whether higher performing countries had more or less equality. I had a couple different measures of variation in achievement, and I compared them to the median score on the TIMSS (an international assessment involving about 50 countries the last time) and found a really strong correlation between equality and achievement (about .8, p<.001 for you stats nerds), meaning that, within the TIMSS countries, that higher achieving countries were distinctly more equal than lower achieving countries. When I compared performance on TIMSS to spread on PISA (another international assessment) the relationship still held and was moderately strong (about .4, p<.001).

The strength of the relationship was of a level that one just doesn't find while doing research, so I was sure I was doing something wrong, but nobody has given me reason to think that this isn't true and I can't think of any.

I don't know what this would look like for different assessments, different years, different subjects, and different ways of measuring variance (or spread, or inequality, or whatever you want to call it), but it's a result that could potentially be meaningful. It's at least as possible, of course, that the result is either meaningless or won't hold up with other data, but I think it's worth further investigation.

As of this moment, I'm seeing that more equality=higher achievement and wondering whether that means what one might assume it means.

Sunday, March 16, 2008

Teachers in Martinique

On Friday I attended a conference for graduate students at Harvard. The conference was run quite well -- if any grad students are reading I'd encourage you to go and present next year -- and the amenities were quite remarkable given the unbeatable $0 registration fee. I saw a number of interesting presentations, but have been quite busy (I'm now in NYC for another conference) so I'm just going to mention one that I found even more interesting than others.

During the last session of the day Nick Gozik, a student from NYU, spoke about his research (I'm quite sure it's his dissertation) that he conducted in Martinique. Martinique is an island in the Caribbean (somewhat near Venezuela) that was a French colony and their education system is still under the control of France. He spent most of a year on the island and says he spoke with half of the high-school teachers in the system, so you can imagine how much information he collected.

Given that this will take him hundreds of pages to explain, I'm just going to briefly summarize two points I found interesting.

1.) He said that teachers repeatedly emphasized that they followed the curriculum because they were professionals (history teachers identified themselves as "historians," etc.) rather than due to any external forces. The current trend here right now is to essentially punish those who don't follow the curriculum by "holding them accountable." Teachers are different in the two different countries, but I found this contrast fascinating.

2.) Even more interesting is that principals were not responsible for evaluating teachers in their schools because they were not experts in most of the fields (e.g. when a math teacher becomes a principal they do not feel prepared to evaluate a history teacher). Instead, inspectors came to the schools occasionally and both trained and evaluated teachers. I do wonder if this model might actually be better than our current model in the U.S. where principals are essentially expected to both know how and actually do everything. We read an article in a class last week about educational administrators lacking deep knowledge of many subject areas and ways to address this. Our class was divided over whether attempts should be made to intensely train principals in all subjects or if instructional leadership should be ceded to an expert in each field (we were thinking perhaps a revision of the dept. head role, but an outside inspector is another interesting idea).

Friday, March 14, 2008

What Should America Do About Math?

I don't have time to think through this in depth, but a blue-ribbon government panel on math has released their final report. I don't have time to read through the report right now (but you can), since I'm still finishing my presentation for tomorrow, but some things jump out based on the article the NY Times published.

- The report lays out when children should know what. This seems like a step in the direction of national standards . . . which could lead to a national curriculum and a national test.

-Apparently American students are the weakest when it comes to fractions and the argument is presented that improving knowledge of fractions should be the main focus. I've read elsewhere that fractions are growing less and less relevant in today's world. I wonder if rebuts or otherwise addresses this claim.

-It argues for a math curriculum focused on deep understanding of the subject matter (fewer topics, more depth). This goes well with Bill Schmidt's criticism that U.S. math curricula are "a mile wide and an inch deep."

-This quote says it all about education research (basically, we can never seem to figure out anything definitively):

“There is no basis in research for favoring teacher-based or student-centered instruction,” Dr. Larry R. Faulkner, the chairman of the panel, said at a briefing on Wednesday. “People may retain their strongly held philosophical inclinations, but the research does not show that either is better than the other.”

Thursday, March 13, 2008

The Federal Government and Education Research

Harvard is hosting a brief conference for grad students tomorrow (Friday) but they started off with a panel discussion on the state of education research tonight. The panel was highlighted by Grover "Russ" Whitehurst, the head of the Institute for Education Sciences (IES), which basically distributes all of the money the federal government devotes to educational research. He has been hailed by some for reforming education research and making it more rigorous and respectables, and reviled by others for limiting the scope of educational research to certain topics and methodologies.

I won't say that all 90 minutes of the panel was enthralling, but a number of interesting points were made. These include:

-Whitehurst noted that the education research community is not a powerful interest group in political discussions of policies surrounding education research. I've definitely noticed that. When you factor in that teachers and principals don't have much say in this either, it really makes you wonder who is controlling education research.

-There was much discussion of the fact that too much education research is not useful for people in schools, but few solutions offered.

-Whitehurst said that he made the decision to focus research funding on studies of academic achievement not because other things aren't important but because they have a very limited amount of money and want to do one thing well before moving on to others. While I'm not convinced they couldn't throw a few million dollars toward other desirable outcomes of schooling, I'm heartened to hear that it has been considered and would say the decision seems rational.

-In the face of severe criticism from Mica Pollock, a professor at Harvard, Whitehurst emphasized that neither he nor the department believe that only quantitative methods or randomized field trials are worthwhile (although he did seem to imply that RTFs answer more interesting questions). This sentence means nothing to you if you're not in education research but, basically, he refutes claims that many have made that he prioritizes certain types of research at the cost of other types that are more appropriate for certain questions.

Alternative Schools

While I was teaching the Bronx discipline was, by far, the biggest problem in our school. So I'm sympathetic to policies that take bold action in an attempt to alleviate discipline problems in schools.

One policy option that's intrigued me is alternative schools -- essentially taking all the kids that severely disrupt learning out of a group of schools and sending them to another school. The idea makes great sense for the initial set of schools: they should have fewer of the worst offenders; fewer ring leaders and bad role models for other kids; and fewer discipline problems (in theory, of course).

The problem is what to do with all the "bad" kids you send to the alternative school. Taking all the worst kids from a city or region and putting them all together is potentially a recipe for disaster.

So I was beyond intrigued when Pittsburgh decided to start an alternative school for students with discipline problems this fall. They contracted out the management of the school to a firm here in Nashville, Community Education Partners, that runs a number of these schools around the country.

I heard a very positive report earlier in the year, but yesterday I was forwarded this article from a local Pittsburgh paper describing chaos and violence in the school. One parent calls the school a "fighting ground," and another a "war zone." Then I woke up today to see a headline in the local paper that read "Nashville Firm Defends Alternative School." Thinking it was about the school in Pittsburgh, I clicked on the link only to see that it's actually about troubles with a school the firm is running in Atlanta. Apparently eight students and the ACLU are suing the company and Atlanta school district for running a "warehouse for poor children of color."

Now, one newspaper article and a lawsuit prove little about the success or failure of the schools run by this company -- one would assume (or at least hope) that the company has some sort of track record of success considering they have received contracts to run 15 schools in 5 states. But it certainly underscores the notion that running a school full of the "worst" kids poses some problems.

I'm not sure if this proves that private companies don't automatically manage schools better or that private companies are subject to more scrutiny when they run schools, but it's an issue I plan to revisit in the future.

Wednesday, March 12, 2008

More on Business-Style Management of Education

I offered some thoughts the other day about business-think in schools, and I noticed this post by Diane Ravitch in the blog she shares with Deborah Meier on EdWeek. It's extremely thoughtful, quite short, and very worthwhile reading.

I share her aggravation over the notion that only people who are not involved with schools are able to fix them and that running schools like businesses is the magic bullet. Not that outsiders and business practices can't offer some valuable insight, but I hardly think that they offer all the answers.

Tuesday, March 11, 2008

Will Bad Teachers Quit for 10K?

Yet another off-the-wall idea, courtesy of the local newspaper. Apparently an anti-union group (The Center for Union Facts) is starting an odd kind of contest in an attempt to stir up conversation about how hard it is to fire bad teachers. The contest? Nominate the worst teacher you know and the center will pick the 10 best entries (or, I guess, worst) and offer them $10,000 to quit. They'll also write about them on their website.

Now the idea of offering severance pay to make it easier to get rid of some bad apples might be a good one, but I'm not sure if they're actually expecting somebody to both quit their job and let their name and misdeeds be exposed for a paltry 10K. I'm sure there are at least 10 awful teachers in the country, but I don't think this is an effective way to get rid of them. To be fair, I don't think the center does either -- they pretty much say that it's a stunt designed to start a national conversation.

What worries me more, however, is the premise behind the idea. While I absolutely agree that bad teachers cause problems in schools, I'm not convinced that there are any more bad teachers than there are bad lawyers or pastors or mechanics. It's odd that teaching gets held to a higher standard in this regard and not in others. I'm also constantly amazed that unions get 100% of the blame for bad teachers -- the narrative being that everybody has their hand tied because unions make it impossible to fire teachers. Is it difficult to fire a teacher? Absolutely, especially one with tenure. Maybe it's harder than need be, but it should be difficult to fire somebody -- particularly in a position where politics can get involved.

Furthermore, I've never understood why it's only the union's fault that a bad teacher doesn't get better or get fired. Isn't it the principal's job to evaluate their teachers? How do principals manage to escape all blame for this? I've never tried to fire a teacher, so I might be wrong, but I think this whole "impossible to fire a teacher" thing is overblown; is it really "impossible," or is it just difficult enough to deter action? Even if it were impossible to fire a tenured teacher, I've seen no evidence that tenure is being used as a screening process. Perhaps if principals or, if they're too busy, somebody else did a better job of evaluating teachers before awarding tenure then there wouldn't be so many bad apples. Part of the reason that the tenure process works (at least a little bit) better at the college level has to be that tenure is somewhat difficult to get; one is thoroughly evaluated by their peers on a number of criteria before being awarded tenure.

The center says they want a national conversation, so here are my two cents:
-Propose a more realistic use of severance pay
-Stop blaming unions for everything
-Hold somebody accountable for evaluating teachers
-Make tenure review a more meaningful process

Sunday, March 9, 2008

Performance Incentives . . . for kids

A lot of attention has been focused recently on performance incentives for teachers and principals. Receiving perhaps more attention, particularly in the NY Times, have been performance incentives . . . for students. Roland Fryer, a young economist at Harvard seems to have caught Joel Klein's ear and is now directing or planning a number of projects in which kids are rewarded in various ways for higher test scores.

One of my colleagues recently wrote that paying kids for test results may be "the reductio ad absurdum of performance pay," though he declined to make the argument himself (reductio ad absurdum basically meaning that it's the logical extension of an argument that is so absurd that it proves the argument is false).

He may have a point. On face, paying kids to do better in school seems quite absurd. What does it teach a kid when the only value of learning is a few bucks in their pocket or more minutes on their cellphone? Whatever happened to learning for the sake of learning? On the other hand, there are other possible benefits. What if the reward makes doing well in school the "cool" thing to do? (this isn't my idea, I read it somewhere and can't remember where -- it might have been in this blog) What if the bonuses work, not because kids work harder for the money but because it removes the stigma of doing well in school? Would the positives then outweight the negatives?

The Irony of Business Intervention in Education

One of the most noticeable trends in education over the past 5-10 years or so has been the increasing role of business-think in schools. People from the business world have become involved in schools and education policy in a myriad of ways: from businessman superintendents (e.g. in NYC and Pittsburgh) to for-profit schools to supplemental educational services to accountability to performance pay, school choice, and many others. As with any trend in governance, the business perspective has brought both good and bad to education.

As a crass generalization, people who argue that schools should be run more like businesses tend to also argue that schools are wasteful and inefficient, that the market should play a larger role in education, and that success and failure can be measured.

The irony is that if schools were run just like the private sector, the changes would infuriate many proponents of business-think. More specifically, think about spending patterns.

Think of the differences between walking into, say, a law firm vs. walking into a typical public school. Which one has a ridiculously expensive conference table? Which one pays for employees to stay in upscale hotels and eat fancy meals? Which one gives employees expense accounts?

It's widely expected that schools (and pretty much all public organizations, for that matter) are supposed to forgo luxury items, make do with second-hand goods, and generally sacrifice and cut corners at every turn. And that's fine in a lot of ways, but it's something that businesspeople tend to forget. If schools were run more like businesses, I find it hard to believe they'd be any more thrifty. If schools were truly run like businesses I think a lot of current critics would be red-faced with anger over their wasted tax dollars. In short: be careful what you wish for.

Friday, March 7, 2008

How to fix the dropout problem?

I try to be open-minded and unbiased when I read a new policy proposal, but I can't help but be excited when I read something that's "outside the box" -- something about new ideas just appeals to my creative side regardless of how good they are.

This blog post and this blog post both pointed me to this article in the Washington Post from Monday that is nothing not "outside the box" (the two blog entries seemed to take opposite sides on the article).

In short, his idea is this: let kids drop-out of high school instead of trying to keep them in, but take the money that would've been spent on their education and put it in an education trust fund that they can access at a later date. Then, at some point in time they realize that dropping out was a dumb idea and they have the resources to finish their education.

The fact that idea, at least in its current form, is politically infeasible makes me hesitate to spend much time analyzing it. One of the biggest problems is that schools don't currently spend the x dollars that it takes to educate one kid when they drop out, so they'd have to raise spending in order to put that amount in a savings account each year. Secondly, they don't save x dollars when a kid drops out -- there are too many fixed costs. Lack of economic feasibility aside, I like the effect such a plan would have on the environment in high schools (all the kids who don't want to be there would be gone) but am doubtful that the later-in-life education would be a panacea or that all problems can be solved simply by giving somebody more options. That said, I do find the idea absolutely fascinating simply because it is so different.

Thursday, March 6, 2008

The Most Compelling Argument Against NCLB

When the No Child Left Behind Act was passed in 2001, it had the potential to do both a world of good and a world of bad for education. Why the law and its implementation has been good or bad (or both) has been the subject of countless articles, commentaries, and debates. Now the law is up for renewal and the debates have become even more focused. A lot of people argue that there is some redeeming quality to NCLB (a lot of people like accountability, for instance, or the fact that that the stated goal is essentially to close the achievement gap) but that it needs to be tweaked. While the cacophony continues in Washington, I was lucky enough to catch a bit of a different argument on campus -- twice.

Richard Rothstein was on campus in the fall, and then was on campus again last week for the performance pay conference and stopped by our class to fill us in on his research. His argument against NCLB is the most compelling (and thorough) that I have heard. It goes something like this:

Throughout American history, virtually every important figure has set forth as the goals of education to not only provide basic academic skills to students, but also to teach many other things, such as: critical thinking, social skills, physical and emotional health, and far too many other things to list here. He has compiled an exhaustive supply of quotes and documents supporting this.

NCLB, meanwhile, holds schools accountable only for their performance on tests of basic skills in reading and math. As a result, many studies have shown, schools are increasing the time they spend on reading, math, and test prep, and reducing the time they spend on other activities.

Given that, throughout our history, basic academic skills have been but one purpose of schooling, Rothstein set out to see how important these skills are compared to other things that people want schools to teach. The general public, school board members, state legislators, and superintendents (through extensive surveying) all agreed that basic skills were the most important function of schooling, but indicated that out of 100 possible points spread across 8 different functions only about 20% should be focused on these. The other 80% of schooling, they responded, should be focused on things such as the list I mentioned earlier.

Here's why I like this approach to critiquing NCLB:
It's not an ideological position. He doesn't argue that standardized testing or accountability are inherently good or evil. What he finds is, quite simply, that NCLB encourages schools to focus on only a small portion of what is important and, for that reason, needs to be changed. In 1830 (to cite one of his incredible list of quotations) the Joint Committee of Philadelphia Workingmen argued that everybody should receive a broad education "rather than being limited, as in our public poor schools, to a simple acquaintance with words and ciphers." Unfortunately this seems to be happening under NCLB; the law needs to encourage schools to teach all the important topics rather than condemning students to learn nothing but the basics.

Tuesday, March 4, 2008

Class Size and the Achievement Gap

I hesitate to say this before having read the article (it's not out yet), but I think I figured out what's going on. This article in was published last Wednesday in EdWeek and describes a forthcoming journal article in which the author claims that smaller classes do not reduce achievement gaps. Meanwhile, another researcher who looked at the same data says that they do. I didn't have time to really think it through the first time I read it, but I think I see the difference now.

Both researchers look at data from the Tennessee STAR project, which is the only randomized trial of class size (students were randomly assigned to a class with either 13-17 or 22-26 students for a few years) and took place about 20 years ago. The author of this article, Spyros Konstantopoulos, says that the gap between the high and achievers was higher within small classes than it was in large classes. In other words, higher achieving students benefited more from smaller classes, so smaller class sizes do not impact achievement gaps.

Meanwhile, another researcher, Alan Kreuger, says that his research shows that lower-performing students and African-American students benefited more, and that, therefore, smaller classes reduce achievement gaps.

What? They're both looking at the same data. How can they reach different conclusions? Barring dishonesty or highly-technical formulas, here's what I think it is:

I think the answer is simple. In the newer article, he looks at the gap between the high and low achievers within each class, not the average score for each class. Meanwhile, the older study looks at the gap between the higher and lower performers across the sample. So the average score for lower performing classes could rise more than the average score for higher performing classes (many, if not most, classes are not extremely diverse -- they're within schools that are in wealthy suburbs or poor inner-city neighborhoods), meaning that the gap between high and low performing classes shrinks while the gap between the high and low performers within these classes actually increases. In other words, the lowest scoring students overall gained more than the highest performing students, but the highest-scoring kids within each class gained more than the lowest scoring kids within each class. So, in short, both of them are right.

If they're both right, to whom should we listen? Does reducing class size work or not? Well, you can argue it both ways. On the one hand, the society-wide "achievement gap" is really what we care about but, on the other, it seems that reducing class size doesn't reduce this in quite the way we'd anticipate. It seems that using the newest study to imply that smaller classes don't reduce the achievement gap is misleading, but that the way these changes affect distributions of achievement within classes offers food for thought. Of course, it would be nice to have a second study, one that's less than twenty years old, to compare to these findings.

Single-Sex Public Schools

The New York Times Magazine ran a very long piece on single-sex education in public schools on Sunday. In short, there seem to be five positions explored: some argue that boys' brains differ fundamentally from girls' brains and that, therefore, they should be separated; some argue that separation is merited in order to end discrimination; some argue that separation is inherently sexist; some argue that separation by gender is only a rough proxy for separation by learning style; and some argue that boys and girls need to interact with each other.

I think each of the five sides has some merit. When this issue came up in our sociology class last year there was some spirited debate, particularly since research has been inconclusive to date. I asked people if research conclusively proved that single-sex schools led to much higher test scores if they would then be willing to enroll their children in these schools; nobody was willing to commit. This says two things to me: that there's more to a good school than high test scores, and that there's something less easily describable driving the opposition to such schools. Personally, I would hesitate to send my (future) children to such a school.

Prior to teaching I would have unequivocally ruled out the idea without so much as a second thought, but my experience led me to reconsider the merits. Teaching sixth graders, I noticed substantial differences between the boys and girls in my class. My personality and teaching style clearly worked better with the girls, and I had significantly more discipline problems with the girls. Maybe it was just the stress, but it often seemed that teaching a class of only girls would've worked a lot better. That said, segregating by sex still seems wrong to me on some level. I wonder if it might be possible to experience the best of both worlds; perhaps have single-sex classes only during middle school or only for certain subjects or activities. Or maybe Jay Giedd is right; gender is too rough a proxy. Has anybody tried separating students into classes based on different styles of learning? Is that feasible or desirable? It would certainly be easier for teachers than the "differentiated instruction" that is currently pushed, but I'm skeptical of any cure-all, especially one that requires some form of segregation or tracking.

Who's Asking What Teachers Think?

This piece in EdWeek caught my eye yesterday: Margaret Spellings is recruiting a team of teachers (5 full-time and 20 part-time) to help refine proposals for changes in education policy at the Dept. of Education starting in the fall. Two thoughts:

1. Wow. Seems like a great idea. Though, to be cynical, anybody could find 25 teachers that agree with their favorite policy proposal if they wanted to (I'll give them the benefit of the doubt and assume that's not what they're doing until I hear otherwise).

2. What momentous changes, exactly, is the Bush administration planning on making in his last few months in office? It's great that they have current teachers to comment on the feasibility of their ideas, but it's hard to believe that they'll have time to implement many of their ideas.

Saturday, March 1, 2008

Is There a Teacher Shortage or Not?

A paper presented yesterday at the Incentive Pay Conference held on campus raised some eyebrows. The authors looked at the relationship between teacher effectiveness and teacher retention -- in some contexts less effective teachers were more likely to leave a school while in others there was no statistically significant difference between more and less effective teachers.

The statistic that seemed to raise eyebrows was that after 5 years about 2/3 of the beginning teachers were still teaching at a public school in Florida (the paper examined 5 cohorts of first-year teachers in Florida public schools over 5 years). One economist suggested that this was a much higher retention rate than any other professional field and that, perhaps, we should be asking why so many teachers stay in the field instead of why so many leave.

The point raised is a valid one, it appears that teacher retention is not a problem in a lot of places and that, taken as a whole, there is no "teacher shortage" in this country. But there's a dangerous caveat that makes saying "teacher retention isn't a problem" just as false as saying that it is; teacher retention is a problem in some areas -- most notably high-poverty schools.

In this particular study, the authors found that only 20% of teachers in high-poverty schools were still teaching in that school in their fifth year and that teachers tended to transfer to schools with fewer minorities, fewer students in poverty, and higher test scores. To be fair, after reading the paper in full, the authors do acknowledge this -- but that doesn't change the fact that the reaction in the room seemed to be agreement that teacher retention was overblown as a problem in America's schools.

Secondly, the utility of an overall number is limited because it remains to be seen how much it helps or hurts a school when a teacher leaves one school in favor of teaching in another. Imagine the most extreme scenario: all teachers remain in teaching for their entire career, leading to an overall retention rates of 100%, but switch to a different school each year; even though teacher retention is not a problem for the system as a whole, it might be within schools.

In summary:
There is no "teacher shortage" in this country, but some schools are short on teachers. Most teachers continue to teach, but not in certain schools. Teacher retention may not be a problem at the school down the street, but it may be at the school across town -- and dismissing it as a problem is just as dangerous as exaggerating the scope of the problem.