Sunday, January 25, 2009

Sunday Commentary: Excuses, Defeatists, and the Reality of Low Achievement

by Corey Bunje Bower

In-School vs. Non-School Factors

If there's anything upon which education researchers agree it's that student achievement is influenced more by non-school factors than in-school factors -- and the evidence is overwhelming. But, for some reason, saying this merits serious criticism in some circles.

On July 4, 1966 the government released what came to be known as "The Coleman Report," a comprehensive study that was commissioned in order to prove that more funding would improve the plight of African-American students. Instead, Coleman found that non-school factors were far more important than in-school factors -- which is why they attempted to minimize publicity by releasing it on a national holiday.

Seymour Martin Lipset, a fellow at the Hoover Institute, was overheard summarizing the results to Senator Daniel Patrick Moynihan thusly: "Guess what Coleman’s found? Schools make no difference; families make the difference."

Indeed, the liberals in Congress were devastated at the results -- they had expected to find that achievement gaps could be solved through more school funding. Meanwhile, conservatives beamed -- the report proved that a strong family was the root of a strong society and that government could not spend its way out of the problem.


Today's Version of the Dispute

At some point between 1966 and 2009 the sides seem to have flipped. The more conservative position now holds that schools can close achievement gaps if people work hard and don't make excuses, while the more liberal position is that we need to fix the underlying causes of the achievement gap (which mostly lie outside of school) before we can truly solve the problem. This is, of course, an oversimplification -- but what's struck me is the reaction of those who argue the latter.

In the original press release announcing the release of David Whitman's book (the new version has been toned down) the Fordham Institute included a passage that lumped together "defeatists" like Charles Murray and Richard Rothstein. I must say that it had never struck me to lump together one of the authors of The Bell Curve, (a conservative who essentially argues that many kids just can't hack it and we should train them for careers instead), with Rothstein (a liberal who argues that we need to address inequality in society as well as in schools if we want to close the achievement gap). I later found out that Fordham wasn't the first to characterize these two as "defeatists -- Jay Greene beat them to the punch.

This classification of people as "defeatists" seems to be a running theme in rhetoric surrounding education policy. Increasingly, many are describing ed policy people as belonging to one of two camps; those who believe that everybody can achieve if we work hard enough and those who make excuses.

In a December op-ed, David Brooks, characterized the battle as one between reformers and defenders of the status quo and pleaded with Obama not to hire Linda Darling-Hammond as Secretary of Education so that the world might not come to an end. He's far from the only one to characterize the split this way -- David Whitman penned an op-ed in the Huffington Post a few days prior using much of the same terminology, and referring to the latter position as "the defeatist view of school improvement." The inflammatory rhetoric has led others to retort that the rift is really one between deformers and realists.


Reality

But, back to my original contention: if there's anything upon which education researchers agree it's that student achievement is influenced more by non-school factors than in-school factors -- and the evidence is overwhelming. If you know an education researcher who would disagree with this statement, I'd like to meet them -- because I don't know any. Over the 42 1/2 years that have elapsed since the Coleman Report those findings have been replicated countless times. That non-school factors are, on average, much more important than in-school factors is simply not up for debate.

Why? We know that a large achievement gap between races and classes exists before students start kindergarten. We know that this gap widens during the summer and stays pretty stable during the school year. We know that attainment and achievement are much more strongly associated with SES level, race, parental occupation, etc. than any in-school factor. We know that when we control for so-called background variables that there simply isn't much variance across schools. We know that students spend only about 22% of their waking hours in school. In other words, we can come a lot closer to guessing how well a student will do on a standardized test if we're given their background information than if we're given information on what type of school they attend.

And this makes logical sense. If the student bodies of a wealthy, Suburban school and poor, inner-city school switched places we'd expect the inner-city school -- with the same staff and resources it previously had -- to out-achieve the Suburban school.

Somehow, in their determination to improve schools, some people seem to have gotten lost. I know when I started teaching I believed very firmly that I could turn around the life of just about every student with whom I came in contact -- regardless of what else was happening in their life. I believed very firmly that improving schools would fix a lot of society's problems. And when I started hearing research findings indicating how difficult it is for schools to dramatically affect students, I balked. But, eventually, I came to understand why this is.

Every student responds differently to every aspect of a school -- who their teacher is, what sports are offered, what color the wall is painted, etc. The worst teachers still help a few students grow exponentially while the best have some students that don't progress. So attending a school that is "bad" or "good" won't magically affect every student in the same way; students succeed at the worst schools and fail at the best. And it became painfully obvious the more I taught that what was happening at home (or at least outside of school) influenced the lives of most of my students more than I ever could.

In NYC, a very strong correlation (r = .8) exists between the percentage of students eligible for free/reduced-price lunch and the percentage of students passing state English tests. Why? The obvious answer is because the students in these schools come from families and neighborhoods that are often in disarray -- they don't have shelves full of books to read; their parents don't have time to read to them every night or take them to the library; they're exposed to far fewer words and different grammatical norms; they have no quiet place to do homework; they miss days of school to look after younger siblings; they have more health problems; and so on.

But, according to some, this isn't what causes low achievement -- it's poor-quality schools and teachers that make excuses rather than teach. In order to make a serious case for the latter, you'd have to believe that the worst, least-dedicated teachers and administrators systematically distribute themselves so that there are more of them in schools with more kids from poor families; that teachers in the South Bronx routinely put forth less effort than teachers in Chelsea, and that their level of effort and competency correlates almost perfectly with the number of kids from impoverished families at the school. This is simply not plausible.


Reality vs. Rhetoric

Now, where it gets interesting is how people react when others say such things. One of the reasons that I reject the notion that there's truly a firm divide between two camps of education policy is because people on different "sides" often believe similar things -- they just use different rhetoric. To be sure, different people believe different things: some believe that charter schools are the way to the promised land, while others claim that choice solves nothing; some believe that unions cripple schools while others believe that unions create a less abusive environment -- and so on. But I reject as false the current dichotomy that many pushing as an accurate description of world of education policy world; the largest split is rhetorical. People on both sides agree that:

Schools are important, but are not omnipotent. All students can succeed, but not all will (depending, of course, on the metric being used). Hard work and dedication can improve schools, but effort and success will not correlate perfectly. We should do everything possible to prevent students from being left behind, but me must also realize that there is no silver bullet.

I value the insights of many people across the spectrum, and I truly believe that the jury is still out on most of the reforms that are pushed by various groups. But it's clear that one side is winning the rhetorical war despite the fact that when you take their rhetoric to the logical end that it simply doesn't stand up to reality.

Regardless of your preferred school reform, let's tone down the rhetoric: When somebody argues that we should improve students' lives outside of school to improve their performance, they're not making excuses or being a defeatist. Likewise, when somebody says that schools can make a big difference, they're not being wholly unrealistic. Rather than calling each other names we should be working to improve schools. And everybody needs to remember that although a school is not usually the largest influence on the life of a child that it doesn't mean that a school cannot do an awful lot of good.

What too few people are willing to admit is that we can start with the fact that non-school factors are more important and build on it in multiple ways -- none of which involve simply giving up. The fact that non-school factors are currently more important could be used as an impetus to radically enlarge the role of schools in the lives of many children. A number of the most successful inner-city schools dramatically extend the school day and year -- the SEED Foundation has even created boarding schools in Baltimore and D.C. In other words, one way to address the situation is to increase the influence of schools on students' lives. Another direction we can go with this is to attempt to improve the lives of children outside of school, whether through the construction of health clinics, the improvement of housing conditions, the creation of tutoring programs and such, or in other ways. Both are fundamentally trying to do the same thing: provide students with a different life experience, expose them to different societal norms, and subsequently improve outcomes.

People who advocate either of these positions have a ton of evidence to support them; they are both eminently reasonable positions. But for some reason, people who support the latter are under rhetorical attack. And this attack is both unproductive and illogical. The reality of the situation is that the problems facing low-income children are huge and small solutions are not enough; this is reality, no matter how badly anybody wants it not to be. And it's possible to both face this reality and help children.

______________________________________________________
Corey Bunje Bower is a Ph.D. student in education policy at Peabody College, Vanderbilt University. Before beginning his studies he taught sixth grade at a low-performing middle school in the Bronx that has since been shuttered. His research focuses on issues surrounding high-poverty, urban schools including teacher retention, discipline, and school climate.
Sunday Commentary is a running feature on Thoughts on Education Policy. Submissions are open to all who are knowledgeable about education and willing to write a concise, thoughtful piece. Submissions may be sent to corey[at]edpolicythoughts.com.

New Feature: Sunday Commentary

Today marks the roll-out of a new feature -- the Sunday Commentary. The first commentary will appear shortly and will be written by me. In the future, I hope that I will share authoring duties with others who have insights into the worlds of education and policy.

In other words, I am soliciting contributions from outsiders -- possibly even from you. Yes, not all thoughts on education policy will be my own in the future. I'll still do most of the day-to-day blogging on the latest news and whatever pops into my head, but I hope that others can help me lift the dialogue to a new level.

Here are the rules:
Sunday Commentary will feature pieces that are a little more in-depth and polished than what normally appears here (in other words, I'll proofread before I post). Sticking with the theme (and title) of the blog, pieces will only appear if I deem them thoughtful -- just because they are commentary on an issue does not mean they need to sink to inflammatory or overly one-sided rhetoric. Some pieces will take a firm position while others will simply explore a number of options, but all will do so in a manner befitting polite discussion.

I welcome submissions from all who are involved in or knowledgeable about education policy and have some thoughts to share with the rest of us. I envision that most will run the equivalent of about 1-2 pages in a word processor -- enough to explore an issue in some detail without turning into a novel. I'm not big on word counts, but I'd say in the range of 500-1,500 would be a good. Feel free to e-mail submissions (or ideas for submissions) to me -- and please encourage anybody from whom you'd like to hear to submit a commentary. And, yes, I'll consider publishing contributions under a pseudonym or anonymously -- as long as you have a reason and it's made clear if there's a conflict of interest.

Update: You can read all of the past Sunday Commentaries by clicking here

Friday, January 23, 2009

More Random Thoughts

-Maybe I was right -- maybe an Obama presidency will shrink the achievement gap. A recent study found that the black-white achievement gap disappeared right after Obama's nomination and election but was present at other times during that period when Obama was not in the news. And African-Americans who watched the events had a bigger boost than those who didn't (NY Times article here).

-If there's anything upon which education researchers agree it's that student achievement is influenced more by non-school factors than in-school factors -- and the evidence is overwhelming. But Eduwonkette took an awful lot of heat from commenters when she made a policy wish based on that evidence.

-Alexander Nazaryn has an interesting post about teachers and alcohol on the NY Times "Proof" blog. I'm not sure it's possible for non-teachers to understand just how draining teaching is. I didn't drown my sorrows in alcohol, but happy hour was definitely a popular place on Friday afternoons for teachers at my school as well.

-Stay tuned for a new feature this weekend. Next week's topics will include the piece on Michelle Rhee I promised a month ago.

Monday, January 19, 2009

Some Random Thoughts

I've gone AWOL as I've been absorbed in a few different projects, but here are a few things I've noticed lately:

-The Economist randomly slipped a section on school discipline into this week's column on America, which reads in part: "In New York City, where more than 60 bureaucratic steps are required to suspend a pupil for more than five days, teachers are so frightened of violating pupils’ rights that they cannot keep order."

-In a travesty, the 2008 Weblog awards for Best Education Blog includes neither Eduwonkette nor Bridging Differences, which I consider to be the two best education blogs around (the former for its quick-hitting insights and the latter for its unsurpassed thoughtfulness).

-I was, however, sent a link to this ranking of education blogs (which I'm 50/50 on whether or not it was spam intended to get me to click on their ads) that includes both Eduwonkette and this blog.

-Social promotion is often villified -- and I find it hard to disagree with much of the criticism -- but I have yet to come across a single piece of research that finds benefits from holding back students (note: I have not conducted an exhaustive search, so this doesn't mean it's not out there). Has anybody else seen something?

-The absurdity of the rhetoric from some people who consider themselves part of the "no excuses" crowd seems to increase on a consistent basis and is really starting to irk me (which is part of the reason I've taken the last month two off from reading most of the education blogs). I might just insert this into every post from now on: if there's anything upon which education researchers agree it's that student achievement is influenced more by non-school factors than in-school factors -- and the evidence is overwhelming. Please don't call me a defeatist for saying that if you want me to take you seriously.

-I plan on rolling out a few new features this year and hoping to start the first one this weekend. Stay tuned.

Monday, December 22, 2008

Is there such a thing as a "Bad Teacher" or "Good Teacher?"

An awful lot is made these days about the need for high-quality teachers in our schools. And I agree: we need good teachers in our schools. But I'm not sure that "good teacher" really means anything -- and the same goes for "bad teacher."

Kevin Carey wrote earlier about recent findings that value-added measures are inconsistent and identify the same teachers as very good or very bad depending on the year or test on which they're based. This means at least one of the following two things:

1.) The tests aren't very precise at measuring student growth
2.) The amount of student growth under the same teacher varies in different years

Let's focus on number two. If the same teacher can get good test scores one year and bad the next (or vice-versa) it may be the case that their level of success depends on many things besides their teaching ability. In other words, the particular kids they have, the dynamics of the class, the dynamics of the school, the teacher's personal life, the subject matter, the weather, and who knows what else may affect the quality of a teacher in any given year.

This makes sense to me. I'm good at teaching kids how to swim faster; I'm not good at teaching them how to dance. I can teach 6th graders about gravity, but I'd be an awful AP Physics teacher. It's easy to see that being a good teacher for one subject or age group does not mean that you'll be a good teacher for another subject or age group. And it makes perfect sense that somebody could be a good teacher for Class A and not for Class B -- or in School A but not in School B. I'd argue that teaching different subjects in different schools across different age groups requires very different skills (not to mention different areas of expertise).

In other words, it's likely that a teacher's success in a particular classroom is due at least as much to contextual conditions as it is to any innate ability. When declaring that a teacher is "good" or "bad" we should take these things into account.

Bottom line: finding the best fit for a teacher may be easier than simply trying to find the best teacher.

Friday, December 5, 2008

A Brief Update . . . Finally

I've disappeared because I find myself in the middle of yet another finals week (my final one with a full courseload) -- when fatigue makes bad jokes and puns seem funnier (punnier?) and cynicism runs rampant. I'll wait until after I'm thinking clearly again to post some thoughts on:

-Michelle Rhee, the intimidator
-"Defeatists," the enemies of David Whitman
-False Dichotomies, David Brooks' specialty when writing about education
and other pressing issues . . .

But for now I'll just chuckle at the headlines in the newest journal of education, EdTweak (e.g. "New Study Links Teacher Unions with Satan"), eat some cookies (side note: don't use whole wheat flour when making chocolate chip cookies -- it's just not worth it), and get back to work.

Wednesday, November 26, 2008

More on "Broken Windows" and Education

I wrote a couple of days ago about the "broken windows" theory and education based on the recent article in Science. The theory essentially holds that small amounts of disorder will lead to more problems b/c people come to accept greater levels of disorder over time as it becomes the norm. And this certainly happens in schools as well as in neighborhoods.

But the authors make a bit of a finer distinction within the notion of disorder and norms. They divide social norms into "injunctive norms" and "descriptive norms" -- with the former representing, essentially, what people believe society deems acceptable and the latter representing solely what people observe, whether they believe it to be acceptable or not. The two are often in opposition -- for instance, if you think that speeding is a bad thing but also believe that almost everybody does it. In such a case I find it odd to call following speed limits a "norm" when you normally see people doing otherwise. I can be convinced otherwise, but I think the two would be better characterized as rules (whether written or unwritten) and norms. One dictates what is acceptable and unacceptable behavior and the other dictates what is normal and abnormal behavior.

The experiments test both how behavior changes when people witness rule-breaking and how it changes when people perceive disorder as a norm. In the case of the former, people are more likely to litter when they hear illegal fireworks being shot off than when they don't. And in the case of the latter, people are twice as likely to steal an envelope with money in it when the mailbox is covered with graffiti or surrounded by litter.

In other words, there are two different ways in which disorder can affect a school: In the case of rule-breaking, when students see other students chewing gum, carrying cell phones, or breaking other rules that might not really be enforced they come to believe that rules, in general, aren't really enforced or important and are more likely to break them. In the case of degradation of norms, when students see others running, shouting, pushing, etc. they're more likely to believe that chaos (or at least unruly behavior) is the norm and conform accordingly. Or at least that's how my experience would lead me to believe their research translates to schools.

So what are the implications? I'd guess the vast majority of teachers and administrators would agree, at least to some extent, with the previous paragraph. But knowing that disorder begets disorder and stopping disorder are two different things. I don't have all the answers, but I would say two things are most important:

1.) Don't make rules that can't or won't be enforced. If you're not actually going to suspend a kid every time you see them carrying a cell phone, then don't say you will. And in the case where the principal sets the rule this applies not just to you personally, but to the staff collectively. If all teachers aren't going to enforce the rule, then it's probably not a good rule to have. If kids see other kids playing on their cell phone in class, and those kids aren't suspended the next day then nobody's going to take that rule seriously. And once kids learn that not every rule has to be taken seriously, every other rule is in peril as well.

2.) Take action against low levels of disorder. That means somebody talking out of turn in class, yelling in the hallway, refusing to do assignments, etc. Maybe rules against such aren't codified, but when something small like this occurs it's important that the student is both made aware that it's unacceptable and that the behvaior is nipped in the bud to whatever degree possible.

Implementing number two is a task that I don't envy. What happens when a kid is made aware that his/her behavior is unacceptable and they continue it anyway? Some sort of consequences must follow. What happens when the student isn't bothered by the consequences and continues the behavior? Now the school's in a pickle (unless, of course, they can expel the student -- but even then such a decision should not be made lightly).

Tuesday, November 25, 2008

Follow-Up on Pittsburgh's Grading Policy

Overheard today in a Pittsburgh school:

Teacher (to Student A, who was refusing to do work): "How many points do you get if you do nothing?"
Student B: "Zero"
Student A: "Fifty"
[Student A continues to refuse to do the assignment]

And that's why I whined so much about Pittsburgh's new grading policy when it was implemented (here, here, here, and here)

Monday, November 24, 2008

Schools and "Broken Windows"

I think virtually everybody is familiar with the "broken windows" theory of crime prevention made famous by Rudy Giuliani -- the notion that small amounts of disorder (e.g. broken windows) can, over time, foster more crime and that the way to prevent large amounts of crime is to crack down on small things before they snowball.

Well, now some researchers have conducted a series of experiments to test out this theory (hat tip: Ideas Blog). The experiments were small-scale. For example, they left an envelope with a 5 euro bill in it sticking out of a mailbox. People were twice as likely to steal the money if mailbox was covered in graffiti or surrounded by trash (abstract). Both the full paper and the article summarizing it are behind a subscription wall, but The Economist has a good synopsis of the findings here.

None of the experiments involve schools, but they do involve people breaking rules and not following directions -- something that happens an awful lot in some schools. The broken windows theory has received a lot of criticism, but I think a lot of this may be due to the way that policies were enacted rather than the underlying truth of the theory. I'd welcome hearing from somebody who disagrees, but it's hard for me to see the general theory as anything but intuitively true.

If you buy the theory, it leads one to believe that we should implement "zero tolerance" policies in our schools. But such policies have become a lightning rod for criticism. Why? Probably because they're zero tolerance for major infractions like bringing weapons to school and sexual harassment. Kids being suspended or expelled for bringing plastic knives or for kindergartners touching members of the opposite sex in ways they couldn't really comprehend was sexual has led to widespread disgruntlement with such policies.

While weapons and sexual harassment are huge problems and should be dealt with as such, they're far from the largest obstacles to learning on a day-to-day basis in most schools. That honor would go to minor issues like talking in class. I would argue that these are the true "broken windows" of schools.

And I think a lot of teachers know as such. A lot of the discussion amongst teachers in my school centered around relatively minor incidents that they believed served as tipping points in their classroom. For example: a teacher next door to me my first year had a student that was momentarily out of control and clearly crossed the line between acceptable and unacceptable behavior. Just then the Asst. Principal stepped into the room and asked if the teacher needed anything. The teacher replied that they needed the AP to remove the student from the classroom for a few minutes. The AP declined to do so. And from that point forward, students knew they could get away with things that they had previously believed they couldn't.

Now, that's not really a true fit for broken windows theory, but I could tell a thousand other similar stories. The bottom line is that it was quite clear to me that students were more likely to act up when they perceived that such actions were acceptable b/c they seemed to be the norm in the classroom, hallway, cafeteria, etc. As such, it seems that a zero-tolerance policy of some sort is wholly merited. How else to prevent the chaos that reigns in too many (note: "too many" does not imply anywhere near a majority) schools today. But how to implement one?

In other words, can a school implement a zero-tolerance policy in a way that is neither draconian nor militaristic? The closest I've seen to such policies (though I'm not sure they're 100% non-draconian/militaristic) is in the so-called paternalistic schools that David Whitman profiles. The schools go to great lengths to ensure that not a single infraction goes unnoticed, no matter how small. I'm not sure that all schools confronted by discipline problems should (or can) mirror these policies, but I think the theory behind them is sound.

Friday, November 21, 2008

Final Problem Set

I am currently in the process of completing what I believe to be the final problem set of my grad school career (which is why I won't be posting anything in the near future). Given my apparent prowess at basic statistics, however, I'm wondering what the odds are of me being exempted from this assignment (bad statistics joke omitted for your protection).

Perhaps the better question is how much longer this will take me to do if I spend any more time trying to get out of it.

p.s. to any of you having a miserable Friday . . . at least you're not converting log odds to probability in hierarchical linear models