Attempts to Change HOPE VI Residents' Aspirations and Behaviors

A little bit on the research I presented today at ASA:

HOPE VI (Housing for People Everywhere) is a federal housing program signed into law in 1992, following the report of the Commission on Severely Distressed Public Housing. In short, the program razes dilapidated housing projects and replaces them with new mixed-income housing. But not only is the quality of the housing higher, the rules are also stricter. In order to reside in a publicly subsidized unit, a resident usually must prove that they have a job, a good rental history, and undergo a criminal background check. In this way, to put it crudely, the program aims to improve neighborhoods and improve people. But virtually all of the research I've seen has been on the former (my explanation: the latter is politically incorrect to discuss).

Please note that I'm not advocating a particular course of action or taking a position on whether or not public housing residents should change the way they lead their lives. I'm simply observing that a number of people are implementing a course of action with this as their goal and exploring how residents are reacting to this.

My study is a qualitative analysis of 24 interviews with residents of one HOPE VI development and explores how they react to attempts to change their behaviors and aspirations.

One of the problems with qualitative research is that it's quite difficult to summarize. The synopsis I passed out is here, full of tables and figures to help explain what I'm saying (Blogger is great, but for some reason they have yet to make it easy to insert tables or copy pictures into posts).

Anyway, here are the basics on what I found:

-The rules that are in place appear to indicate an attempt to enforce upper middle-class social norms on residents (for example, residents reported that they weren't allowed to own pit bulls, grill in their front yard, or fix their car in the street). And the rules are zealously enforced. Management frequently patrols the neighborhood with camera in hand. Small infractions (e.g. leaving a trash can curbside past the day of collection or having visible clutter outside one's house) are dealt with by immediately notifying the resident that they've been fined $25 and placing a photo and summary in their mailbox.

-Management is actively attempting to change residents' behaviors and aspirations -- and most residents are aware of this. Residents must take part in a home ownership class before taking up residence in the neighborhood and they also report meeting with counselors to set goals -- which are discussed in follow-up phone class. When residents purchase their own homes, it's publicized in the community newsletter. As one resident puts it, the development "was built for you to know to be self sufficient, gen on your feet and then move along."

-While some of the residents bristle at the strict rules ("they hold our hand to the fire" says one), complaining about "big brother," most of the residents interviewed had more positive responses. A number of residents reported that their neighborhood was clean, quite a bit better than their old neighborhood, that neighbors were responsible, and that they felt peer pressure to keep things neat and orderly. As one resident puts it, "this is not the projects anymore . . . it's homes, you know?"

-There was limited evidence that, to some extent, the rules and processes in place were leading to desired changes in residents' behaviors and aspirations. For example, various residents reported: not littering because they would be fined for it, saving money for a house after their home ownership class, and becoming better at budgeting since they had to pay their own utility bills.

Research on HOPE VI is decidedly mixed, with the biggest knock being that few of the people living in the neighborhood before redevelopment are allowed to move back in. Accordingly, most of the interviews I analyzed were of people who lived elsewhere before moving. While the evidence was decidedly mixed -- and the sample quite small -- I would say evidence I examined is more positive than negative. The neighborhood seems safer and cleaner than before, rules are routinely enforced, and residents are doing some things that the writers of the legislation would be happy about.

Now, you might be asking yourself what the heck this has to do with education. Well, as we all know by now, if there's anything upon which education researchers agree it's that student achievement is influenced more by non-school factors than in-school factors -- and the evidence is overwhelming. And even though most people don't talk about housing programs as educational interventions, the behaviors the policies seemed designed to elicit are similar to those that an educational intervention might aim for. The neighborhoods appear quieter and more orderly -- both neighborhood characteristics that are positively related to academic performance.

How Bad do the Students Think Their School is?

Here's the synopsis of the paper I'll be presenting at ASA today:

Almost every reform of high-poverty urban schools assumes one thing: that these schools are bad and everybody knows it. Why, for example, would a kid want to work harder, attend more days of school, sign-up for tutoring, or apply to a different school unless they thought there was something wrong with their current school and the way they're doing things now.

We in the policy realm are absolutely certain that these schools are hellholes that doom the kids to a lifetime of underachievement. But I wondered what the kids thought. So I asked them.

In a pilot study, I surveyed 79 students in college-prep and vocational classes at a high school on the brink of closure. The school is the poorest in its district and has a graduation rate that has dipped below 50% in recent years.

To summarize, the kids reported that their school was about average, that other schools weren't much better or worse than theirs, and that their student body didn't differ significantly from other schools. Kids reported that that students in their school graduated at a slightly above average rate. When asked to estimate the percentage of kids in their school that were African-American, they were spot-on (86%), but when asked to estimate the kids in all other schools in the country that were African-American they were a little off -- the mean response was 68% (actual figure is about 14%).

Perhaps I shouldn't be reporting results of a pilot study yet when I still have a lot of red tape to get through before I can start the final version (please don't steal my idea, I'm just a poor grad student who needs to build up his CV), but I find it interesting.

The reason I'm presenting it at a Sociological conference is because my guess as to what is happening relates to sociological theory (look up social construction or status construction if you're interested). My best guess, based on a very sample, is that people tend to assume that their immediate surroundings are normal unless their is explicit evidence to the contrary. In other words, the only way these students would think their school was abnormal was if they were exposed to a number of other schools that were quite different -- something I'm not sure happens. Of course, it could also be the case that the school is simply not as bad as we outsiders think.

How Does Teacher Retention Affect Schools?

I've been on the road for two weeks now and all the conference presentations are starting to muddle together in my head. As such, I'm just going to talk briefly about the paper I presented yesterday. It's one of the few things that I can keep straight in my head right now.

Here's the premise of the paper:

Lots of people have investigated why teachers leave schools, but I haven't seen much on how schools are affected by teachers leaving. The simplest version of this question would be "does retaining more teachers improve student performance?"

As I've discussed before, the nature of teacher retention is very different in different schools. Since retention rates are notably low in high-poverty urban schools, I chose to focus solely on them. I had some data and was able to compile a dataset of 43 NYC middle schools that had large percentages of poor students (not including magnet, K-8, and some other schools with unreliable statistics). Across these schools, about 40% of teachers were in their first or second year of teaching at their current schools and less than half had a total of 5 or more years of teaching experience.

There was also a moderately strong (r=.44, p<.05) relationship between the average student score on the 8th grade state math test and the percent of teachers who had been teaching for at least two years at that school. In other words, schools with higher rates of teacher retention also had higher student achievement.

Using regression analysis and controlling for race and attendance rate, teacher retention was still significantly related to student achievement. Teacher retention had a fairly sizable effect (effect size of .20) and the model did a pretty good job of explaining the variance in test scores (R-squared of .70).

So, in short, among high-poverty middle schools in NYC, those with higher rates of teacher retention also had higher test scores, even when controlling for other things that influence student test scores. The question that I can't answer with the data is whether better schools make teachers want to stay there more or if more teachers staying in a school improves the school and boosts student achievement (or a little of both). I hope to come closer to answering this question in the future.

Discipline: The "Dirty Little Secret?"

I often say that I spent two years trying to teach in the Bronx. Why trying? Because discipline problems in my school were so severe that it was sometimes hard to believe I was teaching much at all. I have no evidence on exactly how typical my school was, but it was abundantly clear that discipline issues were, far and away, the biggest problem.

I spent most of the day in a training session on how to use international datasets rather than attending sessions, so I was planning to go to sleep and not bore anybody with the details, but I just happened to notice Diane Ravitch's latest post right before turning out the lights.

Ravitch basically writes that discipline problems are a major problem in the United States b/c kids don't respect adults and that other countries don't have the same problem (she, of course, says this more eloquently). This immediately stirred a number of thoughts:

1. Based on my experiences, I couldn't agree more. Discipline is a major hurdle for a number of schools. It is much more important than standards or curriculum in these schools, but seems to be researched far less frequently. Is discipline not viewed as a serious problem by academics b/c it's limited to only some schools, b/c it's seen as more of an issue for the popular press, or something else?

2. Based on my research, I cannot uphold her claims. I've found zero evidence that the U.S. is an outlier in terms of discipline problems. There are, however, at least two major problems with my research so far: 1.) The international data on discipline is far from conclusive and 2.) I just found out today that the TIMSS international assessment does not use a nationally representative sample of teachers and, therefore, saying x% of teachers say that behavior is a major problem is not a valid statistic (I have to look at how many students have teachers who said this instead).

3. I wonder how widespread this problem is and how different it is from previous generations. The first question is somewhat easily answered while I'm not sure that it's possible to obtain an answer to the second. The U.S. did stand out in one way from other countries in my research: there was a stronger relationship between problems a principal reported in a school and the SES of the students (schools with poorer students reported both more frequent and more severe problems) than in any other country. I know discipline was the major issue in my school. I know discipline is the major issue in other schools where friends have taught. But I'm unsure how much of an issue discipline is across the country. Do adults always think that kids are less respectful than they should be, or do we have a real problem across the country?

p.s. I stole "dirty little secret" from a commenter on the Ravitch blog entry. I've used similar terminology in the past, but am too tired to think of exactly what it was. I may have more to say on this topic tomorrow.

National Achievement and Inequality

I'm in New York City right now attending the meetings of the Comparative and International Education Society (CIES). I've been mostly hiding away in my room desperately trying to prepare my presentation for this afternoon (Tuesday). As such, I do not yet have much of interest to report other than about my paper.

My paper did not live up to my expectations for it, but I think my presentation made sense and I found one interesting thing. The basic motivation for the paper was to find out why some countries have a small spread in achievement and some countries have a large spread in achievement. In the end, I couldn't really find much that looked like compelling predictors for whether achievement in countries would be more or less spread out.

But I did find one thing that surprised me. The last thing that I checked on, mostly out of curiosity since it was really only tangentially related to my topic, was whether higher performing countries had more or less equality. I had a couple different measures of variation in achievement, and I compared them to the median score on the TIMSS (an international assessment involving about 50 countries the last time) and found a really strong correlation between equality and achievement (about .8, p<.001 for you stats nerds), meaning that, within the TIMSS countries, that higher achieving countries were distinctly more equal than lower achieving countries. When I compared performance on TIMSS to spread on PISA (another international assessment) the relationship still held and was moderately strong (about .4, p<.001).

The strength of the relationship was of a level that one just doesn't find while doing research, so I was sure I was doing something wrong, but nobody has given me reason to think that this isn't true and I can't think of any.

I don't know what this would look like for different assessments, different years, different subjects, and different ways of measuring variance (or spread, or inequality, or whatever you want to call it), but it's a result that could potentially be meaningful. It's at least as possible, of course, that the result is either meaningless or won't hold up with other data, but I think it's worth further investigation.

As of this moment, I'm seeing that more equality=higher achievement and wondering whether that means what one might assume it means.