Monday, November 16, 2009

Sunday Commentary: How Should We Measure the "Achievement Gap"?

Though the term "achievement gap" was first used to reference something rather specific -- the difference between the standardized test scores of White and Black students -- it's now used quite broadly.  I've heard "achievement gap" bandied about in reference to gaps in test scores and other measures of academic performance and outcomes between members of both different races and different classes.  Which leads me to this question: can we close the achievement gap without closing the "achievement gap"?

In other words, is it possible to successfully raise the test scores of low-SES and minority students to the same levels as other students without actually solving the real problem?  Every time I hear about another miracle school that closed a large portion of the achievement gap, I can't help but wonder this.

And I think the answer lies partially in how we conceptualize the achievement gap.  Is the difference in test scores between different groups the actual problem, or just a symptom of the problem?  For me, and I think for many others, it's the latter.  The actual problem is that too many low-income children live in worse neighborhoods, attend worse schools, are less likely to graduate from high school or college, and are subsequently both more likely to have lower quality-of-life later on and to cost society by committing more crimes, relying more on welfare, etc.  The causes of the difference in achievement are myriad, as are the causes of outcomes later in life -- less knowledge, as measured by standardized tests, are only one of many causes of worse academic and professional outcomes later in life.

If the problem goes beyond test scores, then how should we measure the "achievement gap"?  Do test scores sufficiently capture the problem -- i.e. will a change in test scores beget a change in all the outcomes that concern us more?  My gut feeling is that this won't necessarily happen.  It's not hard to imagine a school in a poor neighborhood where students score well on tests, but are still much less likely to graduate from college, obtain prestigious jobs, etc.  And that hypothetical worries me: what if we closed the achievement gap and nothing changed?  Would we still pat ourselves on the back and move on?

Or perhaps we're looking at this the wrong way.  Maybe changing achievement levels isn't our ultimate goal.  Maybe we should be trying to close gaps in graduation rates or degree attainment.  Or maybe we should be trying to close gaps in non-cognitive skills like self-control or executive function.  Or maybe we should be trying to close gaps in content knowledge (which is different from, though related to, testing reading or math skills).  Would closing any of these gaps be more meaningful than closing the test-score gap?  I'm not really sure what the answer is.  But I suspect that none of them are enough on their own.

To truly close the gap in academic performance and life outcomes, we probably need to close gaps in both academic achievement and a number of other areas.  So if we're going to continue to say that we want to close the achievement gap, let's not define "achievement gap" too narrowly.


Sasha M. Harris said...

This past Tuesday we had our fall parent/teacher conference at my middle school in Brooklyn, NY. It was a typical affair for me. I sat with the parents of nearly 70 different students between the afternoon and evening sessions. I've been told by teachers at other schools that that is a large number. We have a "successful" school based on our yearly test scores. I teach students of all levels; remedial, ELL, special needs, average, and gifted.
Here's what really stood out for me this week:
1. Students with health issues during the early development years lag far behind all others. I discovered that one of my lowest level students was adopted at 6-months from a Latin American orphanage where the child was found malnourished in a corner. He'll probably never catch up. Self sustainability will be a life-long challenge for this individual. And the research really illustrates this point, that health care and nutrition in the early years is vital to success in school. When I take informal straw polls in class of how many students ate breakfast that particular morning the results are usually less then half.
2. Students from turbulent domestic situations struggle with focus and consistency. Often times they are the students who are the most demanding of attention of an emotional nature. These are the ones who act-out the most in class and, often times receive negative attention rather then positive from the teacher (me). Their behavior becomes a drag on the lesson, often creating stress in other students who are better equipped to manage the classroom setting.
3. All parents love their children and want what's best for them, but in urban schools only about half of them understand how to provide the resources that their children need to be more successful in school. Many parents have a desire to be more active participants in their child’s learning process but don’t understand how, work late, are tired from the day, have younger children to deal with, etc. Now, I hate teachers who chose to blame parents first, but when I compare the working class and poor parents of students I teach with my neighbors on the Upper West Side of Manhattan and the parents of my son's classmates at the private school he attends, it's like two different countries. And not just because of the wealth gap. It has to do with all these other issues, early childhood health, domestic stability, and parental guidance.
More and more my thinking on the achievement gap is, why do we continue to believe that closing the gap is even a possibility? It’s not that I don’t believe that we have a social justice obligation to do so, it’s just that we, as a society can’t pretend that schools are somehow the answer to the problem of social, racial, and economic disparity in this country. Schools may be one leg of the table, but there are three others that also need fixing. When will education reform include proposals that tackle a child’s experience inside and outside the classroom? Can we at least begin with the understanding that education happens beyond the hours of 8:00 AM – 3:00 PM?

Unknown said...

Well said, Mr. Harris.

Roger Sweeny said...


All good questions--but they clash with the way we market our industry. I deliberately put that crassly but I think most of us sincerely believe our own sales pitch.

The sales pitch goes like this," Look at the correlation between success in school and success in life. Look at the fact that, on average, someone with a college degree earns up to a million dollars more than someone without a college degree.

"Those higher earnings, that success in life, is a direct result of school. In particular, it is a result of the subject matter knowledge gained in school.

So if you are a young person, you should stay in school and learn what we teach. If you are a voter, you should vote for candidates who will give us all the money we need. (And, says one powerful voice from the ed schools, if you are a teacher, you should not grade on the basis of "homework done" or "work turned in on time" but only on the basis of subject matter knowledge acquired, as determined by some assessment.)

You see, if you give us enough money, and compulsory attendance laws and such, American teachers in American schools can teach anyone and make anyone a success."

That is a powerful, and very successful, pitch.

To admit that success in life has much to do with what young people bring to school, to admit that grades on assessments may not have much to do with long-term acquisition of knowledge, to admit that most of the subject matter knowledge of school is never used in later life, that would make a lot of us feel terrible. And eventually cost us a job. Or, at least, the hope of a pay raise.

Corey Bunje Bower said...

RS: There's a difference between saying "right now, non-school factors are more predictive of academic achievement than are in-school factors" and saying "there's nothing schools can do to make up for deficits that begin and widen outside of school"

Roger Sweeny said...

I agree. Schools can make up for some deficits. Unfortunately, our sales pitch says that schools can make up for all deficits, as long as we are given "adequate resources."

That is untrue, and it makes it very difficult to seriously consider what we actually can do.

Unknown said...

Just read your article on the achievement gap. Good stuff. Thanks, Corey!

Corey Bunje Bower said...

Got a little behind on moderating comments . . .

Thanks, Shea!