Part 1: Background Information
According to Eduwonk, results from the Nashville incentive pay experiment are due to be released soon. I've been meaning for a while now to write up some background information on the experiment so that we have some context when the results are released, so this seems like as good a time as any.
The National Center on Performance Incentives was started in 2006 with a 5 year, $10 million grant received from the Department of Education's Institute for Education Sciences. The center is housed at Vanderbilt University's Peabody College and run in conjunction with various partners, including the RAND Corporation and the University of Missouri. Peabody's Matthew Springer and James Guthrie (now of the George W. Bush Institute for Public Policy) are the directors, and the center is staffed by people from a range of institutions across the country (full list). The funding was to cover two experiments plus other related costs. The first experiment was conducted in Nashville from 2006-09 and was dubbed the Project on INcentives in Teaching (POINT).
The center started at Vanderbilt the same time that I did, and I worked there during my first year (2006-07) to earn my keep around here. I haven't been involved with the center since then and have no information on what the results are.
The original experiment design was to encompass 200 middle school math teachers in the Metropolitan Nashville Public Schools -- 100 in the control group and 100 in the treatment group. Teachers in the treatment group were eligible for bonuses of up to $15,000 for each of three consecutive school years. Each teacher received $750 every year for participating as long as they completed all the required surveys, interviews, etc. Teachers were recruited into the experiment in the fall of 2006, not long after the school year had begun.
Bonuses were based on student gain scores* (not quite the same as value-added, see technical note at end) on the Tennessee state test (TCAP). Unlike virtually every state, TN's assement is system is vertically scaled, meaning that scores can be compared across years on the same scale (a score of, say, 250 in 7th grade means the same thing as a score of 250 in 6th grade). This means that a student who goes from 240 to 260 from 6th to 7th grade gained 20 points. Meanwhile, researchers looked at the years preceding the experiment to determine the average growth of students at each level. Taking the previous example, let's assume that the average TN 6th grader scoring a 240 on the state test then scores 255 next year. This would mean that a student who scored 260 was 5 points above average. For that, a teacher would receive a score of +5, and each student the teacher taught would be scored similarly. The average score for a student with teacher x would be calculated. The purpose of calculating scores this way was to strike a balance between statistical rigor and transparency/ease of communication. The result is a calculation that's not quite as rigorous as a value-added score, but a lot easier for teachers to understand.
When the teacher's final score has been calculated, it's then compared to the historic average for middle school math teachers in Nashville. If a teacher scores in the 80th percentile, they earn a $5,000 bonus, the 85th percentile earns a $10,000 bonus, and the 95th percentile yields at $15,000 bonus. The targets for the bonuses stay the same the entire three years, so it's possible for every teacher in the treatment group to earn a bonus each year (in other words, they're not competing against each other). It's my understanding that for the first year the bonuses were distributed along with paychecks the following fall, but I don't know what the procedures were the following two years.
The experiment ended in May, 2009 and a large team of researchers have been poring over data from test scores, interviews, surveys, and other sources of information ever since. This means that there is going to be a lot of analysis released at some point in time -- and that it's going to take a while for even the most informed reader to sort through.
technical note: A "gain score" is simply the gain in a student's score from year to year (260 - 240 = a gain of 20 points), while a "value-added score" is an attempt to isolate a teacher's effect on a student's score and might control not only for a student's previous achievement level but also the other teachers he/she has or has had, the school he/she attends, demographic factors, class size, peer effects, and any number of other things. In other words, a gain score is just the raw growth a student exhibits while a value-added score is a more precise estimate of exactly how a specific teacher influenced that growth (though value-added could be computed for schools, states, etc. as well).
Wednesday, September 15, 2010
Tuesday, September 14, 2010
Today's Random Thoughts
-Jay Mathews echoes a point I've often made in private: most news stories about the cost of college attendance grossly overstate what the average student actually ends up paying. Though student loan debt is not a trivial problem, I think there are probably more people scared off by misperceptions of the costs of college than there are people who are bankrupt because of their attendance.
-Aaron Pallas shoots holes in the claims made in a recent op-ed about the miracles worked by a group of CA schools. In an op-ed I've seen mentioned numerous places, Caitlin Flanagan claims that the ICEF elementary schools closed the achievement gap. Pallas does some number crunching and finds that's not even remotely true (except for one of the five schools, and only in 2nd grade reading) -- indeed, their students' test scores are only slightly better than district averages for African-American students. I have to say I'm mildly surprised that her claims made it past the editor's desk. There are a lot of reasons to be skeptical of the charter school movement writ large, but there's really no arguing the fact that some charters have achieved outstanding test results. In other words, there's plenty of statistical evidence to support arguments for the proliferation of charter schools -- it seems odd that anybody would need to resort to misrepresenting the test scores of a few select charters.
-Stephen Sawchuck makes a reasonable point about the possibility that value-added scores can save the jobs of unfairly maligned good teachers as well as unfairly maligning good teachers. Both sides of the debate would do well to remember that there are many positive and negative aspects of value-added scores.
-Kevin Carey writes about a very interesting chart on the growth in college expenditures. Basically, the chart shows that "student-oriented" expenditures at the top 1% of most selective colleges have skyrocketed, have grown quite quickly at other schools among the top 10%, and haven't grown terribly fast at the rest. I suppose the takeaway points are that the growing concern about runaway spending in higher education really only apply to a select few colleges (where money isn't really an issue in a lot of ways), and that our colleges are growing further and further apart in terms of resources.
-Aaron Pallas shoots holes in the claims made in a recent op-ed about the miracles worked by a group of CA schools. In an op-ed I've seen mentioned numerous places, Caitlin Flanagan claims that the ICEF elementary schools closed the achievement gap. Pallas does some number crunching and finds that's not even remotely true (except for one of the five schools, and only in 2nd grade reading) -- indeed, their students' test scores are only slightly better than district averages for African-American students. I have to say I'm mildly surprised that her claims made it past the editor's desk. There are a lot of reasons to be skeptical of the charter school movement writ large, but there's really no arguing the fact that some charters have achieved outstanding test results. In other words, there's plenty of statistical evidence to support arguments for the proliferation of charter schools -- it seems odd that anybody would need to resort to misrepresenting the test scores of a few select charters.
-Stephen Sawchuck makes a reasonable point about the possibility that value-added scores can save the jobs of unfairly maligned good teachers as well as unfairly maligning good teachers. Both sides of the debate would do well to remember that there are many positive and negative aspects of value-added scores.
-Kevin Carey writes about a very interesting chart on the growth in college expenditures. Basically, the chart shows that "student-oriented" expenditures at the top 1% of most selective colleges have skyrocketed, have grown quite quickly at other schools among the top 10%, and haven't grown terribly fast at the rest. I suppose the takeaway points are that the growing concern about runaway spending in higher education really only apply to a select few colleges (where money isn't really an issue in a lot of ways), and that our colleges are growing further and further apart in terms of resources.
Oh Meyer Goodness! Redux
Last week, Peter Meyer wrote a piece that I called "baffling". Well, at least he's consistent, because today he did the same thing again. Over at Flypaper, he almost seems to be calling for a return to segregated schools.
Here's some context: This NY Times article today referenced this report about suspensions in urban middle schools, the major finding of which was that black students (particularly males) were much more likely to be suspended than white students -- and that the gap had widened over the past few decades.
Meyer makes a leap at the end of his post, implying that the desegregation of schools is responsible for this disparity and referencing an MLK quote from decades ago to back up his support of segregated schools.
In the post where he first references the MLK quote, he does make a reasonable point that desegregation shouldn't be our sole policy aim (though, at the same time, I don't think very many people think it should be).
But there are two major problems with his latest post:
1.) As far as I can tell, the report says zero about any differences in suspension rates between more and less racially diverse schools. In other words, there's really no readily apparent evidence for Meyer's claim.
2.) 1954 has come and gone. We, as a society, have decided that separate but equal is inherently unequal. Nostalgia for the past is one thing, but do we really have to go back and repeat all our mistakes? Black males are also more likely to be arrested, does that mean we should create segregated neighborhoods to accompany our segregated schools?
I hardly think Mr. Meyer's next post is going to argue for separate drinking fountains and bathrooms, but he'd do well to remember that it's a slippery slope. If you're going to advocate for segregated schools, please do so more thoughtfully and use actual evidence.
update: I originally misspelled Mr. Meyer's name as "Mayer" in this post. My apologies; no matter how much I disagree with him on this issue he still deserves to have his name spelled correctly.
Here's some context: This NY Times article today referenced this report about suspensions in urban middle schools, the major finding of which was that black students (particularly males) were much more likely to be suspended than white students -- and that the gap had widened over the past few decades.
Meyer makes a leap at the end of his post, implying that the desegregation of schools is responsible for this disparity and referencing an MLK quote from decades ago to back up his support of segregated schools.
In the post where he first references the MLK quote, he does make a reasonable point that desegregation shouldn't be our sole policy aim (though, at the same time, I don't think very many people think it should be).
But there are two major problems with his latest post:
1.) As far as I can tell, the report says zero about any differences in suspension rates between more and less racially diverse schools. In other words, there's really no readily apparent evidence for Meyer's claim.
2.) 1954 has come and gone. We, as a society, have decided that separate but equal is inherently unequal. Nostalgia for the past is one thing, but do we really have to go back and repeat all our mistakes? Black males are also more likely to be arrested, does that mean we should create segregated neighborhoods to accompany our segregated schools?
I hardly think Mr. Meyer's next post is going to argue for separate drinking fountains and bathrooms, but he'd do well to remember that it's a slippery slope. If you're going to advocate for segregated schools, please do so more thoughtfully and use actual evidence.
update: I originally misspelled Mr. Meyer's name as "Mayer" in this post. My apologies; no matter how much I disagree with him on this issue he still deserves to have his name spelled correctly.
Monday, September 13, 2010
Responsibility With No Responsibility
Researchers and practitioners all seem to agree that teachers are the most important factor within a school. And many have taken that another step and asserted that teacher quality is almost the only thing that matters. I've pushed back by pointing out that lots of things affect a teacher's performance other than a person's talent or moral character.
But here's what blows my mind. Across the country, people seem to argue that teachers need to put up or shut up -- and that if a school fails it must be a result of poor teaching. And, yet, all across the country, teachers are told to do things the way the principal, superintendent, board of ed, or whomever wants them done. The last decade has seen the proliferation of scripted curricula ("teacher-proofing" they call it) and increasing micromanagement in urban schools (ask an NYC teacher if they have their "word wall" up or if their bulletin board properly displays student work). If teachers bear all the responsibility for student success, why are they given so little responsibility for what and how students learn?
Think about it: if a teacher's not given any responsibility for how and what their students learn, then how can we hold them responsible for how and what students learn? It's accountability without autonomy, responsibility with no actual responsibility.
When NYC started their principal accountability program, it was in the context of an "autonomy zone". Principals signed contracts that basically said they would be fired if student achievement didn't improve in 5 years. And, in return, principals had far more say over how their school was run and how professional development funds were spent.
Teachers, on other hand, aren't really offered the same deal. They're essentially being told that they will be held responsible for what happens in their classroom (which isn't entirely unfair) -- but also that they will run their classroom a certain way . . . or else.
If we don't trust teachers to do what's in the best interest of students, then maybe they're not the ones we should be pointing fingers at when students don't learn. If a teacher follows a scripted curriculum and students don't learn, maybe we should point our fingers at the curriculum writers. If a teacher follows the checklist the district passes down and students don't learn, maybe we should point our fingers at the district personnel. If a teacher does everything their principal demands of them and students don't learn, maybe we should point our fingers at the principal.
If we think a teacher's primary responsibility should be to stick to the curriculum, decorate their rooms the way the superintendent says to, and follow the instructions of their principal, then, by all means, we should evaluate them on these things and hold them accountable when they fail to do them. But if, instead, we think a teacher's primary responsibility is to ensure that students learn, maybe we should think about letting them determine what and how students learn before holding them accountable for this.
But here's what blows my mind. Across the country, people seem to argue that teachers need to put up or shut up -- and that if a school fails it must be a result of poor teaching. And, yet, all across the country, teachers are told to do things the way the principal, superintendent, board of ed, or whomever wants them done. The last decade has seen the proliferation of scripted curricula ("teacher-proofing" they call it) and increasing micromanagement in urban schools (ask an NYC teacher if they have their "word wall" up or if their bulletin board properly displays student work). If teachers bear all the responsibility for student success, why are they given so little responsibility for what and how students learn?
Think about it: if a teacher's not given any responsibility for how and what their students learn, then how can we hold them responsible for how and what students learn? It's accountability without autonomy, responsibility with no actual responsibility.
When NYC started their principal accountability program, it was in the context of an "autonomy zone". Principals signed contracts that basically said they would be fired if student achievement didn't improve in 5 years. And, in return, principals had far more say over how their school was run and how professional development funds were spent.
Teachers, on other hand, aren't really offered the same deal. They're essentially being told that they will be held responsible for what happens in their classroom (which isn't entirely unfair) -- but also that they will run their classroom a certain way . . . or else.
If we don't trust teachers to do what's in the best interest of students, then maybe they're not the ones we should be pointing fingers at when students don't learn. If a teacher follows a scripted curriculum and students don't learn, maybe we should point our fingers at the curriculum writers. If a teacher follows the checklist the district passes down and students don't learn, maybe we should point our fingers at the district personnel. If a teacher does everything their principal demands of them and students don't learn, maybe we should point our fingers at the principal.
If we think a teacher's primary responsibility should be to stick to the curriculum, decorate their rooms the way the superintendent says to, and follow the instructions of their principal, then, by all means, we should evaluate them on these things and hold them accountable when they fail to do them. But if, instead, we think a teacher's primary responsibility is to ensure that students learn, maybe we should think about letting them determine what and how students learn before holding them accountable for this.
Friday, September 10, 2010
Today's Random Thoughts
-Tennessee has figured out a solution (hat tip: Stephen Lentz) to the fact that only about 1/3 of teachers teach tested subjects but that all teachers are supposed to have 35% of their evaluation based on value-added scores . . . all "non-TVAAS" (the state test) teachers will simply have their school's average score used for their evaluation. Problem solved!
-Aaron Pallas continues his critique of the LA value-added kerfuffle, arguing that the LA Times did not do enough to inform its readers about the statistical uncertainty in value-added measurements. He argues that they should've used confidence intervals (something that popped into my head the other day) to more accurately describe the estimate of a teacher's effect on student test scores (they send you a confidence interval with your SAT scores, so why not with a value-added score?) in addition to better describing year-to-year and subect-to-subject variability. This is a follow-up to his incisive critique of the Times' failure to follow normal standards of journalism when verifying the student data.
Jay Mathews has Killian Betlach's take on what it's like to be told to restructure a school.
Roger Garfield, a teacher in DC, provides an insider's view of some of the problems the schools face. The first couple paragraphs brought back a lot of memories for me.
Newark's answer to the Harlem Children's Zone is the Global Village, a group of five schools that have received federal turnaround dollars.
Robert Samuelson says the real key to reform is student motivation. It's a pretty short op-ed, and there's a lot more to it, but I think he raises a valid point. If student motivation doesn't change, why would we expect student learning to change? But I don't think it's quite as strong of a repudiation of other policies as he argues, since better principals, better curricula, better teachers, smaller classes, and so on could conceivably alter student motivation (but if they don't, they probably won't work).
-Aaron Pallas continues his critique of the LA value-added kerfuffle, arguing that the LA Times did not do enough to inform its readers about the statistical uncertainty in value-added measurements. He argues that they should've used confidence intervals (something that popped into my head the other day) to more accurately describe the estimate of a teacher's effect on student test scores (they send you a confidence interval with your SAT scores, so why not with a value-added score?) in addition to better describing year-to-year and subect-to-subject variability. This is a follow-up to his incisive critique of the Times' failure to follow normal standards of journalism when verifying the student data.
Jay Mathews has Killian Betlach's take on what it's like to be told to restructure a school.
Roger Garfield, a teacher in DC, provides an insider's view of some of the problems the schools face. The first couple paragraphs brought back a lot of memories for me.
Newark's answer to the Harlem Children's Zone is the Global Village, a group of five schools that have received federal turnaround dollars.
Robert Samuelson says the real key to reform is student motivation. It's a pretty short op-ed, and there's a lot more to it, but I think he raises a valid point. If student motivation doesn't change, why would we expect student learning to change? But I don't think it's quite as strong of a repudiation of other policies as he argues, since better principals, better curricula, better teachers, smaller classes, and so on could conceivably alter student motivation (but if they don't, they probably won't work).
Wednesday, September 8, 2010
First Day of School: Where Are You?
Today is the first day of school in NYC, which always brings back a flood of memories for me. But today it's making me think about something a little different.
I left the classroom. I left the classroom for a number of reasons, but near the top is that my experiences there were horrible. In the end, even if I wanted to, I just couldn't stand the thought of teaching for the next 30 or so years. In the end, I was too weak to make it.
And yet, I now find myself up in the ivory tower consorting with others who regularly cast stones at the lowly teachers (who simply need to put up or shut up if we ever want to fix our disaster of an educational system). And you know what? Most of them couldn't hack it in the classroom either.
There's a lot of lip service from us non-teachers about how important teachers are, but I'd say there's even more disrespect. Whether anybody wants to admit it or not, the word "teacher" is said with at least some amount of disdain in many policy circles.
And that troubles me. Not because there aren't bad teachers out there, but because most people not only couldn't do much better, they don't even have the courage to try. No amount of money could convince me to go back and teach in the Bronx permanently. I shudder even thinking about it.
I still remember my last day of teaching and the conversation I had with another teacher. He was a former business executive twice my age, but had been a teaching fellow just like me. "Worst two of years of my life," he said. "Mine too," I said (which he scoffed at because of our age differential).
Today's the first day of school (in NYC, here in TN some schools started a month ago). Where are you? Are you in the classroom? If not, why? And how should that make you feel about those who are?
Teachers catch a lot of flak for resisting change (among other things). But to everyone else out there who couldn't hack it in the classroom (or doesn't want to try), I ask you this: shouldn't those who actually show up in a classroom every day be at least a little wary of what we say?
If you were a teacher, would you want somebody who can't or won't teach telling you what to do? I'm not arguing that those of us outside the classroom never have anything valuable to suggest, just that we're usually arrogant in the way we suggest it. If we can't hack it in the classroom, we can at least mind our p's and q's when talking to, or about, those who can.
To all you teachers out there who are still doing what I couldn't, I tip my hat to you. Today I acknowledge that, in many ways, you are better than I. Today I acknowledge that you are far more important to our educational system than anybody wearing a fancy suit or carrying fancy credentials. And, as such, today I acknowledge that my role as a researcher should be to help you. I'll do my best. I trust you'll do the same.
I left the classroom. I left the classroom for a number of reasons, but near the top is that my experiences there were horrible. In the end, even if I wanted to, I just couldn't stand the thought of teaching for the next 30 or so years. In the end, I was too weak to make it.
And yet, I now find myself up in the ivory tower consorting with others who regularly cast stones at the lowly teachers (who simply need to put up or shut up if we ever want to fix our disaster of an educational system). And you know what? Most of them couldn't hack it in the classroom either.
There's a lot of lip service from us non-teachers about how important teachers are, but I'd say there's even more disrespect. Whether anybody wants to admit it or not, the word "teacher" is said with at least some amount of disdain in many policy circles.
And that troubles me. Not because there aren't bad teachers out there, but because most people not only couldn't do much better, they don't even have the courage to try. No amount of money could convince me to go back and teach in the Bronx permanently. I shudder even thinking about it.
I still remember my last day of teaching and the conversation I had with another teacher. He was a former business executive twice my age, but had been a teaching fellow just like me. "Worst two of years of my life," he said. "Mine too," I said (which he scoffed at because of our age differential).
Today's the first day of school (in NYC, here in TN some schools started a month ago). Where are you? Are you in the classroom? If not, why? And how should that make you feel about those who are?
Teachers catch a lot of flak for resisting change (among other things). But to everyone else out there who couldn't hack it in the classroom (or doesn't want to try), I ask you this: shouldn't those who actually show up in a classroom every day be at least a little wary of what we say?
If you were a teacher, would you want somebody who can't or won't teach telling you what to do? I'm not arguing that those of us outside the classroom never have anything valuable to suggest, just that we're usually arrogant in the way we suggest it. If we can't hack it in the classroom, we can at least mind our p's and q's when talking to, or about, those who can.
To all you teachers out there who are still doing what I couldn't, I tip my hat to you. Today I acknowledge that, in many ways, you are better than I. Today I acknowledge that you are far more important to our educational system than anybody wearing a fancy suit or carrying fancy credentials. And, as such, today I acknowledge that my role as a researcher should be to help you. I'll do my best. I trust you'll do the same.
Be Careful When Calling For "Great Teachers"
Davis Guggenheim, director of the forthcoming documentary "Waiting for Superman," writes an op-ed for the Huffington Post (hat tip: Gotham Schools) that fits pretty well with conventional wisdom on schools nowadays. In it, he repeatedly asserts that "We can't have great schools without great teachers"
This is true. We can't. But this is also a dangerously overly simplified narrative.
I say this for three reasons:
1.) Teachers are the single most important within-school factor, there's really no dispute over this. But estimates of teacher impact on student test scores find that teacher quality only explains about 20% of the variation in these test scores. So let's be careful not to insinuate that teachers are the only thing that matter or that teachers should be expected to fix everything.
2.) Lines like the one Guggenheim uses are great soundbites, but too many people assume that teachers are simply "good" or "bad" when they read or hear such things. In reality, teachers don't come out of the womb either good or bad; they perform poorly or superbly for any number of reasons. These include, but are not limited to: experience, class size, school quality, curricula, the actions of other teachers, the actions of administrators, and the particular students they've been assigned this year. All of these factors are mostly out of a teacher's control in any given year. So simply searching for great teachers isn't really enough: we have to search for them, train them, place them in a context where they can succeed, and then convince them to keep doing what they do (and doing it well).
3.) Guggenheim suggests that the solution to all our problems in education is a simple one: we need great teachers. He further suggests that curriculum, class size, etc. don't really matter. Both of these are false. Finding, training, and retaining great teachers is anything but simple, and teacher quality is but one of many, many things that matter in education.
I can't fault Guggenheim for his obsession with teacher quality. It's probably the one factor that's both important enough and manipulable enough for policy changes to have an immediate and significant impact. But if there's one thing that I've learned about our educational system it's that changing one factor should never be expected to solve everything.
I need to add to my list of things people should remember about education policy that education is an enormously complicated process involving innumerate moving parts and, as such, we cannot -- and should not -- expect changing one factor to solve all of the system's problems. There is no magic bullet, no simple fix; changing the course of one child's education is a lifelong process and changing the course of millions of kids' educations is infinitely more difficult.
Teacher quality seems like a good place to start, but let's recognize both that changing it won't be easy and that it's not a good place to stop
This is true. We can't. But this is also a dangerously overly simplified narrative.
I say this for three reasons:
1.) Teachers are the single most important within-school factor, there's really no dispute over this. But estimates of teacher impact on student test scores find that teacher quality only explains about 20% of the variation in these test scores. So let's be careful not to insinuate that teachers are the only thing that matter or that teachers should be expected to fix everything.
2.) Lines like the one Guggenheim uses are great soundbites, but too many people assume that teachers are simply "good" or "bad" when they read or hear such things. In reality, teachers don't come out of the womb either good or bad; they perform poorly or superbly for any number of reasons. These include, but are not limited to: experience, class size, school quality, curricula, the actions of other teachers, the actions of administrators, and the particular students they've been assigned this year. All of these factors are mostly out of a teacher's control in any given year. So simply searching for great teachers isn't really enough: we have to search for them, train them, place them in a context where they can succeed, and then convince them to keep doing what they do (and doing it well).
3.) Guggenheim suggests that the solution to all our problems in education is a simple one: we need great teachers. He further suggests that curriculum, class size, etc. don't really matter. Both of these are false. Finding, training, and retaining great teachers is anything but simple, and teacher quality is but one of many, many things that matter in education.
I can't fault Guggenheim for his obsession with teacher quality. It's probably the one factor that's both important enough and manipulable enough for policy changes to have an immediate and significant impact. But if there's one thing that I've learned about our educational system it's that changing one factor should never be expected to solve everything.
I need to add to my list of things people should remember about education policy that education is an enormously complicated process involving innumerate moving parts and, as such, we cannot -- and should not -- expect changing one factor to solve all of the system's problems. There is no magic bullet, no simple fix; changing the course of one child's education is a lifelong process and changing the course of millions of kids' educations is infinitely more difficult.
Teacher quality seems like a good place to start, but let's recognize both that changing it won't be easy and that it's not a good place to stop
Tuesday, September 7, 2010
Oh Meyer Goodness! What was he Thinking?
Peter Meyer's baffling post last night continues to astound me. The post begins by criticizing Pedro Noguera for arguing that there are two sides to the issue of addressing the achievement gap (at which point I was ready to agree with him), and finishes by arguing that Noguera's side is really wrong and Meyer's side is right (at which point I became baffled).
The post continues to re-hash the idea that we have only two options in combating the achievement gap: fix schools and ignore everything else, or fix everything else and ignore schools. I've written in the past that this is a "false distinction" and agreed with Geoffrey Canada's assessment that this is "a terrible, phony debate". We we need to choose option 'c' -- "all of the above".
I've already written a long description of this debate and what research actually says about the influence of non-school factors vs. in-school factors, in addition to a long explanation of how this played out on the ground at my school -- neither of which will be fully re-hashed here except to repeat this: If there's anything upon which education researchers agree it's that student achievement is influenced more by non-school factors than in-school factors -- and the evidence is overwhelming.
Meyer's piece goes downhill with the following paragraph when he asserts that the argument that "poverty causes low academic achievement" is "wrong." Why is it wrong? I'm not quite sure. This is what he writes:
"What some of us have long known is that public schools were started mainly to educate the poor. And the only reason poverty is a predictor of bad academic achievement results is that educators like Noguera have made it so. Instead of schools as tools of liberation, we have made them into great houses of mirrors, reflecting back on students the environment they come from."
I'm genuinely unsure of exactly what this means or how, precisely, Pedro Noguera ensures that students from poor families perform worse in school. But from what I can gather I assume he's arguing that high-poverty schools continue to perform poorly mostly because we expect them to . . . or something like that?
Where I sort of agree with Meyer is where he takes exception to Noguera's statement that "schools alone – not even the very best schools – cannot erase the effects of poverty". Meyer is right to assert that there's growing evidence that a few select schools have achieved outstanding results virtually by themselves (which, let it be noted, is very different from arguing that we are able to replicate these isolated successes or that we should expect all schools (or at least all high-poverty schools) to work miracles). But I only sort of agree with Meyer on this point because while I might have preferred that Noguera use slightly different wording, he's likely still technically correct -- and I say that because he specifically differentiates poverty from test scores in the next sentence. The "effects of poverty" go far beyond just lower test scores, but we conclude that the KIPPs of the world have worked miracles almost solely because they have high test scores. I, personally, haven't seen (not saying that none exists) research linking attendance at these miracle schools to broader outcomes (e.g. health, college graduation, occupational prestige, incarceration, etc.), and I'd have to guess that while one school can do a lot of good, it can't completely transform every single aspect of every single student's life. Lastly, I find it odd that Meyer first argues that poverty doesn't cause low achievement and then that schools can, in fact, erase the effects of poverty.
But where Meyer completely loses me is with his assertion that "until we recognize that education is education and that poverty is poverty, we’re not going to fix our schools or enrich our population." Here he couldn't be more wrong. The truth is that poverty and education are deeply intertwined -- in both directions (i.e. living in poverty negatively affects students' educational outcomes, and students' educational outcomes affect their life outcomes and the odds they'll live in poverty). And this is true regardless of whether or not Meyer's proposed reforms will work or not. While we have plenty of evidence that social conditions and environmental factors experienced disproportionately by those living in poverty influence academic performance, we have precious little evidence that we can consistently change these conditions and factors in ways that will subsequently yield large gains in performance.
In other words, it might be the case that non-school factors are more powerful predictors of student performance but that in-school factors are much easier to change. In which case, Meyer's call for school reform before community perform could, in the end, be the right way to go. But even still, I find his utter disavowal of the relationship between poverty and academic performance to be more than a little disturbing (even despite his more conciliatory tone today). And it leaves me with two sets questions for Mr. Meyer:
1.) What evidence, exactly, do we have that poverty does not influence students' academic performance? If poverty doesn't cause worse performance, why do students from low-income families perform so much worse? Is it solely because schools in low-income neighborhoods are worse? If so, why do high-income students in low-performing schools outperform low-income students in high-performing schools? And, even if it is only the school influencing achievement, is it not possible that neighborhood poverty is influencing school quality?
2.) Why must a disavowal of the relationship between poverty and academic performance be a prerequisite for the support of school reform? Can it not be the case that poverty causes lower achievement but that schools can overcome some of these effects? To argue that poverty is important and that schools are important are not mutually exclusive.
update: I originally misspelled Mr. Meyer's name as "Mayer" in this post. My apologies; no matter how much I disagree with him on this issue he still deserves to have his name spelled correctly.
The post continues to re-hash the idea that we have only two options in combating the achievement gap: fix schools and ignore everything else, or fix everything else and ignore schools. I've written in the past that this is a "false distinction" and agreed with Geoffrey Canada's assessment that this is "a terrible, phony debate". We we need to choose option 'c' -- "all of the above".
I've already written a long description of this debate and what research actually says about the influence of non-school factors vs. in-school factors, in addition to a long explanation of how this played out on the ground at my school -- neither of which will be fully re-hashed here except to repeat this: If there's anything upon which education researchers agree it's that student achievement is influenced more by non-school factors than in-school factors -- and the evidence is overwhelming.
Meyer's piece goes downhill with the following paragraph when he asserts that the argument that "poverty causes low academic achievement" is "wrong." Why is it wrong? I'm not quite sure. This is what he writes:
"What some of us have long known is that public schools were started mainly to educate the poor. And the only reason poverty is a predictor of bad academic achievement results is that educators like Noguera have made it so. Instead of schools as tools of liberation, we have made them into great houses of mirrors, reflecting back on students the environment they come from."
I'm genuinely unsure of exactly what this means or how, precisely, Pedro Noguera ensures that students from poor families perform worse in school. But from what I can gather I assume he's arguing that high-poverty schools continue to perform poorly mostly because we expect them to . . . or something like that?
Where I sort of agree with Meyer is where he takes exception to Noguera's statement that "schools alone – not even the very best schools – cannot erase the effects of poverty". Meyer is right to assert that there's growing evidence that a few select schools have achieved outstanding results virtually by themselves (which, let it be noted, is very different from arguing that we are able to replicate these isolated successes or that we should expect all schools (or at least all high-poverty schools) to work miracles). But I only sort of agree with Meyer on this point because while I might have preferred that Noguera use slightly different wording, he's likely still technically correct -- and I say that because he specifically differentiates poverty from test scores in the next sentence. The "effects of poverty" go far beyond just lower test scores, but we conclude that the KIPPs of the world have worked miracles almost solely because they have high test scores. I, personally, haven't seen (not saying that none exists) research linking attendance at these miracle schools to broader outcomes (e.g. health, college graduation, occupational prestige, incarceration, etc.), and I'd have to guess that while one school can do a lot of good, it can't completely transform every single aspect of every single student's life. Lastly, I find it odd that Meyer first argues that poverty doesn't cause low achievement and then that schools can, in fact, erase the effects of poverty.
But where Meyer completely loses me is with his assertion that "until we recognize that education is education and that poverty is poverty, we’re not going to fix our schools or enrich our population." Here he couldn't be more wrong. The truth is that poverty and education are deeply intertwined -- in both directions (i.e. living in poverty negatively affects students' educational outcomes, and students' educational outcomes affect their life outcomes and the odds they'll live in poverty). And this is true regardless of whether or not Meyer's proposed reforms will work or not. While we have plenty of evidence that social conditions and environmental factors experienced disproportionately by those living in poverty influence academic performance, we have precious little evidence that we can consistently change these conditions and factors in ways that will subsequently yield large gains in performance.
In other words, it might be the case that non-school factors are more powerful predictors of student performance but that in-school factors are much easier to change. In which case, Meyer's call for school reform before community perform could, in the end, be the right way to go. But even still, I find his utter disavowal of the relationship between poverty and academic performance to be more than a little disturbing (even despite his more conciliatory tone today). And it leaves me with two sets questions for Mr. Meyer:
1.) What evidence, exactly, do we have that poverty does not influence students' academic performance? If poverty doesn't cause worse performance, why do students from low-income families perform so much worse? Is it solely because schools in low-income neighborhoods are worse? If so, why do high-income students in low-performing schools outperform low-income students in high-performing schools? And, even if it is only the school influencing achievement, is it not possible that neighborhood poverty is influencing school quality?
2.) Why must a disavowal of the relationship between poverty and academic performance be a prerequisite for the support of school reform? Can it not be the case that poverty causes lower achievement but that schools can overcome some of these effects? To argue that poverty is important and that schools are important are not mutually exclusive.
update: I originally misspelled Mr. Meyer's name as "Mayer" in this post. My apologies; no matter how much I disagree with him on this issue he still deserves to have his name spelled correctly.
Today's Random Thoughts
*From an interesting piece on study habits come a few simple lessons on what helps people actually learn things:
1.) studying the same thing in different places (so that you learn thing in different contexts)
2.) studying more than one thing at once (so that you make connections between the different subjects)
3.) studying the same thing at different times (so that you forget and re-learn things -- "forgetting is the friend of learning")
The piece would seem to support the idea that we should "spiral" when we teach students instead of "scaffold" -- things should be briefly introduced and re-introduced in different contexts instead of teaching one subject one day and a different one the next. I'd also add that this seems like a pretty good argument for more interdisciplinary teaching/learning -- something that's difficult to do at the K-12 level and for which there's little incentive for college professors to try (indeed, there's probably a discentive since more specialization often equals a greater chance of earning tenure).
*Aaron Pallas proposes a good rule on his new blog: "the weight accorded to any one element of a teacher evaluation system should be proportional to the uncertainty about the inference which is drawn from that element". In other words, if we're not sure about how meaningful something is then we shouldn't place the whole weight on that item (that goes for both principal evaluations and value-added scores)
* Are middle schools hurting students? My experience teaching in a middle school has convinced me that those findings are at least plausible (as do my experiences when I was 12-14). This isn't to suggest that K-8 schools are a silver bullet, just that, in theory, they have more potential than middle schools to advance the learning of 6th-8th grade students (especially if those students can take a leadership role in the school by tutoring younger kids, serving as crossing guards, etc.).
1.) studying the same thing in different places (so that you learn thing in different contexts)
2.) studying more than one thing at once (so that you make connections between the different subjects)
3.) studying the same thing at different times (so that you forget and re-learn things -- "forgetting is the friend of learning")
The piece would seem to support the idea that we should "spiral" when we teach students instead of "scaffold" -- things should be briefly introduced and re-introduced in different contexts instead of teaching one subject one day and a different one the next. I'd also add that this seems like a pretty good argument for more interdisciplinary teaching/learning -- something that's difficult to do at the K-12 level and for which there's little incentive for college professors to try (indeed, there's probably a discentive since more specialization often equals a greater chance of earning tenure).
*Aaron Pallas proposes a good rule on his new blog: "the weight accorded to any one element of a teacher evaluation system should be proportional to the uncertainty about the inference which is drawn from that element". In other words, if we're not sure about how meaningful something is then we shouldn't place the whole weight on that item (that goes for both principal evaluations and value-added scores)
* Are middle schools hurting students? My experience teaching in a middle school has convinced me that those findings are at least plausible (as do my experiences when I was 12-14). This isn't to suggest that K-8 schools are a silver bullet, just that, in theory, they have more potential than middle schools to advance the learning of 6th-8th grade students (especially if those students can take a leadership role in the school by tutoring younger kids, serving as crossing guards, etc.).
Thursday, September 2, 2010
National Test, Here We Come
I just noticed this item from the wire feed on the NY Times website. Two groups of states have been given $330 million to develop new, better, tests to be ready by 2014-15. I've argued many times (see here, here, here, here, and here) before that the only way we can continue to use test-based accountability in the future is if we adopt national standards and a national test. A year ago I thought we were likely at least a decade away from the former, but both seem to be approaching far more rapidly than I could've imagined.
If you're pro-test-based accountability, this should make you happy -- these developments will make such systems more accurate and more meaningful . . . plus, politically, this is the only way such systems will survive into the next decade.
If you're anti-test-based accountability, you might have mixed feelings -- these developments will make the current testing regime more accurate and more meaningful, but it also likely means that it's not going away any time soon.
If you're pro-test-based accountability, this should make you happy -- these developments will make such systems more accurate and more meaningful . . . plus, politically, this is the only way such systems will survive into the next decade.
If you're anti-test-based accountability, you might have mixed feelings -- these developments will make the current testing regime more accurate and more meaningful, but it also likely means that it's not going away any time soon.
Subscribe to:
Posts (Atom)