by Kerri Tobin, PhD
Teacher opposition to value-added modeling gets portrayed in the media
as teachers refusing to take any kind of responsibility for student
learning or academic growth. But most teachers do not balk at the
idea that they should be accountable for advancing their students’
knowledge. What they oppose is value-added modeling, or VAM, the
highly-imprecise tool that is being used to measure teachers' impact
on students' learning.. When the New York Times won the right to publishteacher VAM scores this spring,
only the tiny print noted the inaccuracy of the data, including some
standard errors larger than the purported effect sizes. Researchers
familiar with value-added modeling have repeatedly voiced their
concern about its use in high-stakes decisions like teacher
evaluations (for example: here, here, and here).
Often, opponents of current uses of standardized testing fall back on
attempts to draw parallels between teaching and medicine: “It’s like
grading doctors on how many of their patients die!” But these
analogies are imprecise; they do not do the problem justice. John
Ewing’s fertilizer analogy is fascinating but perhaps too
long for the common person. What we need is an “elevator speech” –
one that can be delivered in the time it takes to get from the ground
to the 3rd floor – to explain why teachers oppose VAM. A better
analogy is this: using VAM to evaluate teachers is akin to evaluating
chemotherapists based on how much their patients’ tumors shrink every year.
The expected rate of tumor shrinkage is calculated based on patients’
race and socioeconomic status. So if a patient’s tumor shrinks more,
over that year, than expected, the doctor gets a positive score. If
it shrinks less than it “should,” the doctor gets a negative score.
The average of these patients’ scores becomes the doctor’s overall VAM
score. Sound reasonable? Maybe, until you consider that a) different
kinds of tumors respond differently to chemotherapy; b) the doctor has
no control over what patients do outside the office (for example, lung
cancer sufferers who continue to smoke); c) his patients saw a
different chemotherapist last year and will see another one next year;
and, perhaps most importantly, d) the doctor is not allowed to treat
any co-morbid conditions (for example, a cancer patient with diabetes
gets no treatment to manage his blood sugar) – even if he wanted to,
and even in cases where the patient or his family might prefer that
other conditions be treated instead of the cancer (e.g., when parents
value social or self-confidence issues more than test scores), there
simply aren’t enough hours in his day. Factors like the overall
health of the patient, his lifestyle, eating habits, substance use,
weight, and blood pressure might impact the effectiveness of the
chemo. But the doctor cannot control these, in much the same way
teachers cannot control where students live, if they have enough to
eat and get regular medical care, whether anyone reads to them at
home, how much TV they watch, or what time they go to bed at night.
And in the same way our VAM-assessed doctor would be powerless to
decide that a patient dying of AIDS needs antiretroviral therapy
before he can tolerate chemo for a concurrent cancer, teachers have
neither the time, resources, nor training to solve the problems in
their students’ lives – emotional problems, health challenges, family
issues, etc. – that impact their academic growth. This is how VAM
works, and why teachers oppose it.
Kerri Tobin is Assistant Professor of Education at Marywood University in Scranton, PA, where she researches the educational needs of students living in poverty and prepares teachers and school leaders to meet those needs.
I'm coming increasingly to believe that one of the over-arching problems facing education today is the speed with which "interesting ideas" get turned into "solutions" by policy pundits.
Value-added is certainly a step forward from raw test scores as a way to evaluate schools and teachers. But it seems like only days elapsed between when VAM hit the literature and when it was adopted by reformers as the next silver bullet.
Teachers are certainly not omnipotent, any more than doctors are. The question is, do they have any power?
If there were two doctors, working in the same practice and each seeing a hundred patients a year, and one doctor had an average shrinkage rate of 75% and the other doctor had an average shrinkage rate of 25%, I would have no problem in paying the first doctor more, and saying that there is a very good chance she is a better cancer doctor.
Each of the doctors will be powerless to do a lot of things that would improve their patients' health, but those things average out.
It is an empirical question just how much "better" one teacher is than another. There's a lot of randomness in life and teachers' value added scores are going to bounce around year to year. Only by doing the experiment and looking seriously at the results will we see if some teachers consistently outperform others--and whether the difference is worth extra money or perks.
That was part of my point -- that there needs to be time to do the experiments. But it seems like 3 or 4 years ago VAM was the new new thing, and now we're debating what "percentage" of a teachers evaluation they should be. If there's a body of literature that shows what the "noise" from year to year is, or whether there are interesting unintended consequences, I haven't seen it.
Determine which cancer organization you wish to donate money to. There are a variety of cancer organizations and medical institutions that will accept donations towards cancer research, including Hospital, the National Breast Cancer Foundation and the Cancer Research Institute. Thanks for sharing.
Post a Comment