Wednesday, May 12, 2010

My Revolutionary New Grading System

I taught a section of an undergraduate public policy course this spring (part of the reason the blog updates have been few and far between) and decided to try out a bit of a different grading system for some of the assignments.  I almost blogged about this at the beginning of the semester, but I figured I'd wait and see how it played out first.

First, some background: when I TA'd the class in the past, we had a number of short papers that the students didn't put all that much effort into.  At a place like Vanderbilt, which has become super-competitive in recent years, I figured we'd be teaching some of the best and brightest young minds in the country.  But I can't honestly say that the writing ability of the students terribly impressed me.  So I decided that it would be better for the students if I spent more time emphasizing the quality of the writing than the quantity of the writing.

So this semester instead of giving weekly reflection papers, I assigned three 1-2 page policy analysis papers (in addition to a final paper, some quizzes, and a debate), but with a twist: the assignment wasn't done until the student turned in an 'A' paper, and the grade was based on how many attempts it took them to accomplish this.  At the same time, I tried to give out very specific instructions, sometimes a model paper, and offered copious amounts of help (including meeting with students, reading drafts, and giving students extensive feedback about their papers (which they actually read, at least if they didn't get an 'A')).  I'm not sure how much a student gains from simply slapping together a 'B' paper at the last minute, turning it in, and never thinking about it again (nor am I sure that's a good use of the professor's time either).  So not only would all students be almost forced to do their best (or at least high-quality) work, from which they should learn more, but the incentives would be better aligned with potential benefits (a bright student who is content with a B has it easy under the typical system, but under this system earning a B is at least as much work as earning an A).

My initial intention was to award students an 'A' if it took them one try to write an 'A' paper, a 'B' if it took them two tries, a 'C' if it took them three tries, a 'D' if it took them four tries, and pray nobody took more than four tries.  But I decided that was a little too rough around the edges -- it wouldn't be fair to award a student who turned in a 'D' paper and an 'A' paper the same grade as somebody who turned in an 'A-' and 'A'.  At the same time, I didn't want it to be possible for somebody to earn a higher grade after submitting two papers than somebody else did while submitting only one (which eliminates the simple averaging of all grades that some of the students wanted).  So after some tinkering around, I came up with the following system: I'd average the grade they received for the number of drafts it took them (85 for 2 tries, 75 for 3 tries, etc.) with the average of all the papers they'd turned in.  I calculated some mock grades and handed the following chart out to the students:

1st 2nd 3rd 4th avg. base final grade
60 93 76.5 85 81 B-
90 95 92.5 85 89 B+
91 99 95 85 90 A-
80 95 87.5 85 86 B
75 95 85 85 85 B
70 80 95 81.67 75 78 C+
75 85 95 85 75 80 B-
65 88 93 82 75 79 C+
70 75 90 95 82.5 65 74 C
85 88 93 88.67 75 82 B-
95 95 A
98 98 A+

So, did it work?  Yes and no.

The main goal was to elicit better writing from the students.  I can honestly say that the quality of writing improved quite a bit over the course of the semester (the average grades on the three papers, in order, were 85, 88, and 92 -- and I'd like to think the grading standards were fairly consistent).  I foresaw a couple drawbacks: 1.) the system was a lot more work for me, and 2.) the students might rebel.

The first was was definitely true.  At first I enthusiastically dove into drafts and re-writes and wrote copious comments, but after reading the same papers over and over again it began to get tedious.  After the first draft of the first paper, students figured out that sending me a draft to read over ahead of time was a good idea (they were right), and I gave very few students an 'A' on their first attempt (nobody earned a straight 'A' on more than one of the three papers), meaning that I read drafts of the first version, graded the first versions, read drafts of the second versions, graded the second versions, and so on.  Luckily, most people got it by the second version.

The second issue wasn't as bad as I feared.  I just got my teacher evaluations back and, despite some griping about grading and the papers, they were quite good.  The comments specifically about the papers were decidedly mixed, with a few students obviously bitter over the grading system, but at least as many acknowledging that it helped them.  I've posted all those comments at the end of this post.

So, would I do it again?  Yes . . . well, maybe.  On the one hand, I'm convinced from what I saw and heard that this substantially benefited students in the end.  On the other, it was an awful lot of work for me (and I think it would be horribly draconian and unfair to implement it without offering extra writing feedback).  I'm willing to work on tweaking the grading system a bit to soothe hurt feelings, but it may be as much about framing the issue (e.g. two papers is supposed to be a B, so turning in a 91 and 97 and earning an 89 is actually more fair than earning an 85) as it is about actual grading.  Before doing this again, I think I would try even harder to make expectations and directions comprehensive and crystal clear both to help students and to potentially reduce the amount of drafts and re-writes I have to read.  I think teachers have to walk a fine line between setting a high bar for students and being unnecessarily punitive.  Nobody would accuse me of simply handing out A's just for being there, but I'd like to think I also avoided simply piling on so I could brag about how difficult I made the students' lives.  At least in this case, I think setting a high bar in terms of quality rather than quantity was what was best for the students.

The "revolutionary" in the title was mostly tongue-in-cheek, in case you didn't get that, but I'll continue to tinker with these ideas.  Any feedback or suggestions are most welcome.

Here are the student comments about the papers:

general comments about instructor/course

"I actually learned how to write policy evaluations and proposals and could see my improvement. I really appreciate all the time that Corey took to give feedback on papers no matter how late they were sent in. If every teacher were as helpful as Corey, I would be learning a lot more in my classes."

"The paper grading is atrocious. Requires re-writes until student gets what he deems a 92. Amount of re-writes dictates what average is. For example, one rewrite means the two grades you get are averaged with an 85. I got a 91 on a paper, then a 97, which averages to a 94, but when that is averaged with an 85, it becomes an 89, which is lower than my initial grade. Pointless and penalizes unnecessarily."

The strongest feature of this course was:

"The papers and application to real word current events."

"The emphasis on clear and concise writing and the importance placed on the quality of writing over the quantity."

The weakest feature of this course was:

"the grading policies"

"unclear expectations for policy papers"

"THE PAPERS! Haha. I hated having to do write re-writes, but I understand the reasoning behind it. The grading system is frustrating when one receives a 91 on the first drafts... but I really do understand the importance and value behind doing it this way."

"The grading policy of the papers. I understand the reasoning behind it, but it was still rough to get a 91 on a paper and then actually wind up with a lower grade in the end after rewriting it."

"Grading policy on papers"

"The policy papers were good in theory... they ended up being crap. We cannot read minds and therefore can never satisfy the professor. It is unfair that if someone gets a 91 originally, their grade is LOWERED after improving it."

My suggestions for the improvement of this course are:

"new grading policy"

"Maybe prepare students more for papers because it's hard to learn what exactly is required. It took a few attempts before I fully understood what was needed for a strong paper."

My suggestions for the instructor to improve her/his teaching are:

"Keep doing the re-writes, they were a pain in the rear but improved my writing dramatically. NEVER use the Andersen book and just use CNN articles related to current events."

"Change the grading of paper rewrites so that the three average together instead of how you do it. It seems fairer. Ex. Your Way 91 to 97 = (((91+97)/2)+85)/2 = 89.5 Ex. Ideal Way 91 to 97 = (91+97+85)/3 = 91"

"Paper grading policy. It may seem like a good idea in theory and I agree that it does increase writing skill, but the way grades are calculated is foolish"

"One improvement I would suggest would be tweaking the grading system for the papers. The general system in place is effective in that it challenges the students to learn through experience how to be better writers, but at times it can be a little harsh."


Ophelia said...

Maybe a more amenable policy would be to say that a student couldn't get a lower grade than their first grade (unless they opted for a zero by not working for an A). Writing is, no matter how clear we try to make it, still arguably subjective, and I'm inclined to ere on the side of leniency to accommodate that. I understand the idea behind penalizing additional draft-writing, but I'd agree with your students that it seems rather unfair to get a 91 on your first paper and end up with an 89 as your final grade, even though you've more than exceeded an A paper on the draft. Maybe if you gave them the 91 but still made them work to an A they wouldn't have as much cause to view it as unjust, and you'd still reach the ones who believe it improves their writing.

Attorney DC said...

I agree with Dylan that the final grade should not be lower than the first draft grade: It's bizarre that two A's = B. However, I think it's commendable that you assist the students in polishing their writing...just do it in a more positive way.

Corey Bunje Bower said...

I'm not sure I agree. I can see why that would be frustrating, but the assignment grade is based on how many times it takes one to write an 'A' paper, not on how good the first paper is. Maybe I should just skip assigning a number grade until the last paper to avoid confusion. I'll have to give that some more thought before the next time.

M said...

I agree with the comments about the lower overall grade than the initial paper...that seems unfair. Additionally, were a lot of 91s handed out? If so, that would be irritating.
But I do think that your idea of revision and feedback is a great one. If more teachers did that, students might leave college with writing skills that allow them to actually compose a professional e-mail (which, sadly, is not always--or even usually--the case). Lack of feedback is one of the biggest complaints students have in even though your grading policy is a bit backward in my opinion, the important thing is that those students who might have done the classic write-it-the-night-before number actually wrote the paper in advance, rewrote and rethought it, and probably improved. That's the whole point, so despite the grading policy issue...your experiment sounds like a success.

devry university chicago said...

@Atty DC--I agree with you that the lower overall grade than the initial paper that really sounds unfair.