Tuesday, July 1, 2008

How to Fix Testing?

Maybe it's b/c my brain is still only half-functioning, but I feel like writing something today about which I know little and that will probably result in a lot of people calling me stupid.

Testing is a huge part of American education, but it's riddled with all sorts of problems. One problem is that most tests are simply not a valid measurement of student knowledge/aptitude/performance (or whatever it is that the test strives to measure). Allow me one example. My first year, all my students took the English exam at the end of the year and their performance was compared to the previous year. This makes sense on paper. But, in reality, the exam was 50 minutes long and contained 50 multiple choice questions. The internal validity, in short, was lacking. Economists would say there was "a lot of noise" in the test results -- the scores of the students varied widely from their true ability based on all sorts of random reasons. This is one flaw with the value-added formula -- if we're not really sure how much the kid knew last year, then we can't really tell how much they learned this year.

Many have also complained that students spend all day on "test prep" in the weeks or months (I was told to stop teaching social studies after Christmas and do practice English tests) leading up to the exam -- in effect, teachers are trying to teach kids to game tests. This not only robs kids of other learning opportunities but, if done correctly, artificially raises scores.

Anyway, skoolboy posted a tongue-in-cheek call for more testing on eduwonkette's blog today. While I was teaching I thought up a somewhat similar proposal to his, but my tongue was not in my cheek. Here's one way I think testing could be done that would solve a lot of the problems:

Every class in the school takes one test approximately every two weeks. The time, day, and subject matter of the test are random. The teacher and student find out that they will be taking a test a certain period the following day at the end of the previous day. They are not informed of the subject matter until they arrive in the testing room (a testing room equipped with computers is necessary to ensure that all these tests can be graded quickly). By the end of the year, each student has taken about 20 tests spread across all subjects. This provides a much broader base on which to judge students' knowledge. Since testing runs all year and one never knows on what subject they'll be tested next, test prep is impossible to do effectively or to sustain throughout the pre-test time period. The system will be a bit of a pain in the butt and will generate some complaints from teachers. A well-organized testing coordinator and a well-run testing room would have to present in each school. A wide variety of high-quality tests that can be graded largely by computer will have to be available -- national testing may be the only affordable way to do this. But, if done correctly it would solve a lot of problems.

In short, I'm not convinced that more testing isn't the answer.


Roger Sweeny said...

I think this is an excellent idea.

One problem I have with testing is that right now all tests are gamed.

The teacher finishes up a unit and tells the kids x, y, and z will be on the test tomorrow. Some of the kids go home and memorize x, y, and z. Others remember some of it because, hey, kids have pretty good short term memories and they've heard, if not really understood, x, y, and z for the last two weeks. So they all get certain grades on the test which are supposed to indicate what they have "learned." But talk to them a month or two later and most of them can't tell x, y, or z from what they had for breakfast.

Corey Bunje Bower said...

Good point. We read one book during pre-service training where the author argued that the best test would be one given a year after students had learned something -- then we'd know what they *actually* learned.

Roger Sweeny said...

You wouldn't happen to have an author and title for that book, would you?

Matt Johnston said...

I like the idea of random testing and I think that when it comes to all testing, the more random the better.

But early in your post you note a problem with internal validity. Your proposal won't exactly fix that, although it could if the test were designed properly.

The problem noted by Roger in the comments also presents a slightly differnt problem, that of recall versus retention, by which I mean that students can recall what was taught two weeks ago or yesterday, but can't retain what was taught six months ago, let alone last year. What this calls for is the testing regime to not only test what has been recently taught, but to go back in time to test retention of previous material.

But another matter that a random testing regime needs to account for is multiple test format, from mulitple choice to short answer to essay length prompts. There is also the need in some subjects to have students "show their work" i.e. maths and sciences.

Still, randomization will work wonders to avoid the massive test prep regimes that are to a certain extent complained about by parents and teachers.