How to Fix Testing?

Maybe it's b/c my brain is still only half-functioning, but I feel like writing something today about which I know little and that will probably result in a lot of people calling me stupid.

Testing is a huge part of American education, but it's riddled with all sorts of problems. One problem is that most tests are simply not a valid measurement of student knowledge/aptitude/performance (or whatever it is that the test strives to measure). Allow me one example. My first year, all my students took the English exam at the end of the year and their performance was compared to the previous year. This makes sense on paper. But, in reality, the exam was 50 minutes long and contained 50 multiple choice questions. The internal validity, in short, was lacking. Economists would say there was "a lot of noise" in the test results -- the scores of the students varied widely from their true ability based on all sorts of random reasons. This is one flaw with the value-added formula -- if we're not really sure how much the kid knew last year, then we can't really tell how much they learned this year.

Many have also complained that students spend all day on "test prep" in the weeks or months (I was told to stop teaching social studies after Christmas and do practice English tests) leading up to the exam -- in effect, teachers are trying to teach kids to game tests. This not only robs kids of other learning opportunities but, if done correctly, artificially raises scores.

Anyway, skoolboy posted a tongue-in-cheek call for more testing on eduwonkette's blog today. While I was teaching I thought up a somewhat similar proposal to his, but my tongue was not in my cheek. Here's one way I think testing could be done that would solve a lot of the problems:

Every class in the school takes one test approximately every two weeks. The time, day, and subject matter of the test are random. The teacher and student find out that they will be taking a test a certain period the following day at the end of the previous day. They are not informed of the subject matter until they arrive in the testing room (a testing room equipped with computers is necessary to ensure that all these tests can be graded quickly). By the end of the year, each student has taken about 20 tests spread across all subjects. This provides a much broader base on which to judge students' knowledge. Since testing runs all year and one never knows on what subject they'll be tested next, test prep is impossible to do effectively or to sustain throughout the pre-test time period. The system will be a bit of a pain in the butt and will generate some complaints from teachers. A well-organized testing coordinator and a well-run testing room would have to present in each school. A wide variety of high-quality tests that can be graded largely by computer will have to be available -- national testing may be the only affordable way to do this. But, if done correctly it would solve a lot of problems.

In short, I'm not convinced that more testing isn't the answer.

"No Excuses" . . . Except for College Professors

Finally, the post everybody's been waiting for (Make sure you read this first) . . .

Earlier today, I posted a bunch of random quotes from a commentary written by a teacher (see above). So, what prompted this?

The much-discussed Atlantic piece by "Professor X" is finally available online. Since I'd already mentioned it in a previous post, I decided I should read it and see what all the fuss is about.

Most of the discussion has centered around the theme of the article -- that "The idea that a university education is for everyone is a destructive myth" -- but I was struck by something else.

If a K-12 teacher said half the stuff that Professor X did, they would be crucified . . . and by many of the same people who've been agreeing with him.

The article, in essence, says that although he is a very talented and hard-working professor, that he simply cannot teach many of his students to succeed because they're too far behind and have too many obstacles to success.

Imagine a K-12 teaching saying the same thing, with the conclusion being that the students simply didn't belong in school b/c they were incapable of passing. They would be branded as subscribing to the "myth of helplessness" and blamed for all the problems of our schools. They would be told that they're either not qualified to be teaching or need to work harder. They would be told not to make excuses and that every child can succeed. And there might be a grain of truth in some of these statements.

But, somehow, Professor X's essay is proof not that he's a miserable teacher but that the students aren't qualified to be taking his course -- and that we should stop pushing so many students to take similar courses.

I continue to find it odd that K-12 and higher education are so similar but are treated so differently.

Teachers in Martinique

On Friday I attended a conference for graduate students at Harvard. The conference was run quite well -- if any grad students are reading I'd encourage you to go and present next year -- and the amenities were quite remarkable given the unbeatable $0 registration fee. I saw a number of interesting presentations, but have been quite busy (I'm now in NYC for another conference) so I'm just going to mention one that I found even more interesting than others.

During the last session of the day Nick Gozik, a student from NYU, spoke about his research (I'm quite sure it's his dissertation) that he conducted in Martinique. Martinique is an island in the Caribbean (somewhat near Venezuela) that was a French colony and their education system is still under the control of France. He spent most of a year on the island and says he spoke with half of the high-school teachers in the system, so you can imagine how much information he collected.

Given that this will take him hundreds of pages to explain, I'm just going to briefly summarize two points I found interesting.

1.) He said that teachers repeatedly emphasized that they followed the curriculum because they were professionals (history teachers identified themselves as "historians," etc.) rather than due to any external forces. The current trend here right now is to essentially punish those who don't follow the curriculum by "holding them accountable." Teachers are different in the two different countries, but I found this contrast fascinating.

2.) Even more interesting is that principals were not responsible for evaluating teachers in their schools because they were not experts in most of the fields (e.g. when a math teacher becomes a principal they do not feel prepared to evaluate a history teacher). Instead, inspectors came to the schools occasionally and both trained and evaluated teachers. I do wonder if this model might actually be better than our current model in the U.S. where principals are essentially expected to both know how and actually do everything. We read an article in a class last week about educational administrators lacking deep knowledge of many subject areas and ways to address this. Our class was divided over whether attempts should be made to intensely train principals in all subjects or if instructional leadership should be ceded to an expert in each field (we were thinking perhaps a revision of the dept. head role, but an outside inspector is another interesting idea).

The Most Compelling Argument Against NCLB

When the No Child Left Behind Act was passed in 2001, it had the potential to both a world of good and a world of bad for education. Why the law and its implementation has been good or bad (or both) has been the subject of countless articles, commentaries, and debates. Now the law is up for renewal and the debates have become even more focused. A lot of people argue that there is some redeeming quality to NCLB (a lot of people like accountability, for instance, or the fact that that the stated goal is essentially to close the achievement gap) but that it needs to be tweaked. While the cacophony continues in Washington, I was lucky enough to catch a bit of a different argument on campus -- twice.

Richard Rothstein was on campus in the fall, and then was on campus again last week for the performance pay conference and stopped by our class to fill us in on his research. His argument against NCLB is the most compelling (and thorough) that I have heard. It goes something like this:

Throughout American history, virtually every important figure has set forth as the goals of education to not only provide basic academic skills to students, but also to teach many other things, such as: critical thinking, social skills, physical and emotional health, and far too many other things to list here. He has compiled an exhaustive supply of quotes and documents supporting this.

NCLB, meanwhile, holds schools accountable only for their performance on tests of basic skills in reading and math. As a result, many studies have shown, schools are increasing the time they spend on reading, math, and test prep, and reducing the time they spend on other activities.

Given that, throughout our history, basic academic skills have been but one purpose of schooling, Rothstein set out to see how important these skills are compared to other things that people want schools to teach. The general public, school board members, state legislators, and superintendents (through extensive surveying) all agreed that basic skills were the most important function of schooling, but indicated that out of 100 possible points spread across 8 different functions only about 20% should be focused on these. The other 80% of schooling, they responded, should be focused on things such as the list I mentioned earlier.

Here's why I like this approach to critiquing NCLB:
It's not an ideological position. He doesn't argue that standardized testing or accountability are inherently good or evil. What he finds is, quite simply, that NCLB encourages schools to focus on only a small portion of what is important and, for that reason, needs to be changed. In 1830 (to cite one of his incredible list of quotations) the Joint Committee of Philadelphia Workingmen argued that everybody should receive a broad education "rather than being limited, as in our public poor schools, to a simple acquaintance with words and ciphers." Unfortunately this seems to be happening under NCLB; the law needs to encourage schools to teach all the important topics rather than condemning students to learn nothing but the basics.

More on Standardized Testing and Private Schools

This blurb in the Tennessean covers a bill recently introduced in the TN legislature that would require all school-age students (even if home schooled or attending private school) to take state tests. Two things that I find interesting:

1. The chief objection to this bill seems to be that taking these tests will narrow the curriculum. I wonder if these same people would put forth a similar argument against testing in public schools. On the one hand, it would seem unfair to argue that students in private and home schools should have broader curricula than students in public schools but, on the other, it seems logical to assume that schools run by the government should also be subject to more governmental regulation.

2. Despite the paucity of information in the blurb, the message board is out of control with anger from both sides. Whether or not any accountability would accompany the state tests is left up to the reader to infer -- and people do, passionately. It seems as though most of the opinions expressed are only relevant if the writer is correctly inferring the rest of the facts.

Must opinion on education policy salways be so reactionary and ideological?

Vouchers or Accountability

A blurb in Ed Week today mentioned the results from the first year of a five-year study on vouchers in Milwaukee. Apparently the Governor and legislature cut a deal: more vouchers were authorized in exchange for making the private schools in which voucher recipients enroll to administer the same state tests as public schools. As a result, students who remain in the Milwaukee public school system and students who receive vouchers and enroll in private schools are both taking the same exams (which makes comparing results awfully easy).

After the first year, the researchers could find no difference in performance between students who enrolled in private schools and students who remained in the public school system. Some of the people leading the investigation are clearly proponents of vouchers, but if these results hold up for the next four years they're going to have to scramble in order to spin them in their favor. And there's a good chance that they have a legitimate argument; private schools aren't accountable for their results on the state tests and, therefore, probably spend a great deal less time preparing students to take them. Here's where it gets interesting: generally speaking, people in favor of vouchers are also in favor of accountability (and, therefore, standardized tests) but, in this case, the only argument to support the effectiveness of vouchers may be that the standardized tests did not accurately represent what happened in the schools. In other words, the only logical argument that I can foresee is that either vouchers have shortcomings or standardized tests have shortcomings -- either way somebody is going to put in an uncomfortable position when they present the findings. I love twists of fate.