Friday, November 13, 2009

Interesting Factoid of the Day

Skimming through the latest Bracey Report, I calculated the following statistic from the table on page 3:

If we look at achievement scores on the 2001 Progress in International Reading Literacy Study (PIRLS), Sweden ranks first with a median score of 561, while the U.S. is a little further back with an average score of 540.  If we look only at the at the students who attend schools where less than 50% of the students are in poverty (I assume measured by the percent eligible for free/reduced-price lunch, but I'm not sure), then we have a sample comprising 63.6% of the American population.  And their average score is 564. 

In other words, the average American student attending a school that doesn't rank among the poorest third in the country out-achieved the average student in every other country that took part in the assessment.

I don't know whether other, more recent, international assessments would yield similar results, but we do know that our top students out-achieved the top students in most other G-8 countries, while the opposite was true for our bottom students, on the PISA 2000 Literacy test.  Two pieces of data don't warrant a strong conclusion, but both indicate that our top students are doing pretty well while our bottom students lag far behind -- which would indicate that we should spend most of our time trying to pull up those at the bottom.

5 comments:

Aaron Pallas said...

Corey,

My eduwonkette posts using the 2006 PISA data suggest that our top students were well behind other countries in mathematics and, to a lesser extent, in science. See http://blogs.edweek.org/edweek/eduwonkette/2008/08/skoolboy_goes_to_the_olympics.html and http://blogs.edweek.org/edweek/eduwonkette/2008/08/skoolboy_goes_to_the_olympics_1.html

Aaron Pallas said...

Whoops -- the tail end of the url was cut off for some reason. Anyway, eduwonkette August 2008.

Corey Bunje Bower said...

Hmmm . . . looks like I was right to caution about drawing strong conclusions from two pieces of data.

I'm asking around to see if there's any consensus in the research on this and getting different answers from different people.

Nancy Flanagan said...

May I suggest this piece? (which uses both PISA and TIMSS data)

http://www.nature.com/nature/journal/v453/n7191/pdf/453028a.pdf

A brief snippet: "Still, average performance tells us nothing about the distribution of students with the very best test scores. In maths and science, when looking at average scores, the United States is outranked by countries such as Finland and South Korea. But the rankings change when we examine the percentage of students who perform at the top, those most likely to be tomorrow’s innovators. The South Korean average places it in the top-ranked group of nations,
yet its relative proportion of top performing students is 30% lower than that of the United States. In fact, the United States has a higher percentage of top-performing students than 5 of the 14 others in the top-ranked group of countries with high average scores.

Moreover, it would seem inappropriate to consider the United States, a country with a
population of more than 300 million, in competition with Singapore, a country of 4.5 million, or with even smaller New Zealand."

I'm not arguing here with the estimable and right-thinking Aaron Pallas--only suggesting that there are lots of ways to slice and dice statistics. The problem in American education is that too few of our students are receiving the best and most challenging education, and too few of our students believe that their own hard academic work will lead to a better future.

International tests don't tell us the best way to make those things happen, and divert lots of energy into "my statistical modeling is better than your statistical modeling."

Corey Bunje Bower said...

Thanks for the link.

While I agree that international tests, along with many others, can lead people into a battle over statistical models rather than help us improve schools, I still think their benefits outweigh their drawbacks.

Don't forget that TIMSS, PISA, and PIRLS also include extensive surveys of other aspects of each country's educational system that *can* help us find the best way to improve our system.