I'm in New York City right now attending the meetings of the Comparative and International Education Society (CIES). I've been mostly hiding away in my room desperately trying to prepare my presentation for this afternoon (Tuesday). As such, I do not yet have much of interest to report other than about my paper.
My paper did not live up to my expectations for it, but I think my presentation made sense and I found one interesting thing. The basic motivation for the paper was to find out why some countries have a small spread in achievement and some countries have a large spread in achievement. In the end, I couldn't really find much that looked like compelling predictors for whether achievement in countries would be more or less spread out.
But I did find one thing that surprised me. The last thing that I checked on, mostly out of curiosity since it was really only tangentially related to my topic, was whether higher performing countries had more or less equality. I had a couple different measures of variation in achievement, and I compared them to the median score on the TIMSS (an international assessment involving about 50 countries the last time) and found a really strong correlation between equality and achievement (about .8, p<.001 for you stats nerds), meaning that, within the TIMSS countries, that higher achieving countries were distinctly more equal than lower achieving countries. When I compared performance on TIMSS to spread on PISA (another international assessment) the relationship still held and was moderately strong (about .4, p<.001).
The strength of the relationship was of a level that one just doesn't find while doing research, so I was sure I was doing something wrong, but nobody has given me reason to think that this isn't true and I can't think of any.
I don't know what this would look like for different assessments, different years, different subjects, and different ways of measuring variance (or spread, or inequality, or whatever you want to call it), but it's a result that could potentially be meaningful. It's at least as possible, of course, that the result is either meaningless or won't hold up with other data, but I think it's worth further investigation.
As of this moment, I'm seeing that more equality=higher achievement and wondering whether that means what one might assume it means.
2 comments:
One selection bias that could produce the correlation would be if the countries that were higher achieving (and showed less inequality) in fact tested a smaller (and more uniformly high achieving) cohort of students than the countries with lower achievement and more inequality.
Yes, that's true. I know that the sampling for the international assessments hasn't been perfect, but I'm not sure if there's any reason to believe that there's a systematic bias toward sampling less diverse groups in higher achieving countries. Though I guess avoiding sampling the lowest achievers in a country would both narrow the achievement gap in your country and raise the median score. I'll look into it. Thanks.
Post a Comment