Skimming through the latest Bracey Report, I calculated the following statistic from the table on page 3:
If we look at achievement scores on the 2001 Progress in International Reading Literacy Study (PIRLS), Sweden ranks first with a median score of 561, while the U.S. is a little further back with an average score of 540. If we look only at the at the students who attend schools where less than 50% of the students are in poverty (I assume measured by the percent eligible for free/reduced-price lunch, but I'm not sure), then we have a sample comprising 63.6% of the American population. And their average score is 564.
In other words, the average American student attending a school that doesn't rank among the poorest third in the country out-achieved the average student in every other country that took part in the assessment.
I don't know whether other, more recent, international assessments would yield similar results, but we do know that our top students out-achieved the top students in most other G-8 countries, while the opposite was true for our bottom students, on the PISA 2000 Literacy test. Two pieces of data don't warrant a strong conclusion, but both indicate that our top students are doing pretty well while our bottom students lag far behind -- which would indicate that we should spend most of our time trying to pull up those at the bottom.