America’s World Education Ranking Is No Reason to Panic
There has been no shortage of sound and fury about the recently released education rankings from the Programme for International Student Assessment (PISA). Every three years, the organization tests 15-year-olds from across the globe, comparing results from about 60 nations. As soon as the scores from the latest exam were released last month, Americans began wringing their hands over the results.
At the top of the heap were the usual suspects from East Asia: Singapore, Taiwan, South Korea, and three Chinese cities—Shanghai, Hong Kong, and Macao. The United States came in 36th, trailing nations such as Estonia and the Slovak Republic. The national consensus, articulated in a New York Times editorial, was that America’s students are falling “further and further behind.”
Placing behind 35 other nations can be disheartening. Yet it’s worth asking: What do the PISA test results actually signify?
First, let’s assume that PISA is a gold-standard measure. From this standpoint, we might take heart in the fact that the results showed no statistically significant difference between the performance of American students and those in a place like Norway, which was recently named by the United Nations as the best place in the world to live. So no, we didn’t come in first, but we did land in the middle of the pack among a highly select group of prosperous nations.
But let’s stop assuming so much about PISA. According to Cambridge statistician David Spiegelhalter, the statistical model used to generate PISA scores is “demonstrably inadequate.” As a result, the accuracy of the national rankings is highly disputable. Also, PISA results conflict with trends in other international tests on which American students have performed better. In the latest Trends in International Mathematics and Science Study (TIMSS), only seven education systems had higher math performance among fourth graders. The results were fairly similar among eighth graders. In short, we’re 36th in the world; that is, unless we’re seventh.
We might further ask what exactly is on a test such as PISA. How can a test measure all of the different things that students learn across various national school systems? After all, comparative measures require a common denominator, and no such denominator exists. PISA’s approach is to measure a set of “real world” skills “not constrained” by national curricula. This creates an obvious problem: The test measures what many schools do not teach. In short, it establishes a level playing field, but it does so by ignoring what each system is actually trying to accomplish. As my colleague Professor David Labaree put it, “this is leveling the playing field with a bulldozer.” Such issues are further compounded by the fact that the validity of PISA test items is unsupported by research.
Finally, when digesting the results of a test such as PISA, it is worth reflecting on the degree to which it presents a complete picture of schools. Unlike the exam-oriented nations perpetually at the top of international rankings, Americans want their schools to be more than just test-prep paddocks. We want students to develop social skills and a work ethic. We want them to grow as citizens and gain a sense of community responsibility. We want them to be physically and emotionally healthy. We want them to be exposed to the arts and to literature. And we want them to feel cared for. These values are not measured by tests such as PISA, nor is it reasonable to think they ever will be.
Keeping all of this in mind is a matter of critical importance because the rhetoric of falling skies causes stakeholders to doubt the quality of our public schools. Without question, most K–12 schools could improve, and a small number of them are truly failing. Yet when we overgeneralize about failure, or when we react thoughtlessly to tests like PISA, we undermine the faith that draws a diverse group of parents and their children into the schools, we erode the goodwill of taxpayers who fund this collective enterprise, and we justify a reactionary and interventionist policy agenda. Heedlessly perpetuating a narrative of crisis, we just might bring one about.
This article was created as part of the social action campaign for the documentary TEACH, produced by TakePart's parent company, Participant Media, in partnership with Bill and Melinda Gates.