Numbers don’t lie, right? But they also don’t always tell the whole story. That’s the case with the most recent results from a key global education test, the Program for International Student Assessment or PISA. 

In the past, PISA results have often spurred anguished discussion about why U.S. students are so far behind other countries like Finland, Korea and Poland. But the most recent rankings, released in December 2023, indicated that U.S. 15-year olds moved up in the international rankings for all three subjects –  math, reading and science. Education Secretary Miguel Cardona credited the largest federal investment in education in history – roughly $200 billion – for keeping the United States “in the game” during the pandemic. (The tests were administered in 2022.)

But that rosy spin hides a much grimmer picture. Rankings may have risen, but test scores did not. The only reason the U.S. rose is because academic performance in once higher ranking countries, such as Iceland and the Slovak Republic, fell by even more since the previous testing round in 2018. Neither India nor China, which topped the rankings in 2018, participated in the 2022 PISA. In math, the U.S. rose from 29th place to 28th place, still in the bottom half of economically advanced nations of the Organization for Economic Cooperation and Development (OECD), an international organization of 38 member countries that oversees the PISA exam.

The deterioration in math was particularly devastating. American students scored 13 points lower than in 2018, equivalent to losing two-thirds of a year of education in the subject. These were the lowest U.S. math scores recorded in the history of the PISA math test, which began in 2003. More than a third of U.S. 15-year-olds (mostly 10th graders) are considered to be low performers, unable to compare distances between two routes or convert prices into a different currency. Over the past decade, the share of U.S. students in this lowest level has swelled; back in 2012, a little over a quarter of U.S. students were considered to be low performers.

Only seven percent of American students can do math at advanced levels. The United States has more students in the bottom group and fewer students in the top group than most other industrialized countries that are part of the OECD. (Click here to see an international ranking of low and top performers in each country.)

The results also confirmed the widespread inequalities in U.S. education. Black and Hispanic students, on average, scored far below Asian and white students. Those from low-income backgrounds scored lower than their more affluent peers.

Andreas Schleicher, director for education and skills at the OECD, emphasized that the inequities in the U.S. are often misunderstood to be primarily problem of weak schools in poor neighborhoods. His analysis indicates that low math performance is common throughout U.S. schools. Some students are performing much worse than others within the same school, and that range between low and advanced students within U.S. schools is much greater than the range in scores between schools. 

This new PISA test is the first major international education indicator since the Covid pandemic closed schools and disrupted education. Test scores declined all around the world, but the OECD found there was only a small relationship between how long schools were closed and their students’ performance on the PISA test. School closures explained only 11 percent of the variation in countries’ test scores; nearly 90 percent is attributable to other, unclear reasons. However, the OECD looked at the absolute level of test scores and not how much test scores fell or rose. More analysis is needed to see if there’s a stronger link between school closures and test score changes. 

Even if school closures eventually prove to be a more important factor, the pandemic isn’t the only reason students are struggling. Global scores have generally been declining for the past two decades. One hypothesis is that technology is distracting teenagers. Students were asked about technology distraction for the first time on the 2022 PISA. Forty-five percent of students said they feel anxious if their phones are not near them. Sixty-five percent report being distracted by digital devices during math lessons. Up to an hour a day of computer time for leisure was associated with higher performance. But heavy users, those who spent five to seven hours on computers for fun, had lower academic performance, even after adjusting for family and school socioeconomic profiles. 

Another factor could be the rise in migration across the world. Perhaps declining test scores reflect the challenge of educating new immigrants. However, the OECD didn’t find a statistically significant correlation between immigration and academic performance on average. In the United States, immigrants outscored students with native-born parents in math after adjusting for socio-economic status. There was no difference between immigrants and non-immigrants in reading.

Japan was one of the few countries to defy the trends. Both its math and reading scores rose considerably between 2018 and 2022. Akihiko Takahashi, professor emeritus of mathematics and mathematics education at Chicago’s DePaul University, said schools were closed for a shorter period of time in Japan and that helped, but he also credits the collective spirit among Japanese teachers. In his conversations with Japanese teachers, Takahashi learned how teachers covered for each other during school closures to make sure no students in their schools fell behind. Some went house to house, correcting student homework.  

It’s tempting to look at the terrible PISA math scores and say they are evidence that the U.S. needs to change how it teaches math. But the PISA results don’t offer clear recommendations on which math approaches are most effective. Even Japan, one of the top performing nations, has a mixed approach. Takahashi says that students are taught with a more progressive approach in elementary school, often asking students to solve problems on their own without step-by-step instructions and to develop their own mathematical reasoning. But by high school, when this PISA exam is taken, direct, explicit instruction is more the norm.

The new results also highlighted the continued decline of a former star. For years, Finland was a role model for excellent academic performance. Education officials visited from around the world to learn about its progressive approaches. But the country has dropped 60 points over the past few testing cycles – equivalent to losing three full school years of education. I suspect we won’t be hearing calls to teach the Finnish way anymore. “You have to be careful because the leaders of today can be the laggards of tomorrow,” said Tom Loveless, an independent researcher who studies international assessments.

There was one bright spot for American students. Fifteen-year-olds scored comparatively well on the PISA reading test, with their scores dropping by just one point while other countries experienced much steeper declines. But that good news is also tempered by the most recent scores on the National Assessment for Educational Progress (NAEP) test, often called the Nation’s Report Card. Reading scores of fourth and eighth graders deteriorated over the last two testing cycles in 2019 and 2022.

Overall, the PISA results provide additional confirmation that U.S. students are in trouble, especially in math, and we can’t put all the blame on the pandemic.

This story about the 2022 PISA results was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: There is a worldwide problem in math and it’s not just about the pandemic appeared first on The Hechinger Report.

Parents are often stymied by the process of picking a good school for their kids. Word-of-mouth recommendations can be misleading. High test scores provide only a limited picture of a school’s effectiveness since they often reflect family income with wealthier students scoring better. Northwestern University economist C. Kirabo Jackson believes two additional elements point to an effective school: social and emotional skills and student behavior.  He argues that schools that boost those two plus test scores propel more students to graduate high school, go to college and reduce the number of students who get arrested. 

“We’re talking about schools that have a positive, causal impact on children’s subsequent outcomes,” said Jackson. “Not everyone buys into these non-test score measures of school effectiveness. I think what we’re showing here is that they do require attention.”

To prove his point, Jackson studied more than 150,000 ninth graders in Chicago public high schools from 2011 to 2017. He picked Chicago because the school system annually surveys students about their social and emotional skills. Students report how much effort they believe they put into their school work and how they feel about their relationships with peers. All the answers can be quantified on a scale of one to four, enabling Jackson to calculate how much students’ social and emotional development improved at each school, along with increases in test scores and decreases in disciplinary incidents, such as suspensions. Then he combined it all into one composite index, somewhat similar to how a combined SAT score adds together math and verbal tests. 

He found that Chicago high schools that ranked high on his three-part index for ninth graders subsequently reduced the number of arrests at school while increasing high school graduation and college enrollment rates. Students at schools that only produced the highest growth in test scores had less impressive long-term outcomes.

The non-test score aspects of school quality seem to be driving many of the results. Schools that improve student behavior the most had the largest drop in school-based arrests. Schools that boost social and emotional skills had larger increases in college attendance.

Jackson’s study, “Who Benefits From Attending Effective Schools? Examining Heterogeneity in High School Impacts,” was conducted with three other researchers, including John Q. Easton, the former head of the U.S. Department of Education’s Institute for Education Sciences and a senior fellow at the University of Chicago’s Consortium on School Research.*  The team also included Shanette Porter, research director at the Mindset Scholars Network, which promotes “soft” skills, such as the belief that intelligence can be developed through hard work. 

The study is still a draft, working paper, which means it hasn’t yet been peer-reviewed, but it was circulated by the National Bureau of Economics Research in December 2020. The ideas in it rely on previously published research about the benefits of social and emotional development and fostering good behavior. 

The names of Chicago’s more effective high schools were not disclosed because of the researchers’ data sharing agreement with the city’s public school system. That’s too bad because parents might appreciate knowing that information. All students benefit from attending more effective high schools that boost a combination of test scores, behavior and social and emotional skills, the researchers found. 

However, students with the weakest eighth grade academic records, who are the most likely to drop out of high school, received larger benefits from attending one of Chicago’s most effective schools while more advantaged students were more likely to attend these higher quality schools in the first place. 

For example, a disadvantaged student in the bottom 10 percent would be 3.4 percentage points more likely to graduate from high school, 2.2 percentage points more likely to enroll in college and 2.1 percent less likely to be arrested by attending a high school in the top 15 percent of effectiveness compared to an average high school. For a more advantaged student at the other end of the spectrum, there were very small improvements in high school graduation and arrests but college going rose by almost as much as it did for disadvantaged students. 

The researchers also noticed a shift in college behavior. Disadvantaged students were more likely to end up at a two-year community college after attending a more effective high school. Wealthier students were not only more likely to go to college but also more likely to enroll in a four-year college. “You’re getting a shift from two-year to four-year colleges among those that are more advantaged,” said Jackson. That’s a good outcome because graduation rates are much higher at four-year institutions than at two-year community colleges and adults with bachelor’s degrees generally earn higher salaries.

It’s unclear whether the researchers’ index would be as useful outside of Chicago in cities with more affluent students. Chicago’s schools are overwhelmingly poor; 86 percent of the ninth graders in the study were from disadvantaged families whose incomes were low enough to qualify for free or reduced price lunch. Forty-two percent were Black students and 44 percent were Latino students. Only half of the ninth graders subsequently attended college.

If this approach to measuring schools turns out not to be useful for affluent students, that’s okay. Our problem in America isn’t figuring out which schools serve upper middle class students well  but how to educate children living in poverty. And this study is shining a light in a new direction. Unfortunately, this study didn’t delve into exactly what the effective high schools are doing to improve students’ soft skills and behavior. We don’t know if the schools were following specific programs or staffed by caring adults or some combination of both. The next step is to figure out exactly what these effective Chicago high schools are doing and bottle it. 

* Correction: An earlier version of this story incorrectly described John Easton’s title at the University of Chicago’s Consortium on School Research.

This story about effective schools was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Researchers pinpoint three elements of effective schools appeared first on The Hechinger Report.