Silence may be golden, but when it comes to learning with a tutor, talking is pure gold. It’s audible proof that a student is paying attention and not drifting off, research suggests. More importantly, the more a student articulates his or her reasoning, the easier it is for a tutor to correct misunderstandings or praise a breakthrough. Those are the moments when learning happens.

One India-based tutoring company, Cuemath, trains its tutors to encourage students to talk more. Its tutors are in India, but many of its clients are American families with elementary school children. The tutoring takes place at home via online video, like a Zoom meeting with a whiteboard, where both tutor and student can work on math problems together. 

The company wanted to see if it could boost student participation so it collaborated with researchers at Stanford University to develop a “talk meter,” sort of a Fitbit for the voice, for its tutoring site. Thanks to advances in artificial intelligence, the researchers could separate the audio of the tutors from that of the students and calculate the ratio of tutor-to-student speech.

In initial pilot tests, the talk meter was posted on the tutor’s video screen for the entire one-hour tutoring session, but tutors found that too distracting. The study was revised so that the meter pops up every 20 minutes or three times during the session. When the student is talking less than 25 percent of the time, the meter goes red, indicating that improvement is needed. When the student is talking more than half the time, the meter turns green. In between, it’s yellow. 

More than 700 tutors and 1,200 of their students were randomly assigned to one of three groups: one where the tutors were shown the talk meter, another where both tutors and students were shown the talk meter, and a third “control” group which wasn’t shown the talk meter at all for comparison.

When just the tutors saw the talk meter, they tended to curtail their explanations and talk much less. But despite their efforts to prod their tutees to talk more, students increased their talking only by 7 percent. 

When students were also shown the talk meter, the dynamic changed. Students increased their talking by 18 percent. Introverts especially started speaking up, according to interviews with the tutors. 

The results show how teaching and learning is a two-way street. It’s not just about coaching teachers to be better at their craft. We also need to coach students to be better learners. 

“It’s not all the teacher’s responsibility to change student behavior,” said Dorottya Demszky, an assistant professor in education data science at Stanford University and lead author of the study. “I think it’s genuinely, super transformative to think of the student as part of it as well.”

The study hasn’t yet been published in a peer-reviewed journal and is currently a draft paper, “Does Feedback on Talk Time Increase Student Engagement? Evidence from a Randomized Controlled Trial on a Math Tutoring Platform,” so it may still be revised. It is slated to be presented at the March 2024 annual conference of the Society of Learning Analytics in Kyoto, Japan. 

In analyzing the sound files, Demszky noticed that students tended to work on their practice problems with the tutor more silently in both the control and tutor-only talk meter groups. But students started to verbalize their steps aloud once they saw the talk meter. Students were filling more of the silences.

In interviews with the researchers, students said the meter made the tutoring session feel like a game.  One student said, “It’s like a competition. So if you talk more, it’s like, I think you’re better at it.” Another noted:  “When I see that it’s red, I get a little bit sad and then I keep on talking, then I see it yellow, and then I keep on talking more. Then I see it green and then I’m super happy.” 

Some students found the meter distracting.  “It can get annoying because sometimes when I’m trying to look at a question, it just appears, and then sometimes I can’t get rid of it,” one said.

Tutors had mixed reactions, too. For many, the talk meter was a helpful reminder not to be long-winded in their explanations and to ask more probing, open-ended questions. Some tutors said they felt pressured to reach a 50-50 ratio and that they were unnaturally holding back from speaking. One tutor pointed out that it’s not always desirable for a student to talk so much. When you’re introducing a new concept or the student is really lost and struggling, it may be better for the teacher to speak more. 

Surprisingly, kids didn’t just fill the air with silly talk to move the gauge. Demszky’s team analyzed the transcripts in a subset of the tutoring sessions and found that students were genuinely talking about their math work and expressing their reasoning. The use of math terms increased by 42 percent.

Unfortunately, there are several drawbacks to the study design. We don’t know if students’ math achievement improved from the talk meter. The problem was that students of different ages were learning different things in different grades and different countries and there was no single, standardized test to give them all. 

Another confounding factor is that students who saw the talk meter were also given extra information sessions and worksheets about the benefits of talking more. So we can’t tell from this experiment if the talk meter made the difference or if the information on the value of talking aloud would have been enough to get them to talk more.

Demszky is working on developing a talk meter app that can be used in traditional classrooms to encourage more student participation. She hopes teachers will share talk meter results with their students. “I think you could involve the students a little more: ‘It seems like some of you weren’t participating. Or it seems like my questions were very closed ended? How can we work on this together?’”

But she said she’s treading carefully because she is aware that there can be unintended consequences with measurement apps. She wants to give feedback not only on how much students are talking but also on the quality of what they are talking about. And natural language processing still has trouble with English in foreign accents and background noise. Beyond the technological hurdles, there are psychological ones too.

 “Not everyone wants a Fitbit or a tool that gives them metrics and feedback,” Demszky acknowledges.

This story about student participation was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Proof Points newsletter.

The post PROOF POINTS: How to get teachers to talk less and students more appeared first on The Hechinger Report.

Every December, I compile a list of the most popular Proof Points columns. In 2023, the big story was the failure of schools to help children catch up from pandemic learning losses. I was proud to write several watchdog stories about the online tutoring that schools are buying and the crazy job of an online tutor. I was surprised that those were not among the most popular stories. What struck a chord with readers was research about math, math and more math….and some handy study tips. 

Thank you to everyone who read and commented on my weekly stories about education data and research. I look forward to continuing this conversation with you next year. If you would like to receive my email newsletter and be notified when the column comes out each week, please click here and fill out the form. I’ll be back again on Jan. 2, 2024 with a story about college applications. Happy New Year!

1. PROOF POINTS: Inside the perplexing study that’s inspired colleges to drop remedial math

CUNY’s seven-year study of its experiment to get rid of remedial math has arguably been one of the most influential attempts to use experimental evidence to change how higher education operates and is now affecting the lives of millions of college students. However, many colleges may not realize that the study never looked at how to help students pass college algebra, an important prerequisite course. Some researchers argue that the shift to statistics instead of the elimination of remedial ed might have driven the results.

2. PROOF POINTS: How a debate over the science of math could reignite the math wars

This column explained the origins of the new “science of math” movement, and how a group of special education researchers are seeking to copy the science of reading playbook. Their first manifesto attacked what they described as common misconceptions about teaching math.

3. PROOF POINTS: Professors say high school math doesn’t prepare most students for their college majors

College professors often lament that their students are unprepared for the rigor of college-level math. But this Alabama survey highlighted another problem: high schools aren’t teaching the math skills that college professors in non-STEM fields want students to have. 

4. PROOF POINTS: Do math drills help children learn?

I took a look at  the contentious research debate on timed math tests. Short quizzes might be a more efficient way to memorize multiplication tables, but even advocates caution that there are many pitfalls. 

5. PROOF POINTS: New higher ed data by race and ethnicity

Five charts document that white, Black and Hispanic Americans are going to college at about the same rates. But starting does not mean finishing. The likelihood of making it through the coursework and tuition payments and ultimately earning a degree varies by race and ethnicity. 

6. PROOF POINTS: New research review questions the evidence for special education inclusion

People often assert that the research shows that students with disabilities learn better when they are included in regular classrooms. But a new meta-analysis says the research isn’t nearly as clear cut as advocates assert; some students with disabilities may be worse off and learn less in general education classes.

7. PROOF POINTS: A spate of recent studies on the “Google effect” adds to evidence that the internet is making us dumber

Here’s some news all of us can use. The stuff we’re Googling isn’t sticking in our memories and is quickly forgotten. One experiment finds that we’d retain more if we tried to guess before Googling.

8. PROOF POINTS: The best way to teach might depend on the subject

Researchers find that math students learn best through individual practice while English students thrive in groups.

9. PROOF POINTS: Flashcards prevail over repetition in memorizing multiplication tables

Here’s some concrete research advice for teachers on how to help their students memorize things. Since reading about this flashcard experiment, I’ve been telling my daughter to go straight to the cards and skip the studying. Kids may get frustrated not knowing something and looking at the answer over and over again. But she aced two vocab quizzes. (I was sad that multiplication songs weren’t the winner, but if you are nostalgic for Schoolhouse Rock and Billy Leach, I’ve included bonus links in this piece for you to sing along to.) 

10. PROOF POINTS: One expert on what students do wrong

The University of Virginia’s Daniel Willingham explains why reading our notes, using highlighters and making to-do lists are not the best ways to learn. Read the piece for the most effective methods.

This story about the top education research stories of 2023 was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Proof Points newsletter.

The post PROOF POINTS: 2023 in review appeared first on The Hechinger Report.

Numbers don’t lie, right? But they also don’t always tell the whole story. That’s the case with the most recent results from a key global education test, the Program for International Student Assessment or PISA. 

In the past, PISA results have often spurred anguished discussion about why U.S. students are so far behind other countries like Finland, Korea and Poland. But the most recent rankings, released in December 2023, indicated that U.S. 15-year olds moved up in the international rankings for all three subjects –  math, reading and science. Education Secretary Miguel Cardona credited the largest federal investment in education in history – roughly $200 billion – for keeping the United States “in the game” during the pandemic. (The tests were administered in 2022.)

But that rosy spin hides a much grimmer picture. Rankings may have risen, but test scores did not. The only reason the U.S. rose is because academic performance in once higher ranking countries, such as Iceland and the Slovak Republic, fell by even more since the previous testing round in 2018. Neither India nor China, which topped the rankings in 2018, participated in the 2022 PISA. In math, the U.S. rose from 29th place to 28th place, still in the bottom half of economically advanced nations of the Organization for Economic Cooperation and Development (OECD), an international organization of 38 member countries that oversees the PISA exam.

The deterioration in math was particularly devastating. American students scored 13 points lower than in 2018, equivalent to losing two-thirds of a year of education in the subject. These were the lowest U.S. math scores recorded in the history of the PISA math test, which began in 2003. More than a third of U.S. 15-year-olds (mostly 10th graders) are considered to be low performers, unable to compare distances between two routes or convert prices into a different currency. Over the past decade, the share of U.S. students in this lowest level has swelled; back in 2012, a little over a quarter of U.S. students were considered to be low performers.

Only seven percent of American students can do math at advanced levels. The United States has more students in the bottom group and fewer students in the top group than most other industrialized countries that are part of the OECD. (Click here to see an international ranking of low and top performers in each country.)

The results also confirmed the widespread inequalities in U.S. education. Black and Hispanic students, on average, scored far below Asian and white students. Those from low-income backgrounds scored lower than their more affluent peers.

Andreas Schleicher, director for education and skills at the OECD, emphasized that the inequities in the U.S. are often misunderstood to be primarily problem of weak schools in poor neighborhoods. His analysis indicates that low math performance is common throughout U.S. schools. Some students are performing much worse than others within the same school, and that range between low and advanced students within U.S. schools is much greater than the range in scores between schools. 

This new PISA test is the first major international education indicator since the Covid pandemic closed schools and disrupted education. Test scores declined all around the world, but the OECD found there was only a small relationship between how long schools were closed and their students’ performance on the PISA test. School closures explained only 11 percent of the variation in countries’ test scores; nearly 90 percent is attributable to other, unclear reasons. However, the OECD looked at the absolute level of test scores and not how much test scores fell or rose. More analysis is needed to see if there’s a stronger link between school closures and test score changes. 

Even if school closures eventually prove to be a more important factor, the pandemic isn’t the only reason students are struggling. Global scores have generally been declining for the past two decades. One hypothesis is that technology is distracting teenagers. Students were asked about technology distraction for the first time on the 2022 PISA. Forty-five percent of students said they feel anxious if their phones are not near them. Sixty-five percent report being distracted by digital devices during math lessons. Up to an hour a day of computer time for leisure was associated with higher performance. But heavy users, those who spent five to seven hours on computers for fun, had lower academic performance, even after adjusting for family and school socioeconomic profiles. 

Another factor could be the rise in migration across the world. Perhaps declining test scores reflect the challenge of educating new immigrants. However, the OECD didn’t find a statistically significant correlation between immigration and academic performance on average. In the United States, immigrants outscored students with native-born parents in math after adjusting for socio-economic status. There was no difference between immigrants and non-immigrants in reading.

Japan was one of the few countries to defy the trends. Both its math and reading scores rose considerably between 2018 and 2022. Akihiko Takahashi, professor emeritus of mathematics and mathematics education at Chicago’s DePaul University, said schools were closed for a shorter period of time in Japan and that helped, but he also credits the collective spirit among Japanese teachers. In his conversations with Japanese teachers, Takahashi learned how teachers covered for each other during school closures to make sure no students in their schools fell behind. Some went house to house, correcting student homework.  

It’s tempting to look at the terrible PISA math scores and say they are evidence that the U.S. needs to change how it teaches math. But the PISA results don’t offer clear recommendations on which math approaches are most effective. Even Japan, one of the top performing nations, has a mixed approach. Takahashi says that students are taught with a more progressive approach in elementary school, often asking students to solve problems on their own without step-by-step instructions and to develop their own mathematical reasoning. But by high school, when this PISA exam is taken, direct, explicit instruction is more the norm.

The new results also highlighted the continued decline of a former star. For years, Finland was a role model for excellent academic performance. Education officials visited from around the world to learn about its progressive approaches. But the country has dropped 60 points over the past few testing cycles – equivalent to losing three full school years of education. I suspect we won’t be hearing calls to teach the Finnish way anymore. “You have to be careful because the leaders of today can be the laggards of tomorrow,” said Tom Loveless, an independent researcher who studies international assessments.

There was one bright spot for American students. Fifteen-year-olds scored comparatively well on the PISA reading test, with their scores dropping by just one point while other countries experienced much steeper declines. But that good news is also tempered by the most recent scores on the National Assessment for Educational Progress (NAEP) test, often called the Nation’s Report Card. Reading scores of fourth and eighth graders deteriorated over the last two testing cycles in 2019 and 2022.

Overall, the PISA results provide additional confirmation that U.S. students are in trouble, especially in math, and we can’t put all the blame on the pandemic.

This story about the 2022 PISA results was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: There is a worldwide problem in math and it’s not just about the pandemic appeared first on The Hechinger Report.

The typical ambitious high school student takes advanced algebra, trigonometry, pre-calculus and calculus. None of that math may be necessary for the vast majority of undergraduates who don’t intend to major in science or another STEM field. 

But those same students don’t have many of the math skills that professors think they actually do need. In a survey, humanities, arts and social science professors say they really want their students to be able to analyze data, create charts and spreadsheets and reason mathematically – skills that high school math courses often skip or rush through.

“We still need the traditional algebra-to-calculus curriculum for students who are intending a STEM major,” said Gary Martin, a professor of mathematics education at Auburn University in Alabama who led the team that conducted this survey of college professors. “But that’s maybe 20 percent. The other 80 percent, what about them?” 

Martin said that the survey showed that high schools should stress “reasoning and critical thinking skills, decrease the emphasis on specific mathematical topics, and increase the focus on data analysis and statistics.”

This damning assessment of the content of high school math comes from a survey of about 300 Alabama college professors who oversee majors and undergraduate degree programs at both two-year and four-year public colleges in the humanities, arts, social sciences and some natural sciences. Majors that require calculus were excluded. 

The 2021 survey prompted Alabama’s public colleges and universities to allow more students to meet their math requirements by taking a statistics course instead of a traditional math class, such as college algebra or calculus. 

Martin and his colleagues later realized that the survey had implications for high school math too, and presented these results at an Oct. 26, 2023 session of the National Council of Teachers of Mathematics annual conference in Washington D.C.  Full survey results are slated to be published in the winter 2024 issue of the MathAMATYC Educator, a peer-reviewed journal of the American Mathematical Association of Two-Year Colleges.

In the survey, professors were asked detailed questions about which mathematical concepts and skills students need in their programs. Many high school math topics were unimportant to college professors. For example, most professors said they wanted students to understand functions, particularly linear and exponential functions, which are used to model trends, population changes or compound interest. But Martin said that non-STEM students didn’t really need to learn trigonometric functions, which are used in satellite navigation or mechanical engineering. 

College professors were more keen on an assortment of what was described as mathematical “practices,” including the ability to “interpret quantitative information,” “strategically infer, evaluate and reason,” “apply the mathematics they know to solve everyday life, society and the workplace,” and to “look for patterns and relationships and make generalizations.”

“Teachers are so focused on covering all the topics that they don’t have time to do the practices when the practices are what really matters,” said Martin.

Understanding statistics was high on the list. An overwhelming majority of college professors said students in their programs needed to be familiar with statistics and data analysis, including concepts like correlation, causation and the importance of sample size. They wanted students to be able to “interpret displays of data and statistical analyses to understand the reasonableness of the claims being presented.” Professors say students need to be able to produce bar charts, histograms and line charts. Facility with spreadsheets, such as Excel, is useful too.

“Statistics is what you need,” said Martin. “Yet, in many K-12 classrooms, statistics is the proverbial end-of-the-year unit that you may or may not get to. And if you do, you rush through it, just to say you did it. But there’s not this sense of urgency to get through the statistics, as there is to get through the math topics.”

Though the survey took place only in Alabama and professors in other states might have different thoughts on the math that students need, Martin suspects that there are more similarities than differences.

The mismatch between what students learn in high school and what they need in college isn’t easy to fix. Teachers generally don’t have time for longer statistics units, or the ability to go deeper into math concepts so that students can develop their reasoning skills, because high school math courses have become bloated with too many topics. However, there is no consensus on which algebra topics to jettison.

Encouraging high school students to take statistics classes during their junior and senior years is also fraught. College admissions officers value calculus, almost as a proxy for intelligence. And college admissions tests tend to emphasize math skills that students will practice more on the algebra-to-calculus track. A diversion to data analysis risks putting students at a disadvantage. 

The thorniest problem is that revamping high school math could force students to make big choices in school before they know what they want to study in college. Students who want to enter STEM fields still need calculus and the country needs more people to pursue STEM careers. Taking more students off of the calculus track could close doors to many students and ultimately weaken the U.S. economy.

Martin said it’s also important to remember that vocational training is not the only purpose of math education.  “We don’t have students read Shakespeare because they need it to be effective in whatever they’re going to do later,” he said. “It adds something to your life. I felt that it really gave me breadth as a human being.”  He wants high school students to study some math concepts they will never need because there’s a beauty to them. “Appreciating mathematics is a really intriguing way of looking at the world,” he said.

Martin and his colleagues don’t have any definitive solutions, but their survey is a helpful data point in demonstrating how too few students are getting the mathematical foundations they need for the future. 

This story about high school math was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Professors say high school math doesn’t prepare most students for their college majors appeared first on The Hechinger Report.

Young students around the world struggle to memorize multiplication tables, but the effort pays off. Cognitive scientists say that learning 6 x 7 and 8 x 9 by heart frees up the brain’s working memory so that students can focus on the more demanding aspects of problem solving. 

Math teachers debate the best way to make multiplication automatic. Some educators argue against drills and say fluency will develop with everyday usage. Others insist that schools should devote time to helping children memorize times tables. 

Even among proponents of memorization, it’s unclear which methods are the most effective. Should kids draw their own color-coded tables and study them, or copy their multiplication facts out dozens of times? Should they play multiplication songs and videos? Should they learn mnemonic tricks, like how the digits of the multiples of nine add up to nine (1+8, 2+7, 3+6, etc.)?  My daughter’s gym teacher used to make students shout “7 x 5 is 35” and “6 x 8 is 48” as they did jumping jacks. (It was certainly a way to make jumping less monotonous.) 

To help advise teachers, a team of learning scientists compared two common methods: chanting and flashcards. 

The 2022 experiment took place in four second grade classrooms in the Netherlands. The teachers began by delivering a lesson on multiplying by three. Using the same scripted lesson, they explained multiplication concepts, such as: “If I grab three apples, and I do this only one time, how many apples do I have?” 

After the lesson, half the classrooms practiced by reciting equations displayed on a whiteboard:  “One times three is three, two times three is six…” through to 10. The other half practiced with flashcards. Students had their own personal sets with answers on the reverse side. Both groups spent five minutes practicing three times during the week for a total of 15 minutes. (More details on the experiment’s design here.)

When the teachers moved on to multiplication by fours, the groups switched. The chanters quizzed themselves with flashcards, and the flashcard kids started chanting. All the students practiced memorizing both ways. 

The results added up to a clear winner. 

On a pre-test before the lesson, the second graders got an average of three math facts right. Afterwards, the chanters tended to double their accuracy, answering six facts correctly. But the flashcard users averaged eight correct. Students were tested again a full week later without any additional practice sessions, and the strong advantage for flashcard users didn’t fade. It was a sign that flashcard practice not only produces better short-term memories, but also better long-term ones –  the ultimate goal.

The study, “The effect of retrieval practice on fluently retrieving multiplication facts in an authentic elementary school setting,” was published online in October 2023 in the journal Applied Cognitive Psychology.  Though a small study of 48 students, this classroom experiment is a good example of the power of what cognitive scientists call “spaced retrieval practice,” in which the act of remembering consolidates information and helps the brain form long-term memories.  

Retrieval practice can seem counterintuitive. One might think that students should study before being assessed or quizzing themselves. But there’s a growing body of evidence that trying to recall something is itself a powerful tool for learning, particularly when you are given the correct answer immediately after making a stab at it and then get a chance to try again. Testing your memory – even when you draw a blank – is a way to build new memories. 

Many experiments have shown that retrieval practice produces better long-term memories than studying. Flashcards are one way to try retrieval practice. Quizzes are another option because they also require students to retrieve new information from memory. Indeed, many teachers opt for speed drills, asking students to race through a page of multiplication problems in a minute. 

Flashcards can be less anxiety provoking, provide students immediate feedback with answers on the reverse side and allow students to repeat the retrieval practice immediately, running through the deck more than once. Still, kids are kids and they easily drift off task during independent practice time. With a timed quiz, the teacher can be more confident that everyone has benefited from a round of retrieval practice. I’d be curious to see flashcards and quizzes pitted against each other in a future classroom experiment. 

As charming as multiplication songs are – I have a soft spot for School House Rock and my editor fondly recalls her Billy Leach multiplication records – they are unlikely to be as effective as flashcards because they don’t involve retrieval practice, according to Gino Camp, a professor of learning sciences at Open University in the Netherlands and one of the researchers on the study.

That doesn’t mean we should jettison the songs or all the other memorization methods just because some aren’t as effective as others. Researchers may eventually find that a combination of techniques is even more powerful. Still, there are limited minutes in the school day, and knowing which learning methods are the most effective can help everyone – teachers, parents and students – use their time wisely.

This story about multiplication flashcards was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.

The post PROOF POINTS: Flashcards prevail over repetition in memorizing multiplication tables appeared first on The Hechinger Report.

Ever since schools reopened and resumed in-person instruction, districts have been trying to help students catch up from pandemic learning losses. The Biden Administration has urged schools to use tutoring. Many schools have purchased an online version that gives students 24/7 access to tutors. Typically, communication is through text chat, similar to communicating with customer service on a website. Students never see their tutors or hear their voices. 

Researchers estimate that billions have been spent on these online tutoring services, but so far, there’s no good evidence that they are helping many students catch up. And many students need extra help. According to the most recent test scores from spring 2023, 50 percent more students are below grade level than before the pandemic; even higher achieving students remain months behind where they should be.

Low uptake

The main problem is that on-demand tutoring relies on students to seek extra help. Very few do. Some school systems have reported usage rates below 2 percent. A 2022 study by researchers at Brown University of an effort to boost usage among 7,000 students at a California charter school network found that students who needed the most help were the least likely to try online tutoring and only a very small percentage of students used it regularly. Opt-in tutoring could “exacerbate inequalities rather than reduce them,” warned a  September 2023 research brief by Brown University’s Annenberg Center, Results for America, a nonprofit that promotes evidence-backed policies, the American Institutes for Research and NWEA, an assessment firm.

In January 2023, an independent research firm Mathematica released a more positive report on students’ math gains with an online tutoring service called UPchieve, which uses volunteers as tutors. It seemed to suggest that high school students could make extraordinary math progress from online homework help. 

UPchieve is a foundation-funded nonprofit with a slightly different model. Instead of schools buying the tutoring service from a commercial vendor, UPchieve makes its tutors freely available to any student in grades eight to 12 living in a low-income zip code or attending a low-income high school. Behind the scenes, foundations cover the cost to deliver the tutoring, about $5 per student served. (Those foundations include the Bill & Melinda Gates and the Overdeck Family foundations, which are also among the many funders of The Hechinger Report.)

UPchieve posted findings from the study in large font on its website: “Using UPchieve 9 times caused student test scores to meaningfully increase” by “9 percentile rank points.” If true, that would be equivalent to doubling the amount of math that a typical high school student learns. That would mean that students learned an extra 14 weeks worth of math from just a few extra hours of instruction. Not even the most highly regarded and expensive tutoring programs using professional tutors who are following clear lesson plans achieve this.

The study garnered a lot of attention on social media and flattering media coverage “for disrupting learning loss in low-income kids.” But how real was this progress? 

Gift card incentives

After I read the study, which was also commissioned by the Gates foundation, I immediately saw that UPchieve’s excerpts were taken out of context. This was not a straightforward randomized controlled trial, comparing what happens to students who were offered this tutoring with students who were not. Instead, it was a trial of the power of cash incentives and email reminders. 

For the experiment, Mathematica researchers had recruited high schoolers who were already logging into the UPchieve tutoring service. These were no ordinary ninth and 10th graders. They were motivated to seek extra help, resourceful enough to find this tutoring website on their own (it was not promoted through their schools) and liked math enough to take extra tests to participate in the study. One group was given extra payments of $5 a week for doing at least 10 minutes of math tutoring on UPchieve, and sent weekly email reminders. The other group wasn’t. Students in both groups received $100 for participating in the study.

The gift cards increased usage by 1.6 hours or five to six more sessions over the course of 14 weeks. These incentivized students “met” with a tutor for a total of nine sessions on average; the other students averaged fewer than four sessions. (As an aside, it’s unusual that cash incentives would double usage. Slicing the results another way, only 22 percent of the students in the gift-card group used UPchieve more than 10 times compared with 14 percent in the other group. That’s more typical.) 

At the end of 14 weeks, students took the Renaissance Star math test, an assessment taken by millions of students across the nation. But the researchers did not report those test scores. That’s because they were unlucky in their random assignment of students. By chance, comparatively weaker math students kept getting assigned to receive cash incentives. It wasn’t an apples-to-apples comparison between the two groups, a problem that can happen in a small randomized controlled trial. To compensate, the researchers statistically adjusted the final math scores to account for differences in baseline math achievement. It’s those statistically adjusted scores that showed such huge math gains for the students who had received the cash incentives and used the tutoring service more.

However, the huge 9 percentile point improvement in math was not statistically significant. There were so few students in the study – 89 in total – that the results could have been a fluke. You’d need a much larger sample size to be confident.

A caution from the researcher 

When I interviewed one of the Mathematica researchers, he was cautious about UPchieve and on-demand tutoring in general.  “This is an approach to tutoring that has promise for improving students’ math knowledge for a specific subset of students:  those who are likely to proactively take up an on-demand tutoring service,” said Greg Chojnacki, a co-author of the UPchieve study. “The study really doesn’t speak to how promising this model is for students who may face additional barriers to taking up tutoring.”

Chojnacki has been studying different versions of tutoring and he says that this on-demand version might prove to be beneficial for the “kid who may be jumping up for extra help the first chance they get,” while other children might first need to “build a trusting relationship” with a tutor they can see and talk to before they engage in learning. With UPchieve and other on-demand models, students are assigned to a different tutor at each session and don’t get a chance to build a relationship. 

Chojnacki also walked back the numerical results in our interview. He told me not to “put too much stock” in the exact amount of math that students learned. He said he’s confident that self-motivated students who use the tutoring service more often learned more math, but it could be “anywhere above zero” and not nearly as high as 9 percentile points – an extra three and a half months worth of math instruction.

UPchieve defends “magical” results

UPchieve’s founder, Aly Murray, told me that the Mathematica study results initially surprised her, too. “I agree they almost seem magical,” she said by email. While acknowledging that a larger study is needed to confirm the results, she said she believes that online tutoring without audio and video can “lead to greater learning” than in-person tutoring “when done right.”

“I personally believe that tutoring is most effective when the student is choosing to be there and has an acute need that they want to address (two things that are both uniquely true of on-demand tutoring),” she wrote. “Students have told us how helpful it is to get timely feedback and support in the exact moment that they get confused (which is often late at night in their homes while working on their homework). So in general, I believe that on-demand tutoring is more impactful than traditional high-dosage tutoring models on a per tutoring session or per hour of tutoring basis. This could be part of why we were able to achieve such outsized results despite the low number of sessions.”

Murray acknowledged that low usage remains a problem. At UPchieve’s partner schools, only 5 percent of students logged in at least once during the 2022-23 year, she told me. At some schools, usage rates fell below 1 percent. Her goal is to increase usage rates at partner schools to 36 percent. (Any low-income student in grades eight to 12 can use the tutoring service at no cost and their schools don’t pay UPchieve for the tutoring either, but some “partner” schools pay UPchieve to promote and monitor usage.) 

The downside to homework help

Helping students who are stuck on a homework assignment is certainly nice for motivated kids who love school, but relying on homework questions is a poor way to catch up students who are the most behind, according to many tutoring experts. 

“I have a hard time believing that students know enough about what they don’t know,” said Susanna Loeb, a Stanford University economist who founded the National Student Support Accelerator, which aims to bring evidence-based tutoring to more students. 

For students who are behind grade level, homework questions often don’t address their gaps in basic math foundations. “Maybe underneath, they’re struggling with percentages, but they’re bringing an algebra question,” said Loeb. “If you just bring the work of the classroom to the tutor, it doesn’t help students very much.” 

Pre-pandemic research of once-a-week after-school homework help also produced disappointing results for struggling students. Effective tutoring starts with an assessment of students’ gaps, Loeb said, followed by consistent, structured lessons.

Schools struggle to offer tutors for all students

With so little evidence, why are schools buying on-demand online tutoring? Pittsburgh superintendent Wayne Walters said he was unable to arrange for in-person tutoring in all of his 54 schools and wanted to give each of his 19,000 students access to something. He signed a contract with Tutor.com for unlimited online text-chat tutoring in 2023-24. 

“I’m going forward with it because it’s available,” Walters said. “If I don’t have something to provide, or even offer, then that limits opportunity and access. If there’s no access, then I can’t even push the needle to address the most marginalized and the most vulnerable.”

Walters hopes to make on-demand tutoring “sexy” and appealing to high schoolers accustomed to texting. But online tutoring is not the same as spontaneous texting between friends. One-minute delays in tutors’ replies to questions can test students’ patience. 

On-demand tutoring can appear to be an economical option. Pittsburgh is able to offer this kind of tutoring, which includes college admissions test prep for high schoolers, to all 19,000 of its students for $600,000. Providing 400 students with a high-dosage tutoring program – the kind that researchers recommend – could cost $1.5 million. There are thousands of Pittsburgh students who are significantly behind grade level. It doesn’t seem fair to deliver high-quality in-person tutoring to only a lucky few.  

However, once you factor in actual usage, the economics of on-demand tutoring looks less impressive. In Fairfax County, Va., for example, only 1.6 percent of students used Tutor.com. If Pittsburgh doesn’t surpass that rate, then no more than 300 of its students will be served.

There are no villains here. School leaders are trying to do the best they can and be fair to everyone. Hopes are raised when research suggests that online on-demand tutoring can work if they can succeed in marketing to students. But they should be skeptical of studies that promise easy solutions before investing precious resources. That money could be better spent on small-group tutoring that dozens of studies show is more effective for students.

This story about drop-in tutoring was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Proof Points and other Hechinger newsletters.

The post PROOF POINTS: Schools keep buying online drop-in tutoring. The research doesn’t support it appeared first on The Hechinger Report.

Amid the growing debate over how best to teach math, there is another ballooning problem: grades. They’re becoming increasingly untethered to how much students know. That not only makes it harder to gauge how well students are learning math and catching up from pandemic learning losses, but it’s also making math grades a less reliable indicator of who should be admitted to colleges or take advanced courses.

The latest warning sign comes from college admissions test maker ACT, which compared students’ ACT test scores with their self-reported high school grades between 2010 and 2022. Grade inflation struck all high school subjects, ACT found, but it was highest for math, followed by science, English, and social studies.

Grade inflation accelerated after 2016 and intensified during the pandemic, as schools relaxed standards. But as schools settled back into their usual rhythms in 2021-22, grades didn’t fall back to pre-pandemic norms and remained elevated. Grades continued to rise in math and science even as grade inflation stabilized in English and social studies. For a given score on the math section of the ACT, students said they had earned higher grades than students had reported in previous years.

Edgar Sanchez, an ACT researcher who conducted the analysis, said the inflation makes it hard to interpret high school grades, especially now that A grades are the norm. “Does 4.0 really mean complete content mastery or not?” Sanchez asked, referring to an A grade on the 0 to 4 grade-point scale.

Grade inflation is a big trend across the country.  “It’s not just happening in some classrooms or with some teachers, it’s happening across the system,” said Sanchez. “What is happening in the system that is pushing this trend?”

Grades represent more than just content mastery. Many teachers factor in attendance, participation and effort in calculating a final grade. It’s possible that even math teachers are weighing soft skills more heavily with the increasing popularity of social-emotional learning. Or, perhaps high schools have watered down the content in math courses and students are genuinely mastering easier material.

A’s on the rise

Sanchez speculates that test optional admissions have elevated the importance of high school grades. He encouraged journalists and other researchers to look into the increased pressures on high school teachers of math and science courses, which Sanchez described as ”pivotal” for getting into competitive STEM college programs.  

Sanchez said he shared his grade inflation findings with college administrators, who told him that incoming STEM students are not as prepared as students in previous years. (The Hechinger Report has also found that college students are struggling with basic math.) But college professors didn’t report a similar academic deterioration with their humanities students. “That was an interesting confirmation of these findings,” Sanchez said.

ACT isn’t an unbiased research organization. The nonprofit sells tests and it has been advocating for colleges to re-establish exam requirements. However, neutral observers have also found strong evidence of high school grade inflation. The U.S. Department of Education documented rising grades on high school transcripts between 2009 and 2019, while 12th grade math scores fell on the National Assessment of Educational Progress (NAEP). The National Center for Education Statistics plans to update this transcript study in 2024. 

The ACT analysis, published in August 2023, covered almost 6.9 million high school seniors who took the ACT between 2010-22. They attended over 3,800 different public schools. It was a follow up to a 2022 report, which also detected grade inflation through 2021. This 2023 update looked at grade inflation by subject and added 2022.

Sanchez calculated that average math grades, adjusted for student and school characteristics, increased 0.30 grade points from 3.02 in 2010 to 3.32 in 2022. This translates to a movement from  “B” average to above a “B+” average in a decade. During this same time period, science grades increased by 0.24 points, while English and social studies rose by 0.22 points and 0.18 points, respectively. (The analysis excluded bonus points that some high schools award for Advanced Placement and other courses. A 4.0 was the maximum grade.)

Measuring grade inflation: Grades rise as ACT test scores fall

Grades are rising against a backdrop of declining achievement. English, math, reading and scientific reasoning ACT scores fell slightly between 2010-22. The sharpest declines were in math, in which the average ACT score dropped from 21.4 to 20.2. Three quarters of this math deterioration has taken place since 2020. 

Grade inflation may indeed be an unintended consequence of a well-intended policy to de-emphasize testing. More than 1,800 colleges have adopted test-optional or test-blind admissions. That’s increased the importance of grades. The losers here are students who still need to understand math – no matter what their grade.

This story about grade inflation in high school was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Proof Points and other Hechinger newsletters.

The post PROOF POINTS: It’s easier and easier to get an A in math appeared first on The Hechinger Report.

In theory, education technology could redesign school from a factory-like assembly line to an individualized experience. Computers, powered by algorithms and AI, could deliver custom-tailored lessons for each child. Advocates call this concept “personalized learning” but this sci-fi idyll (or dystopia, depending on your point of view) has been slow to catch on in American classrooms.

Meanwhile one piece of ed tech, called ASSISTments, takes the opposite approach. Instead of personalizing instruction, this homework website for middle schoolers encourages teachers to assign the exact same set of math problems to the entire class. One size fits all. 

Unlike other popular math practice sites, such as Khan Academy, IXL or ALEKS, in which a computer controls the content, ASSISTments keeps the control levers with the teachers, who pick the questions they like from a library of 200,000. Many teachers assign the same familiar homework questions from textbooks and curricula they are already using.

And this deceptively simple – and free –  tool has built an impressive evidence base and a following among middle school math teachers. Roughly 3,000 teachers and 130,000 students were using it during the 2022-23 school year, according to the husband and wife team of Neil and Cristina Heffernan who run ASSISTments, a nonprofit based at Worcester Polytechnic Institute in Massachusetts, where Neil is a computer science professor.

After Neil built the platform in 2003, several early studies showed promising results, and then a large randomized control trial (RCT) in Maine, published in 2016, confirmed them. For 1,600 seventh-grade students whose classrooms were randomly selected to use ASSISTments for math homework, math achievement was significantly higher at the end of the year, equivalent to an extra three quarters of a year of schooling, according to one estimate. Both groups – treatment and control – were otherwise using the same textbooks and curriculum. 

On the strength of those results, an MIT research organization singled out ASSISTments as one of the rare ed tech tools proven to help students. The Department of Education’s What Works Clearinghouse, which reviews education evidence, said the research behind ASSISTments was so strong that it received the highest stamp of approval: “without reservations.”

Still, Maine is an unusual state with a population that is more than 90 percent white and so small that everyone could fit inside the city limits of San Diego. It had distributed laptops to every middle school student years before the ASSISTments experiment. Would an online math platform work in conditions where computer access is uneven? 

The Department of Education commissioned a $3 million replication study in North Carolina, in which 3,000 seventh graders were randomly assigned to use ASSISTments. The study, set to test how well the students learned math in spring of 2020, was derailed by the pandemic. But a private foundation salvaged it. Before the pandemic, Arnold Ventures had agreed to fund an additional year of the North Carolina study, to see if students would continue to be better at math in eighth grade. (Arnold Ventures is among the many funders of The Hechinger Report.)

Those longer-term results were published in June 2023, and they were good.  Even a year later, on year-end eighth grade math tests, the 3,000 students who had used ASSISTments in seventh grade outperformed 3,000 peers who hadn’t. The eighth graders had moved on to new math topics and were no longer using ASSISTments, but their practice time on the platform a year earlier was still generating dividends. 

Researchers found that the lingering effect of practicing math on ASSISTments was similar in size to the long-term benefits of Saga Education’s intensive, in-person tutoring, which costs $3,200 to $4,800 per year for each student. The cost of ASSISTments is a tiny fraction of that, less than $100 per student. (That cost is covered by private foundations and federal grants. Schools use it free of charge.)

Another surprising result is that students, on average, benefited from solving the same problems, without assigning easier ones to weaker students and harder ones to stronger students. 

How is it that this rather simple piece of software is succeeding while more sophisticated ed tech has often shown mixed results and failed to gain traction?

The studies aren’t able to explain that exactly. ASSISTments, criticized for its “bland” design and for sometimes being “frustrating,” doesn’t appear to be luring kids to do enormous amounts of homework. In North Carolina, students typically used it for only 18 minutes a week, usually split among two to three sessions. 

From a student’s perspective, the main feature is instant feedback. ASSISTments marks each problem immediately, like a robo grader. A green check appears for getting it right on the first try, and an orange check is for solving it on a subsequent attempt. Students can try as many times as they wish. Students can also just ask for the correct answer. 

Nearly every online math platform gives instant feedback. It’s a well established principle of cognitive science that students learn better when they can see and sort out their mistakes immediately, rather than waiting days for the teacher to grade their work and return it. 

The secret sauce might be in the easy-to-digest feedback that teachers are getting. Teachers receive a simple data report, showing them which problems students are getting right and wrong. 

ASSISTments encourages teachers to project anonymized homework results on a whiteboard and review the ones that many students got wrong. Not every teacher does that. On the teacher’s back end, the system also highlights common mistakes that students are making. In surveys, teachers said it changes how they review homework.

Other math platforms generate data reports too, and teachers ought to be able to use them to inform their instruction. But when 30 students are each working on 20 different, customized problems, it’s a lot harder to figure out which of those 600 problems should be reviewed in class. 

There are other advantages to having a class work on a common set of problems. It allows kids to work together, something that motivates many extroverted tweens and teens to do their homework. It can also trigger worthwhile class discussions, in which students explain how they solved the same problem differently.

ASSISTments has drawbacks. Many students don’t have good internet connections at home and many teachers don’t want to devote precious minutes of class time to screen time. In the North Carolina study, some teachers had students do the homework in school. 

Teachers are restricted to the math problems that Heffernan’s team has uploaded to the ASSISTments library. It currently includes problems from three middle school math curricula:  Illustrative Mathematics, Open Up Resources and Eureka Math (also known as EngageNY). For the Maine and North Carolina studies, the ASSISTments team uploaded math questions that teachers were familiar with from their textbooks and binders. But outside of a study, if teachers want to use their own math questions, they’ll have to wait until next year, when ASSISTments plans to allow teachers to build their own problems or edit existing ones.

Teachers can assign longer open-response questions, but ASSISTments doesn’t give instant feedback on them. Heffernan is currently testing how to use AI to evaluate students’ written explanations. 

There are other bells and whistles inside the ASSISTments system too. Many problems have “hints” to help students who are struggling and can show step-by-step worked out examples. There are also optional “skill builders” for students to practice rudimentary skills, such as adding fractions with unlike denominators.  It is unclear how important these extra features are. In the North Carolina study, students generally didn’t use them.

There’s every reason to believe that students can learn more from personalized instruction, but the research is mixed. Many students don’t spend as much practice time on the software as they should. Many teachers want more control over what the computer assigns to students. Researchers are starting to see good results in using differentiated practice work in combination with tutoring. That could make catching up a lot more cost effective.

I rarely hear about “personalized learning” any more in a classroom context. One thing we’ve all learned during the pandemic is that learning has proven to be a profoundly human interaction of give and take between student and teacher and among peers. One-size-fits-all instruction may not be perfect, but it keeps the humans in the picture. 

This story about ASSISTments was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Proof Points and other Hechinger newsletters.

The post PROOF POINTS: The value of one-size-fits-all math homework appeared first on The Hechinger Report.

This story was produced by The Associated Press as part of a series, The Math Problem, from the Education Reporting Collaborative, a coalition of eight  newsrooms that is documenting the math crisis facing schools and highlighting progress. Members of the collaborative are AL.com, The Associated Press, The Christian Science Monitor, The Dallas Morning News, The Hechinger Report, Idaho Education News, The Post and Courier in South Carolina, and The Seattle Times.  

FAIRFAX, Va. – Diego Fonseca looked at the computer and took a breath. It was his final attempt at the math placement test for his first year of college. His first three tries put him in pre-calculus, a blow for a student who aced honors physics and computer science in high school. 

Functions and trigonometry came easily, but the basics gave him trouble. He struggled to understand algebra, a subject he studied only during a year of remote learning in high school. 

“I didn’t have a hands-on, in-person class, and the information wasn’t really there,” said Fonseca, 19, of Ashburn, Virginia, a computer science major who hoped to get into calculus. “I really struggled when it came to higher-level algebra because I just didn’t know anything.” 

Fonseca is among 100 students who opted to spend a week of summer break at George Mason University brushing up on math lessons that didn’t stick during pandemic schooling. The northern Virginia school started Math Boot Camp because of alarming numbers of students arriving with gaps in their math skills. 

Colleges across the country are grappling with the same problem as academic setbacks from the pandemic follow students to campus. At many universities, engineering and biology majors are struggling to grasp fractions and exponents. More students are being placed into pre-college math, starting a semester or more behind for their majors, even if they get credit for the lower-level classes.  

Colleges largely blame the disruptions of the pandemic, which had an outsize impact on math. Reading scores on the national test known as NAEP plummeted, but math scores fell further, by margins not seen in decades of testing. Other studies find that recovery has been slow. 

Related: How can schools dig out of a generation’s worth of lost math progress?  

At George Mason, fewer students are getting into calculus — the first college-level course for some majors — and more are failing. Students who fall behind often disengage, disappearing from class.  

“This is a huge issue,” said Maria Emelianenko, chair of George Mason’s math department. “We’re talking about college-level pre-calculus and calculus classes, and students cannot even add one-half and one-third.” 

For Jessica Babcock, a Temple University math professor, the magnitude of the problem hit home last year as she graded quizzes in her intermediate algebra class, the lowest option for STEM majors. The quiz, a softball at the start of the fall semester, asked students to subtract eight from negative six. 

It’s not just that they’re unprepared, they’re almost damaged. I hate to use that term, but they’re so behind.”

Brian Rider, math chair, Temple University 

“I graded a whole bunch of papers in a row. No two papers had the same answer, and none of them were correct,” she said. “It was a striking moment of, like, wow — this is significant and deep.” 

Before the pandemic, about 800 students per semester were placed into that class, the equivalent of ninth grade math. By 2021, it swelled to nearly 1,400. 

“It’s not just that they’re unprepared, they’re almost damaged,” said Brian Rider, Temple’s math chair. “I hate to use that term, but they’re so behind.” 

Researchers say math learning suffered for various reasons. An intensely hands-on subject, math was hard to translate to virtual classrooms. When students fell behind in areas like algebra, gaps could go unnoticed for a year or more as they moved to subjects such as geometry or trigonometry. And at home, parents are generally more comfortable helping with reading than math. 

As with other learning setbacks, math issues are most pronounced among Black, Latino, low-income and other vulnerable students, said Katharine Strunk, who led a study on learning delays in Michigan and is now dean of the graduate school of education at the University of Pennsylvania. 

“Those are the students who were most impacted by the pandemic, and they’re the ones who are going to suffer the longer-term consequences,” she said. “They’re not going to have the same access.” 

Related: How can math education in America be improved? Help us count the ways 

Colleges say there’s no quick fix. Many are trying to identify gaps sooner, adopting placement tests that delve deeper into math skills. Some are adding summer camps like George Mason’s, which helped participants increase placement test scores by 56 percent on average.* 

In lieu of traditional remedial classes, which some research finds to be ineffective, more schools are offering “corequisite” classes that help students shore up on the basics while also taking higher courses like calculus.  

Penn State tackled the problem by expanding peer tutoring. Professors report that students who participate have scored 20 percent higher on exams, said Tracy Langkilde, dean of Penn State’s College of Science. 

What’s becoming a persistent problem at some colleges has been a blip for others. At Iowa State University, known for its engineering program, students entering in 2020 were far more likely to be placed in lower-level math classes, and grades fell. That group of students has had continued trouble, but numbers improved for the following year’s class, said Eric Weber, math department chair. 

At Temple, there’s been no rebound. Professors tried small changes: expanded office hours, a new tutoring center, pared-down lessons focused on the essentials. 

But students didn’t come for help, and they kept getting D’s and F’s. This year, Babcock is redesigning the algebra course. Instead of a traditional lecture, it’ll focus on active learning, an approach that demands more participation and expands students’ role in the learning process. Class will be more of a group discussion, with lots of problems worked in-class. 

“We really want students to feel like they’re part of their learning,” Babcock said. “We can’t change their preparation coming in, but we can work to meet their needs in the best way possible.” 

Related: After the pandemic disrupted their high school educations, students are arriving at college unprepared  

George Mason also is emphasizing active learning. Its new placement test helps students find gaps and fill them in before taking it again, with up to four attempts. During the school year, students struggling in math can switch to slower-paced versions that take two terms instead of one. 

At math camp, Fonseca felt he was making up ground. He studied hard, even doing practice problems on the train ride to camp. But when he got to the placement test’s algebra portion, he made the same mistakes. His final score again placed him in pre-calculus. 

The setback would have meant spending at least one extra semester catching up on math at George Mason. In the end, Fonseca decided to start at Northern Virginia Community College. After two years, he plans to transfer to one of Virginia’s public four-year universities. 

A couple weeks after camp, Fonseca again found himself taking a placement test, this time for the community college.  

“I managed to use the knowledge of the boot camp, and I got into calculus,” he said. “I didn’t have any expectation I’d do that.” 

Update: A figure in this sentence on George Mason’s summer camp has been updated.

This story was produced by The Associated Press as part of a series, The Math Problem, from the Education Reporting Collaborative, a coalition of eight  newsrooms that is documenting the math crisis facing schools and highlighting progress. Members of the collaborative are AL.com, The Associated Press, The Christian Science Monitor, The Dallas Morning News, The Hechinger Report, Idaho Education News, The Post and Courier in South Carolina, and The Seattle Times.  

The Associated Press education team receives support from the Carnegie Corporation of New York. The AP is solely responsible for all content. 

The post College students are still struggling with basic math. Professors blame the pandemic  appeared first on The Hechinger Report.

Kids around the country are still suffering academically from the pandemic. But more than three years after schools shut down, it’s hard to understand exactly how much ground students have lost and which children now need the most attention.

Three new reports offer some insights.  All three were produced by for-profit companies that sell assessments to schools. Unlike annual state tests, these interim assessments are administered at least twice a year and help track student progress, or learning, during the year. These companies may have a business motive in sounding an alarm to sell more of their product, but the reports are produced by well-regarded education statisticians.

The big picture is that kids at every grade are still behind where they would have been without the pandemic. All three reports look at student achievement in the spring of 2019, before the pandemic, and compare it to the spring of 2023. A typical sixth grader, for example, in the spring of 2023 was generally scoring much lower than a typical sixth grader in 2019.

The differences are in the details. One report says that students are still behind the equivalent of four to five months of school, but another says it’s one to three months. A third doesn’t measure months of lost learning, but notices the alarming 50 percent increase in the number of students who are still performing significantly below grade level.

Depending on how you slice and dice the data, older students in middle school and beyond seem to be in the most precarious position and younger children seem to be more resilient and recovering better. Yet, under a different spotlight, you can see troubling signs even among younger children. This includes the very youngest children who weren’t school age when the pandemic hit.

The most recent data, released on Aug. 28, 2023, is from Curriculum Associates, which sells i-Ready assessments taken by more than 11 million students across the country and focuses on “grade-level” skills.*  It counts the number of students in third grade, for example, who are able to read at a third-grade level or solve math problems that a third grader ought to be able to solve. The standards for what is grade-level achievement are similar to what most states consider to be “proficient” on their annual assessments.

The report concludes that the percentage of students who met grade-level expectations was “flat” over the past school year. This is one way of noting that there wasn’t much of an academic recovery between spring of 2022 and spring of 2023. Students of every age, on average, lagged behind where students had been in 2019.

For example, 69 percent of fourth graders were demonstrating grade-level skills in math in 2019. That dropped to 55 percent in 2022 and barely improved to 56 percent in 2023. (The drop in grade-level performance isn’t as dramatic for seventh and eighth graders, in part, because so few students were meeting grade-level expectations even before the pandemic.)

“It’s dang hard to catch up,” said Kristen Huff, vice president of assessment and research at Curriculum Associates.

To make up for lost ground, students would have to learn more in a year than they typically do. That generally didn’t happen. Huff said this kind of extra learning is especially hard for students who missed foundational math and reading skills during the pandemic.

While most students learned at a typical pace during the 2022-23 school year, Curriculum Associates noted a starkly different and troubling pattern for children who are significantly below grade level by two or more years. Their numbers spiked during the pandemic and have not gone down. Even worse, these children learned less during the 2022-23 school year than during a typical pre-pandemic year. That means they are continuing to lose ground.

Huff highlighted three groups of children who need extra attention: poor readers in second, third and fourth grades; children in kindergarten and first grade, and middle school math students.

There’s been a stubborn 50 percent increase in the number of third and fourth graders who are two or more grade levels behind in reading, Huff said. For example, 19 percent of third graders were that far behind grade level in 2023, up from 12 percent in 2019.  “I find this alarming news,” said Huff, noting that these children were in kindergarten and first grade when the pandemic first hit. “They’re missing out on phonics and phonemic awareness and now they’re thrust into grades three and four,” she said. “If you’re two or more grade levels below in grade three, you’re in big trouble. You’re in big, big, big trouble. We’re going to be seeing evidence of this for years to come.”

The youngest students, who were just two to four years old at the start of the pandemic, are also behind. Huff said that kindergarteners and first graders started the 2022-23 school year at lower achievement levels than in the past. They may have missed out on social interactions and pre-school. “You can’t say my current kindergartener wasn’t in school during the pandemic so they weren’t affected,” said Huff.

Math achievement slipped the most after schools shuttered and switched to remote learning. And now very high percentages of middle schoolers are below grade level in the subject. Huff speculates that they missed out on foundational math skills, especially fractions and proportional reasoning.

Renaissance administered its Star tests to more than six million students around the country. Its spring 2023 report was released on Aug, 9. Like Curriculum Associates, Renaissance finds that, “growth is back, but performance is not,” according to Gene Kerns, Renaissance’s chief academic officer.** That means students are generally learning at a typical pace at school, but not making up for lost ground. Depending on the subject and the grade, students still need to recover between one and three months of instruction.

Math is rebounding better than reading. “Math went down an alarming amount, but has started to go back up,” Kerns said. “We’ve not seen much rebound to reading.” Reading achievement, however, wasn’t as harmed by school disruptions. 

Kerns generally sees a sunnier story for younger children and a more troubling picture for older students.

The youngest children in kindergarten and first grade are on par with pre-pandemic history, he said. Middle elementary school grades are a little behind but catching up. 

“The older the student, the more lingering the impact,” said Kerns. “The high school data is very alarming. If you’re a junior in high school, you only have one more year. There’s a time clock on this.” 

Seventh and eighth graders showed tiny decreases in annual learning in math and reading. Kerns says he’s “hesitant” to call it a “downward spiral.”

The third report come from NWEA, which administers the Measures of Academic Progress (MAP) Assessment to more than 6 million students. Its spring 2023 data, released on July 11, showed that students on average need four to five months of extra schooling, on top of the regular school year, to catch up. This graph below, is a good summary of how much students are behind as expressed in months of learning.

Spring 2023 achievement gaps and months of schooling required to catch up to pre-COVID achievement levels

Like the Renaissance report, the NWEA report shows a bigger learning loss in math than in reading, and indicates that older students have been more academically harmed by the pandemic. They’ll need more months of extra schooling to catch up to where they would have been had the pandemic never happened. It could take years and years to squeeze these extra months of instruction in and many students may never receive them.

From my perspective, Renaissance and NWEA came to similar conclusions for most students. The main difference is that Renaissance has additional assessment data for younger children in kindergarten through second grade, showing a recovery, and high school data, showing a worse deterioration. The discrepancies in their measurement of months of learning loss, whether it’s four to five months or one to three months, is inconsequential. Both companies admit these assumption-filled estimates are imprecise.

One of the most substantial differences among the reports is that Curriculum Associates is sounding an alarm bell for kindergarteners and first graders while Renaissance is not.

The three reports all conclude that kids are behind where they would have been without the pandemic. But some sub-groups are doing much worse than others. The students who are the most behind and continuing to spiral downward really need our attention. Without extra support, their pandemic slump could be lifelong. 

* Correction: An earlier version of this story incorrectly said that more than 3 million students took i-Ready assessments.

** Correction: An earlier version of this story incorrectly spelled Gene Kerns’s last name.

This story about pandemic recovery was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Proof Points and other Hechinger newsletters. 

The post PROOF POINTS: Three views of pandemic learning loss and recovery appeared first on The Hechinger Report.