Sunday, November 15, 2026

 

New research could lead to faster, trustworthy AI models for climate prediction



MIT scientists developed a training method to generate models that simulate the effects of human activities on climate much more quickly than other approaches.




Massachusetts Institute of Technology





Accurate, up-to-date climate projections are essential for planning decisions that enhance the resilience of power grids, protect public health, maximize agricultural yields, and more.

Now, MIT researchers have developed a technique that streamlines the production of fast, trustworthy, and adaptable artificial intelligence models for climate prediction.

These specialized AI models, known as climate emulators, can rapidly simulate the effects of human activities onto a future climate, predicting impacts on temperatures or sea level, for example. 

Because emulators run in a matter of seconds, scientists use them to efficiently test thousands of varied climate scenarios. This provides more reliable estimates and can identify unexpected outcomes of regional interventions.

The MIT researchers developed a training technique that boosts the performance and adaptability of a climate emulator while using only a fraction of the data typically required. This significantly cuts production time for these practical scientific tools.

They show that an emulator trained with a much smaller amount of carefully selected synthetic data accurately predicts outcomes of scenarios it did not see during training, while traditional emulators failed to adapt to new situations.

“Climate emulators are especially well-suited for modeling policy-relevant scenarios. By using our method to generate optimized training data for an emulator, we can speed up the production of much more reliable emulators, so we can better trust their outputs for making policy decisions,” says Christoher Womack PhD ’26, who is now a postdoc at ETH Zurich and lead author of a paper on this technique.

He is joined on the paper by Shahine Bouabid, a postdoc in the MIT Department of Earth, Atmospheric and Planetary Sciences (EAPS); Andrei Sokolov, a research scientist in MIT’s Center for Sustainability Science and Strategy (CS3); Popat Salunke, a postdoc in CS3; Glenn Flierl, a professor of oceanography in EAPS; Sebastian D. Eastham, an associate professor at Imperial College London; and senior author Noelle E. Selin, a professor in the MIT Institute for Data, Systems, and Society (IDSS) and EAPS, and director of CS3. The research appears this week in the Proceedings of the National Academy of Sciences. 

Speedier surrogates

Climate scientists typically use earth system models (ESMs) to make long-term projections about how global temperature or precipitation could change in response to varying greenhouse gas emissions.

These enormous, physics-based models capture the major components and interactions of the earth’s climate in extreme detail. While powerful, ESMs can take months to run on a cluster of supercomputers — and that’s only to project one possible future climate scenario. 

Selin and her collaborators design climate emulators, which are fast AI surrogate models that mimic the behavior of full-scale ESMs. Emulators help scientists test many scenarios, make more accurate forecasts, and explore specific pollutants and regional effects.

“Climate emulators can run in just a few seconds, so it really expands what we can do in terms of exploring these future climate scenarios,” Womack says.

When scientists design a climate emulator, they often train the AI model using a set of six to eight future scenarios based on realistic future trajectories of emissions. They must run each scenario through a full-scale ESM, making these datasets costly and time-consuming to generate.  

The MIT researchers recently found that using realistic future scenarios as training data can lead to underperforming emulators that fail to predict unfamiliar or novel scenarios. 

In this work, they developed an algorithm that generates a synthetic dataset of optimized climate scenarios tailored to boost one emulator’s predictive performance.

“Our algorithm might generate some weird scenarios which look nothing like a realistic future, but we don’t care so much if the training scenarios are realistic. We are just trying to make the best possible dataset to train an emulator,” Womack says.

Importantly, rather than needing to run a full ESM six or eight times to generate training data, a user would only need to run that model once.

Tuning the training data 

The researchers begin by generating an arbitrary climate scenario using a fast and simple climate model. They test how well their emulator can reproduce the results of that ESM, which gives them a measure of error.

Because of the programming technique they used to build the training pipeline, they can iteratively update the training data to increase the emulator’s predictive skill. For instance, the algorithm could increase CO2 emissions dramatically in the year 2050 or rapidly decrease methane emissions in 2037.

Then the optimization updates the training data until the emulator’s error level reaches a certain threshold.

In the end, rather than resembling realistic scenarios in which emissions follow smooth trajectories, the synthetic data contain rapid variations.

“By introducing these high-frequency wiggles and oscillations, our data enable the emulator to extract a lot more information about the overall system,” Womack explains.

Their training algorithm outputs these emissions scenarios, which they can use to train an emulator that is more accurate and adaptable.

“In the future, we are definitely going to see climates and scenarios we haven’t seen before. If we can simulate these different scenarios more robustly, we can provide more reliable information to the stakeholders who need it,” Selin says.

For instance, an emulator trained with their technique can capture the effects of local pollutant control policies that reduce one pollutant while other emissions continue rising. The traditional emulators they evaluated failed this test.  

In the long run, Womack hopes this work improves the usefulness of fast and adaptable climate emulators for policy formulation. 

“This is such an important piece of the pipeline when it comes to creating useful tools that speak to stakeholder needs. Because the process of training an emulator can takes months or even years, the emulators we have are usually out-of-date. By cutting down that production time, I think this technique will be a game-changer in terms of getting updated tools into the hands of researchers and practitioners,” Selin adds.

In the future, the researchers want to test their technique using an emissions scenario generated by a full ESM. They also want to apply it to other types of climate models and explore its effectiveness in broader machine learning and computer vision applications.

This research was part of the Bringing Computation to the Climate Challenge (BC3) project and supported by Schmidt Sciences through the MIT Grand Challenges.

###

Written by Adam Zewe, MIT News

Paper: "Optimal scenario design for climate emulation"

D.E.I.

Three entrepreneurs from the UC3M Science Park recognized as STEAM Role Models by APTE





Universidad Carlos III de Madrid
Pictured, from left to right: Fateme Baneshi, Isabel Espejo Díaz, and Marta Lacuna Mohedano.

image: 

Pictured, from left to right: Fateme Baneshi, Isabel Espejo Díaz, and Marta Lacuna Mohedano.

view more 

Credit: UC3M



Three entrepreneurs linked to the Center for Innovation in Entrepreneurship and Artificial Intelligence (C3N-IA) at the Universidad Carlos III de Madrid (UC3M) Science Park—Fateme Baneshi, Isabel Espejo Díaz, and Marta Lacuna Mohedano—have been selected for the second edition of the list of 90 female STEAM role models published by the Association of Science and Technology Parks of Spain (APTE).

This recognition highlights the contributions of female researchers and tech company founders who are transforming the socioeconomic landscape through R&D+i, while also aiming to inspire scientific and technical vocations among secondary school girls.

The list is part of the ninth edition of the "Science and Technology in Female" (Ciencia y Tecnología en femenino) program, an initiative promoted by APTE alongside 17 of its member parks. The program combines visibility exhibitions, technological challenge workshops, and outreach sessions aimed at educational centers. In this edition, the UC3M Science Park ecosystem—located in Leganés Tecnológico—reinforces its position as a hub attracting female talent in complex fields such as artificial intelligence, biotechnology, and engineering.

From the management of the UC3M Entrepreneurship and Innovation Support Service, Ana Acebrón emphasizes the need for role models that young women can look up to: "Giving visibility to the work of our female scientists is one of the most effective tools for breaking down gender gaps in engineering degrees and fostering inclusive leadership right from the classroom."

Three Young STEAM Role Models

Among the professionals recognized in this new list are Fateme Baneshi, Isabel Espejo Díaz, and Marta Lacuna Mohedano, whose career paths exemplify entrepreneurial drive within STEAM disciplines (science, technology, engineering, arts, and mathematics). Their projects range from advanced software solutions to the development of business models based on high-value-added scientific knowledge.

Fateme Baneshi (Iran, 1995) holds a PhD in Aerospace Engineering from UC3M, with a doctoral thesis focused on using multi-agent reinforcement learning to optimize aircraft trajectories and mitigate their climate impact. She also holds undergraduate and master's degrees in control engineering, as well as research experience at EPFL and UC3M. She is currently CTO of AI Methods, where she leads innovative projects such as VARIANT, HARMONY-MRV, ISSR-Aware, Contrail Aware, and KAIROS, applying artificial intelligence to aeronautical meteorology and climate intelligence. Her distinguished career includes seven scientific publications, eight international conference papers, and prestigious awards such as the 2023 Luis Azcárraga Aeronautical Innovation Award.

Isabel Espejo Díaz (Seville, 1994) holds a PhD in Biomedicine from Universitat Pompeu Fabra and is a biotechnologist specializing in genetics and bioinformatics. She currently heads the Scientific Product Department at Altum Sequencing and is an associate professor at Universidad CEU San Pablo. She has developed her career across the CRG, Institut Curie, and CNIO, working on cancer genomics, epigenetics, next-generation sequencing, and molecular diagnostics. She has participated in over 12 scientific publications and presentations, is the inventor of a pending patent, and has contributed to the development of diagnostic solutions. At Altum, she leads the scientific strategy for liquid biopsy technologies and minimal residual disease monitoring in oncology.

Marta Lacuna Mohedano (Madrid, 2002) is a Research Success Specialist at 60Nd, a UC3M spin-off, where she participates in the development, validation, and commercialization of NeoMag, a platform designed to accelerate the integration of mechanobiology into preclinical phases. She holds a degree in Biotechnology from Imperial College London and a master's degree in Synthetic and Systems Biology. During her academic career, she worked at the Severo Ochoa Molecular Biology Center (CBMSO), contributing to projects that resulted in three scientific publications. Her work combines research, innovation, and technology transfer to bring new tools closer to the scientific community.

 

Brain-inspired memristors open new pathways to energy-efficient artificial intelligence





Shanghai Jiao Tong University Journal Center

Analysis and Applications of Neuromorphic Memristors in Artificial Intelligence Computing

image: 

  • Task-oriented perspective on neuromorphic memristors for artificial intelligence computing.
  • From material systems and device structures to applications, flexibility, and system integration.
  • A critical roadmap toward scalable, manufacturable, and practical neuromorphic hardware.
view more 

Credit: Jianhui Wang, Qingxin Chen, Jinhao Zhang, Jialin Meng*, Tianyu Wang*.






As artificial intelligence continues to advance, conventional computing architectures face growing challenges in energy consumption, data-transfer latency, and hardware scalability. The separation of memory and processing units in traditional von Neumann systems creates a significant bottleneck, particularly for data-intensive AI applications. Neuromorphic computing, inspired by the parallel and energy-efficient information processing of the human brain, offers a promising alternative. In a comprehensive review, researchers from Shandong University, Fudan University, and collaborating institutions examine the development of neuromorphic memristors, highlighting how materials, device architectures, and system integration can be tailored to meet the demands of next-generation artificial intelligence.

Why Neuromorphic Memristors Matter

Biological neurons and synapses process information through electrical signals and dynamically adjustable connections. Neuromorphic memristors mimic these functions by regulating and retaining electrical conductance states, enabling artificial synaptic plasticity and neuron-like behavior. Their compact structures, nonvolatile memory characteristics, and potential for dense integration make them attractive for in-memory computing, where data storage and processing can occur within the same hardware. By reducing data movement between separate memory and processing units, these devices offer opportunities to improve computational efficiency and reduce power consumption.

Innovative Design and Mechanism

The review systematically examines representative memristor platforms, including oxide-based materials, ferroelectric systems, polymers, and two-dimensional materials. It also explores different device architectures and switching mechanisms, such as conductive filament formation, ionic migration, polarization modulation, phase transitions, and charge storage. Importantly, the authors emphasize that device optimization should be guided by the intended computational task rather than conventional switching performance alone. Different platforms offer distinct advantages: ferroelectric oxides provide opportunities for CMOS-compatible integration, polymer systems support flexible electronics, and layered materials enable ion-controlled neuromorphic functions. Reconfigurable devices capable of switching between volatile and nonvolatile operating modes further expand the range of possible computing applications.

Outstanding Performance

Recent demonstrations highlighted in the review illustrate the practical potential of these technologies. A multifunctional memristive synapse achieves approximately two orders of magnitude lower energy consumption than a GPU-based computing solution. Memristor arrays have also enabled epilepsy-related neural signal analysis with 93.46% accuracy and nearly 400-fold higher power efficiency than state-of-the-art CMOS systems. Other examples include artificial neurons operating at voltages as low as 80 mV, flexible three-dimensional memristor arrays, and wearable textile networks capable of combining sensing and neuromorphic processing. Together, these advances demonstrate the potential of memristors for efficient, adaptive, and mechanically flexible intelligent hardware.

Applications and Future Outlook

The review connects neuromorphic memristor technologies with a broad range of AI applications, including medical signal recognition, image reconstruction, artificial vision, intelligent sensing, and gesture-based human–machine interaction. Nevertheless, challenges remain in manufacturing reproducibility, device variability, standardized performance evaluation, scalable fabrication, and system-level integration. The authors highlight the need to coordinate materials engineering, CMOS-compatible architectures, peripheral circuits, and software–hardware co-design. Ultimately, integrating sensing, memory, computing, and interaction into compact hardware platforms could bring energy-efficient neuromorphic intelligence closer to practical applications in wearable electronics, edge AI, and intelligent human–machine interfaces.

 

KAIST develops lip-reading AI that decodes clear words first to resolve ambiguous mouth movements



Professor Yong Man Ro’s team in the School of Electrical Engineering has developed an AI system that first identifies words with clearly distinguishable lip movements, then uses those words as context to decode ambiguous parts of a sentence



The Korea Advanced Institute of Science and Technology (KAIST)

KAIST Develops Lip-Reading AI That Decodes Clear Words First to Resolve Ambiguous Mouth Movements

image: 

Conceptual comparison between autoregressive decoding and the proposed method. (a) The conventional approach generates words one at a time from left to right, so it cannot revise an early error, such as misrecognizing “PACK” as “BACK.” (b) The proposed method first identifies words it can recognize with confidence, such as “BAGS” and “TRIP,” and then uses them as context to correctly identify the ambiguous early word.

view more 

Credit: KAIST






Understanding speech from lip movements alone is difficult, particularly in noisy environments. A research team at the Korea Advanced Institute of Science and Technology (KAIST) has developed an AI technology that first recognizes words it can identify with high confidence and then uses the surrounding context to resolve ambiguous parts of the sentence. The technology is expected to enable communication without relying on sound. For example, it could convert speech obscured by noise into text or help people who are deaf or hard of hearing follow conversations.

On November 15, KAIST (President Choongsik Bae) announced that a research team led by Professor Yong Man Ro from the School of Electrical Engineering had developed a technology that improves the accuracy of visual speech recognition (VSR), which converts lip movements into text. The team used a diffusion large language model (DLLM), which generates a sentence by repeatedly filling in masked positions. This allows the model to decode ambiguous lip movements using words that appear both before and after them.

Lip reading is challenging because different sounds can produce very similar lip movements. In English, for example, /p/, /b/, and /m/ are nearly indistinguishable based on visual information alone. Widely used AI models generate sentences in a fixed order, from the first word to the last. As a result, if the model selects an incorrect word early in a sentence, the error can affect the words that follow.

The research team replaced this first-to-last order with an approach that decodes the most certain parts first. The model first identifies words associated with clear lip movements and then uses those words as context to decode the remaining positions. By considering information from both earlier and later parts of the sentence, the model progressively resolves ambiguous positions.

The researchers also improved the training process by dividing it into two stages. In the first stage, the model learns to align lip movements in the video with the corresponding text. In the second stage, it learns to model sentence length. During decoding, the model estimates several plausible sentence lengths based on the video duration and generates a candidate sentence for each length. It then evaluates the candidates based on the likelihood of each sentence length and the model’s confidence in its predictions, selecting the most reliable sentence.

The team trained the model using LRS3, a dataset containing 433 hours of lip-reading video, as its labeled training data, meaning that the videos are paired with ground-truth transcripts. The model achieved a word error rate of 19.4%. Word error rate measures the proportion of words that are incorrectly recognized, omitted, or added compared with the correct transcript. A lower value indicates better performance. This is the lowest error rate reported among existing methods that use only LRS3 as labeled training data.

The technology could be used in communication tools that convert lip movements into text in situations where speech is difficult to hear. For example, it could help determine what workers are saying at industrial sites with loud machinery. It could also be developed into devices that support communication for people who are deaf or hard of hearing.

The team plans to continue research on reducing the size and computational cost of the AI model so that it can be deployed on a wider range of devices. A further goal is to enable the technology to run in real time on wearable devices and robots without an internet connection.

“This study is meaningful because it demonstrates that flexibly controlling the order in which AI generates words can, by itself, overcome a fundamental limitation of speech recognition caused by visual ambiguity,” said Professor Yong Man Ro. He added that the research would lay the foundation for physical AI that people can reliably use on a variety of devices in everyday life and at industrial sites.

Dr. Jeong Hun Yeo, a postdoctoral researcher in the School of Electrical Engineering at KAIST, is the first author. Ph.D. candidates Chae Won Kim and Hyeongseop Rha are co-authors, and Professor Yong Man Ro is the corresponding author. The paper received an Outstanding Paper Award at the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026), a top-tier international conference in natural language processing.

※ Paper title: Diffusion Large Language Models for Visual Speech Recognition
※ DOI: https://doi.org/10.48550/arXiv.2605.28456

This research was supported in part by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (Ministry of Science and ICT) (No. RS-2020-II200004, Development of Previsional Intelligence based on Long-Term Visual Memory Network) and the KAIST Jang Young Sil Fellow Program.

 

New advanced measurement techniques could influence the future of AI and quantum computing




California NanoSystems Institute
Quantum amino acides

image: 

New research started with a special material that behaves unusually when it’s extremely cold — 135 degrees below zero Fahrenheit, close to the most frigid temperature ever recorded in Antarctica — and showed that the unusual behavior could be extended to well above room temperature — up to about 135 F, close to the highest heat ever measured in Death Valley, California.

view more 

Credit: Balandin Lab/UCLA





FINDINGS

Researchers at the UCLA Samueli School of Engineering and Auburn University achieved a scientific first: They measured atomic vibrations that carry heat and sound waves, and connected them to a quantum property called spin, which is a way of characterizing a subatomic particle’s built-in magnetism.

The team looked at thin sheets — only a few atoms tall — made of the semiconductor nickel phosphorous trisulfide in a repeating crystal pattern. At room temperature, the spin of the material’s electrons is random. But cooled below about 180 degrees below zero Fahrenheit, the electrons’ spin organizes into a zigzag pattern with the poles of adjacent electrons pointing in opposite directions. By contrast, the poles point in the same direction in a magnet.

The study found that quantum spins and the vibrations, called acoustic phonons, had a stronger correlation below -180 F, when the electrons’ spin was organized. Around the temperature at which the change occurs, acoustic phonons on the surface of the semiconductor slowed down. 

Taken together, the findings indicate that the magnetic order of the material affects how vibrations travel through its crystal lattice.

BACKGROUND

The processing behind today’s popular artificial intelligence applications requires lots of energy to power servers and water to cool them down. At the same time, the accumulation of heat on computer chips may soon hamper the capacity to add more and more features year after year, which has been at the heart of progress in computing since the 1950s.

Controlling quantum spin and its interaction with phonons could help overcome these limitations. The possibilities include systems that process data using spin, phonons or magnetic waves. Platforms such as these are expected to use energy more efficiently and dissipate heat more effectively than current computers, which use the flow of electrons for processing. 

METHOD

The team used a technique with the unique ability to visualize acoustic phonons at the Brillouin – Mandelstam Inelastic Light Scattering Spectroscopy (BMS) Facility located in the California NanoSystems Institute at UCLA (CNSI) building. The instrumentation was custom-built by Alexander Balandin, holder of the Fang Lu Endowed Chair in Engineering at UCLA Samueli and a distinguished professor of materials science and engineering, and his colleagues. The researchers also confirmed the mechanisms they observed using computer simulations.

IMPACT

 

At left is an electron microscope image of nickel phosphorous trisulfide’s crystal lattice. At right is a graph measuring vibrations from heat and sound. (Image courtesy: Balandin Lab / UCLA)

New avenues for technology open up when the counterintuitive quantum behaviors of electrons and other subatomic particles are linked to events on the macroscale such as acoustic phonons. For instance, the touchscreen on a smartphone or tablet relies on quantum tunneling, in which particles seem to pop up on the other side of what should be an impassable barrier.

Measuring spin and phonons is a necessary step toward turning them into foundations for energy-efficient devices resilient to overheating. While magnetic hard drives that store data using “spintronics” have been available to consumers for many years, processing data using spin is still out of reach. 

Looking for materials suitable for this sort of processing, scientists have mostly focused on ones that conduct or insulate electricity. Using semiconductors such as nickel phosphorous trisulfide could provide the flexibility to combine electrons, spins and phonons to perform computing more efficiently than traditional microelectronics.

The findings may also benefit the development of future quantum computers, which are expected to yield unbreakable cybersecurity and model nature more accurately than traditional computers can. A huge challenge in this field is that the quantum states needed to realize this potential are fragile. Phonons easily disrupt them, so breaking the link between phonons and spin would be a major boon.

Beyond the implications for computing, this study demonstrates an entirely new capability for measurement. By allowing scientists to access information about acoustic phonons and their relationship with quantum properties, Brillouin – Mandelstam spectroscopy may play a role in new fundamental physics discoveries and new technologies we can’t imagine today.

AUTHORS

The study’s co-corresponding authors are Balandin, a CNSI member, the director of the BMS Facility and the research thrust lead in future electronics materials in the Semiconductor Hub at UCLA Samueli; and Fariborz Kargar of Auburn. Zahra Ebrahim Nataj, a UCLA doctoral candidate, is the study’s first author. Other authors are affiliated with UCLA and Auburn.

DISCLOSURES

There are no disclosures associated with this study.

JOURNAL

The study was published in Nature Communications.

FUNDING

The research was supported by the National Science Foundation and Auburn University’s Easley High Performance Computing Cluster.

 

How good is AI at guiding student thinking? $2 million grant helps PSU researchers find out




Portland State University






Generative AI systems can solve math problems in seconds, but for students learning math, working through a problem matters far more than simply getting it right. Yet AI still struggles to evaluate student reasoning and provide the kind of feedback a teacher gives to help students take the next step without doing the thinking for them.

A team of Portland State University researchers is helping bridge that gap through a new project — "Seeing Through Data: Benchmarking Adaptive Feedback on Grades 6–12 Data Reasoning" — that will develop evaluation tools and benchmarks for testing how well AI systems respond to students’ mathematical reasoning.

Eva Thanheiser, a professor of mathematics education, and Ameeta Agrawal, an associate professor of computer science, are evaluating whether AI models can recognize the mathematical reasoning in a student's response and suggest a useful next step. They are joined by Amanda Sugimoto, professor of education, who leads the project’s literacy component.

But they're adamant about one thing: It's not about replacing teachers; it's about creating better ways to evaluate AI that could eventually be used to support teaching and learning without compromising authentic learning or student privacy.

The two-year project is supported by a $2 million grant from the Gates Foundation and Learning Commons through the Digital Promise K-12 AI Infrastructure Program. The initiative focuses on developing openly shared datasets, models and benchmarks to advance the accuracy and relevance of AI in education.

"Any model can give you feedback, but there are degrees of feedback," Agrawal said. "No matter the system, we want to give different dimensions and evaluate how well the feedback responds to student reasoning. How well does Claude give scaffolding versus blurting out the answer? How well does GPT-6 Astra improve a student's reasoning versus something else?"

The dimensions that the models will be scored against will be informed by real math tasks collected from students and teachers in partnering school districts.

Thanheiser will lead that effort, working with teachers to develop tasks that invite multiple forms of reasoning. Teachers will then guide their students in a back-and-forth loop, identifying the student's reasoning, suggesting next steps and repeating as students respond.

"The feedback is grounded in the student's thinking rather than in the correct answer," she said.

The tasks will focus on data visualization reasoning, which Thanheiser calls one of the most important forms of communication today.

"To participate in larger conversations, you have to figure out what the intent was behind the data visualization so you can consume it properly," she said. "We know this is really difficult for a lot of students."

Thanheiser said critical data literacy is not simply about students making graphs but about interpreting, critiquing and using data to make sense of real-world issues. Students have to decide what data to include, how to represent those data, what comparisons are fair, what claims are warranted and what additional information is needed.

And students encounter graphs beyond math class. They appear in science articles, in the news and on their phones, usually attached to a claim somebody wants them to believe. Sugimoto is extending the work into English language arts, building tasks around infographics and data-based claims in informational text. Her research focuses on how linguistically and culturally diverse students are assessed, which, for this project, means asking whether AI feedback recognizes sound reasoning when it doesn't arrive in polished English. Recent evaluations suggest this is where these systems are weakest, performing worst on work from the students who need the most support.

Thanheiser said these types of data visualization problems serve as a good test case for evaluating whether AI-generated feedback can recognize and respond to students' reasoning.

For example, a student might create a bar graph showing that School A has 80 suspensions and School B has 50 suspensions, then claim that School A has a "bigger discipline problem." Thanheiser said a weak AI response might praise the graph and suggest adding labels, while a stronger response would notice that the schools have different numbers of students and prompt the student to consider whether raw counts or suspension rates better support the claim.

"AI is more than happy to just do the work for you, but we don't want to do that," Agrawal said. "We want AI to enable your thinking."

The iterative data pieces that the researchers collect — 1,200 in total, which will be stripped of any student identity — will help Agrawal establish the benchmark.

Some of the data will be used to teach an internal model what constitutes a good example of feedback, but most will be used to test whether the models have learned. Rather than producing a single score for whether an AI response is "good," the benchmark will examine questions such as: Did the system accurately interpret the student’s reasoning? Did it choose an appropriate next instructional move? Did the generated feedback actually carry out that move? Did it preserve meaningful mathematical work for the student? And did the system change its response appropriately when the student demonstrated new understanding?

In a world of synthetic, contaminated AI-generated data, the researchers said authentic samples of real student work and real teacher feedback are like gold.

"Once we've developed our model, we want to use this to test any other new model's ability," Agrawal said. "For a school or district wanting to adopt an AI model in the future, this project will provide an independent way to examine specific capabilities and limitations that matter when AI systems respond to student reasoning."

For the researchers, that distinction is fundamental. The question is not whether AI can solve the math, but whether an AI system can recognize the thinking a student has already done and support the student in doing the thinking that comes next.