Tuesday, August 26, 2025

Simpler models can outperform deep learning at climate prediction

New research shows the natural variability in climate data can cause AI models to struggle at predicting local temperature and rainfall

Massachusetts Institute of Technology

CAMBRIDGE, MA -- Environmental scientists are increasingly using enormous artificial intelligence models to make predictions about changes in weather and climate, but a new study by MIT researchers shows that bigger models are not always better.

The team demonstrates that, in certain climate scenarios, much simpler, physics-based models can generate more accurate predictions than state-of-the-art deep-learning models.

Their analysis also reveals that a benchmarking technique commonly used to evaluate machine-learning techniques for climate predictions can be distorted by natural variations in the data, like fluctuations in weather patterns. This could lead someone to believe a deep-learning model makes more accurate predictions when that is not the case.

The researchers developed a more robust way of evaluating these techniques, which shows that, while simple models are more accurate when estimating regional surface temperatures, deep-learning approaches can be the best choice for estimating local rainfall.

They used these results to enhance a simulation tool known as a climate emulator, which can rapidly simulate the effect of human activities onto a future climate.

The researchers see their work as a “cautionary tale” about the risk of deploying large AI models for climate science. While deep-learning models have shown incredible success in domains such as natural language, climate science contains a proven set of physical laws and approximations, and the challenge becomes how to incorporate those into AI models.

“We are trying to develop models that are going to be useful and relevant for the kinds of things that decision-makers need going forward when making climate policy choices. While it might be attractive to use the latest, big-picture machine-learning model on a climate problem, what this study shows is that stepping back and really thinking about the problem fundamentals is important and useful,” says study senior author Noelle Selin, a professor in the MIT Institute for Data, Systems, and Society (IDSS) and the Department of Earth, Atmospheric and Planetary Sciences (EAPS).

Selin’s co-authors are lead author Björn Lütjens, a former EAPS postdoc who is now a research scientist at IBM Research; senior author Raffaele Ferrari, the Cecil and Ida Green Professor of Oceanography in EAPS and director of the MIT Program in Atmospheres, Oceans, and Climate; and Duncan Watson-Parris, assistant professor at the University of California at San Diego. Selin and Ferrari are also co-principal investigators of the Bringing Computation to the Climate Challenge project, out of which this research emerged. The paper appears today in the Journal of Advances in Modeling Earth Systems.

Comparing emulators

Because the Earth’s climate is so complex, running a state-of-the-art climate model to predict how pollution levels will impact environmental factors like temperature can take weeks on the world’s most powerful supercomputers.

Scientists often create climate emulators, simpler approximations of a state-of-the art climate model, which are faster and more accessible. A policymaker could use a climate emulator to see how alternative assumptions on greenhouse gas emissions would affect future temperatures, helping them develop regulations.

But an emulator isn’t very useful if it makes inaccurate predictions about the local impacts of climate change. While deep learning has become increasingly popular for emulation, few studies have explored whether these models perform better than tried-and-true approaches.

The MIT researchers performed such a study. They compared a traditional technique called linear pattern scaling (LPS) with a deep-learning model using a common benchmark dataset for evaluating climate emulators.

Their results showed that LPS outperformed deep-learning models on predicting nearly all parameters they tested, including temperature and precipitation.

“Large AI methods are very appealing to scientists, but they rarely solve a completely new problem, so implementing an existing solution first is necessary to find out whether the complex machine-learning approach actually improves upon it,” says Lütjens.

Some initial results seemed to fly in the face of the researchers’ domain knowledge. The powerful deep-learning model should have been more accurate when making predictions about precipitation, since those data don’t follow a linear pattern.

They found that the high amount of natural variability in climate model runs can cause the deep learning model to perform poorly on unpredictable long-term oscillations, like El Niño/La Niña. This skews the benchmarking scores in favor of LPS, which averages out those oscillations.

Constructing a new evaluation

From there, the researchers constructed a new evaluation with more data that address natural climate variability. With this new evaluation, the deep-learning model performed slightly better than LPS for local precipitation, but LPS was still more accurate for temperature predictions.

“It is important to use the modeling tool that is right for the problem, but in order to do that you also have to set up the problem the right way in the first place,” Selin says.

Based on these results, the researchers incorporated LPS into a climate emulation platform to predict local temperature changes in different emission scenarios.

“We are not advocating that LPS should always be the goal. It still has limitations. For instance, LPS doesn’t predict variability or extreme weather events,” Ferrari adds.

Rather, they hope their results emphasize the need to develop better benchmarking techniques, which could provide a fuller picture of which climate emulation technique is best suited for a particular situation.

“With an improved climate emulation benchmark, we could use more complex machine-learning methods to explore problems that are currently very hard to address, like the impacts of aerosols or estimations of extreme precipitation,” Lütjens says.

Ultimately, more accurate benchmarking techniques will help ensure policymakers are making decisions based on the best available information.

The researchers hope others build on their analysis, perhaps by studying additional improvements to climate emulation methods and benchmarks. Such research could explore impact-oriented metrics like drought indicators and wildfire risks, or new variables like regional wind speeds.

###

This research is funded, in part, by Schmidt Sciences, LLC, and is part of the MIT Climate Grand Challenges team for “Bringing Computation to the Climate Challenge.”

Journal

Journal of Advances in Modeling Earth Systems

DOI

10.1029/2024MS004619

Method of Research

Computational simulation/modeling

Article Title

The Impact of Internal Variability on Benchmarking Deep Learning Climate Emulators

Article Publication Date

26-Aug-2025

Expert on catfishes publishes updated volume on catfish biology and evolution

University of Kansas

Critically endangered catfish — image:

The beauty of Hypancistrus zebra (Loricariidae), a critically endangered species, heavily demanded by the aquarium trade and threatened by the Belo Monte hydropower plant in Brazil.
view more
Credit: Leandro Sousa

LAWRENCE — Few people on Earth know as much about catfishes as University of Kansas researcher Gloria Arratia, who serves as editor and contributor to the just-published first volume of “Catfishes: A Highly Diversified Group” (CRC Press, 2025), a two-volume reference. While the first volume focuses on the fascinating anatomy of catfishes, the second will focus on their evolution and genetic relationships.

Arratia’s new work, co-written by Roberto Reis of Pontifícia Universidade Católica do Rio Grande do Sul in Brazil, reflects the latest understanding of the family tree of Siluriformes (the scientific name for catfishes), incorporating genetic data from the most recent molecular studies to better represent phylogenetic relationships and including species from understudied areas like Africa and Asia.

Arratia said catfishes have long captured the popular imagination.

“Curiously, catfish often attract people’s attention because of their unusual and sometimes ‘ugly’ appearance,” Arratia said. “This fascination isn’t limited to specialists; ordinary people are also drawn to stories about catfish. Hearing about a fish that has ‘barbels,’ like whiskers, or that lives at night — it gives catfish an air of mystery. People are especially intrigued when they learn that many catfish appear on certain nights, often with the rising of the moon, and others produce sounds and ‘talk’ to each other.”

The first edition of Arratia’s "Catfishes" came out in 2003, but new scientific methods and understanding compelled the researcher to compile a new edition.

“Copies sold out soon, and I began receiving requests from people around the world asking for a second edition,” she said. “Given the many changes in catfish research over 20 years, I proposed a new edition. A colleague, Roberto Reis from Brazil, home to the largest number of catfish species in the world, joined me as co-editor. Brazil has become the global center of ichthyology, with specialists in morphology, molecular biology, parasitology and many other areas. It was logical to bring this expertise into the new book.”

Arratia said the result is not simply an updated edition but an entirely new book, larger than the first, with expanded chapters and a dedicated section on phylogenetic relationships.

“Our understanding of the group has advanced so much that a new work was essential,” she said.

Arratia’s scholarly interest in catfishes stems from first studying them in her home country of Chile, where the two catfish species that dominate freshwater ecosystems are the most ancient of the fish group worldwide.

“I was born in Chile, where there are two catfish groups considered among the most primitive families in the world,” she said. “One family is Diplomystidae, which is endemic to Argentina and Chile. The other, Nematogenyidae, is found only in Chile. As a young researcher, this uniqueness caught my attention, and I dedicated much of my academic work to studying these fishes, which appear and act like ancient catfish would have.”

“I even produced three monographs on Diplomystidae, publishing descriptions of more than 300 pages,” Arratia said. “Anyone studying catfish anywhere in the world — whether in Asia, Europe or South America — must understand these two families to grasp the evolutionary history of catfish as a group. That is how I became deeply interested in them.”

Asked what she found most interesting about the biology of catfishes, Arratia mentioned the “Weberian apparatus,” also found in a larger group of fishes that includes the Cypriniformes (for example, carps), Characiformes (like tetras) and Gymnotiformes or Neotropical electric fishes.

“Located just behind the cranium, the first four to six vertebrae are highly modified,” she said. “In most fishes, vertebrae are relatively simple, consisting of a centrum and associated processes. But in catfish, these vertebrae are modified in connection with the gas bladder. The result is a specialized organ that enhances hearing and balance. Sound vibrations from the gas bladder are transmitted through the modified vertebrae to the inner ear, allowing catfish to detect and interpret sounds more effectively than most other fishes. This system not only improves hearing but also plays a role in buoyancy control. Some catfish can even produce sounds and communicate with one another using the Weberian apparatus and its connection to the gas bladder.”

Arratia said her new work would incorporate research from understudied areas like Africa and Asia, where fieldwork and sampling have been limited, leading to gaps in catfish knowledge.

“Looking ahead, our expectation is that Africa and Asia will produce many new discoveries,” she said. “Research there is only beginning. In Asia, most studies so far have been molecular, with very little work on morphology, even though those catfish are quite different and deserve close study. The same is true in Africa. Most of the research has been concentrated in South Africa and especially on one family, while we know very little about species across the rest of the continent. In general, we still don’t know much about the anatomy of catfish where many areas are just beginning to be explored, like the nervous system and reproductive system, as well as strategies in reproduction and early development. There’s a lot of work to do, especially with species from Asia and Africa — two continents where our knowledge is limited.”

Next up is Arratia’s second volume of the updated edition, due out this October.

Mount Sinai scientists identify three potent human antibodies against mpox, paving the way for new protective therapies

The Mount Sinai Hospital / Mount Sinai School of Medicine

image:
Mount Sinai scientists identified antibodies from a recovered mpox patient that specifically recognize the viral protein A35. These antibodies not only neutralize the virus and protect animal models from disease but also mirror protective immune responses in humans, establishing A35 as a key target for future antiviral strategies.
view more
Credit: Raianna Fantin, PhD, Postdoctoral Fellow, Coelho Lab at Mount Sinai

A team from the Microbiology Department at the Icahn School of Medicine at Mount Sinai has discovered three powerful monoclonal antibodies from a person who had previously been infected with mpox (formerly known as monkeypox).

These antibodies, which target the viral protein A35, blocked viral spread in laboratory in vitro tests and, most importantly, protected rodents from severe disease and fully prevented death. The findings, published August 22 in Cell, also reveal that humans previously infected with mpox carry high levels of these protective antibodies in their blood, and that their presence is associated with milder symptoms and the absence of hospitalization.

Mpox is a viral disease caused by an orthopoxvirus (the family of viruses that causes smallpox, which killed more than 300 million people in the 20th century alone). This virus spreads mainly through close contact with an infected individual. The virus causes a painful rash, enlarged lymph nodes, and fever. It can make people very sick and leave scars.

A global outbreak of mpox began in 2022 and continues to this day. In the last three years, the World Health Organization twice declared the disease a public health emergency of international concern, yet effective treatment for this virus family is lacking. Recent human clinical trials for the leading candidate therapy did not show it to have efficacy. Thus, there is currently no approved drug to treat mpox. This study from Mount Sinai might change that.

“A previous study published in 2023 by our team showed that human antibodies targeting the viral protein A35 were unusually increased in sera in response to mpox infection compared with vaccination for smallpox or antibodies against other viral proteins,” said Camila Coelho, PhD, MBA, Assistant Professor of Microbiology at the Icahn School of Medicine and senior author of the paper. “Based on this earlier finding, we hypothesized that antibodies targeting A35 from mpox-infected individuals would be highly protective against orthopoxviruses, since the viruses in this family share high genetic similarity. We aimed to address the urgent unmet need for effective treatments for orthopoxviruses, and with the help of our outstanding collaborators, I am very proud to say that we are close to achieving that goal.”

The newly discovered antibodies bind to a region that is highly conserved across not only the orthopoxvirus genus, to which MPXV belongs to, but also the entire poxvirus family, which means the region is conserved among different viruses and is not prone to mutations or antibody escape. The ability of these antibodies to block viral spread, protect from severe disease, and fully prevent death suggests that they are promising drug candidates that could be tested in humans for the prevention or treatment of mpox. The antibodies have been patented by the Icahn School of Medicine at Mount Sinai.

“Ours is the first report of the crystal structure of a human antibody bound to an mpox virus protein, providing a detailed map of a vulnerability in the virus,” said Raianna Fantin, PhD, the postdoctoral researcher who is the first author of the study. “It is also the first time that monoclonal antibodies against orthopoxviruses were quantified in human sera. We were surprised by how consistently people recovering from mpox infection produced antibodies targeting the same A35 epitope of the virus as the antibodies we discovered.”

While the researchers see this as a promising next-generation therapy against mpox in humans, it is important to note that this is still in the research stage. Future testing in human clinical trials is needed to evaluate how the antibodies behave in the body, how long they last, where they go after being administered to the human body, and how well they prevent orthopoxvirus infection.

The research team plans to move the newly discovered antibodies into advanced preclinical safety and efficacy testing. Simultaneously, they are using the structural and immunological insights from the study to guide future work on understanding and enhancing the human immune response to mpox.

The study was conducted by the following laboratories from Mount Sinai’s Department of Microbiology: Coelho, Schotsaert, Simon, Bajic, and Krammer.

About the Mount Sinai Health System
Mount Sinai Health System is one of the largest academic medical systems in the New York metro area, with 48,000 employees working across seven hospitals, more than 400 outpatient practices, more than 600 research and clinical labs, a school of nursing, and a leading school of medicine and graduate education. Mount Sinai advances health for all people, everywhere, by taking on the most complex health care challenges of our time—discovering and applying new scientific learning and knowledge; developing safer, more effective treatments; educating the next generation of medical leaders and innovators; and supporting local communities by delivering high-quality care to all who need it.

Through the integration of its hospitals, labs, and schools, Mount Sinai offers comprehensive health care solutions from birth through geriatrics, leveraging innovative approaches such as artificial intelligence and informatics while keeping patients’ medical and emotional needs at the center of all treatment. The Health System includes approximately 9,000 primary and specialty care physicians and 10 free-standing joint-venture centers throughout the five boroughs of New York City, Westchester, Long Island, and Florida. Hospitals within the System are consistently ranked by Newsweek’s® “The World’s Best Smart Hospitals, Best in State Hospitals, World Best Hospitals and Best Specialty Hospitals” and by U.S. News & World Report's® “Best Hospitals” and “Best Children’s Hospitals.” The Mount Sinai Hospital is on the U.S. News & World Report® “Best Hospitals” Honor Roll for 2025-2026.

For more information, visit https://www.mountsinai.org or find Mount Sinai on Facebook, Instagram, LinkedIn, X, and YouTube.

###