Thursday, October 29, 2026

 

Engineering researcher seeks to prevent AI agents from going rogue


University of Florida





As artificial intelligence systems become increasingly capable of working on their own, keeping them focused on their designated tasks is a growing challenge. 

University of Florida computer science professor Sumit Jha, Ph.D., is working to keep agents from “going rogue.” Jha and his team have developed a system that can detect when an AI model begins to drift from its assigned task and steer it back without retraining the model or adding significant computing demands.  

The research comes as news reports have chronicled AI agents gaining more autonomy and occasionally behaving in ways their developers did not intend. Recent incidents have included AI agents accessing government websites and an Anthropic model submitting an anonymous tip to a police department in July, claiming to have information about an unsolved murder. 

Jha, a professor in UF’s Department of Computer & Information Science & Engineering, or CISE, studies how to make AI systems more controllable and understandable. He is working to engineer simple, non-invasive tweaks to keep systems on task. 

Jha and his collaborators have authored a paper entitled “Drift No More: Spectral Locking for Long-Horizon Open-Weight Multi-agent AI,” detailing a system, dubbed ATLAS, which stands for Algebraic Task–role Locking with Adaptive Spectral Memory. 

The acronym hints at the dense mathematics at the heart of the system. 

Jha and his co-authors, Vishal Pramanik, Maisha Maliha and Susmit Jha, devised a system that works inside the large language model, or LLM, as it self-generates. When the model begins to drift from its task, the ATLAS system steers the LLM back toward the assignment, as well as its original responsibility and the latest shared information.   

“An AI agent can start going wrong before its response looks wrong,” Jha explained. “Inside the model, it may already be drifting from the task, confusing its role or relying on information that has been replaced. ATLAS is built to correct that drift before it reaches the next message, so other agents do not build on the mistake.”  

Existing strategies usually involve policing an AI agent’s answers after the fact. The ATLAS system doesn’t require retraining the model, and it therefore doesn’t expend extra computing resources or slow the model down.  

According to Jha, so-called task drift is especially possible for tasks in which groups of AI agents coordinate with each other over long periods of time. 

“Say, initially you told the agent you’re a doctor, you are supposed to treat patients,” Jha said. “By day No. 7, the fact that it's a doctor and it should treat patients is now just one of the millions of things it has been told. We call this problem role decoherence. The agent was tasked to do something, but after some time it has been told so many other things, it loses track." 

AI teams are usually managed from the outside through prompts, retrieval systems and rules, Jha said.  

“ATLAS adds something closer to an internal compass,” he said. “In mathematical form, it continually reinforces three questions: What is the mission? What is this agent responsible for? And what is true right now? Keeping those three answers aligned is essential when a task lasts much longer than a single conversation.” 

Sounds complicated, but as Jha tells it, the system is akin to taking MRI snapshots of the AI’s brain, or neural network. Imagine an AI model that is supposed to be performing the tasks of a medical professional.  

“Initially when I do the MRI, it looks like the brain of a doctor,” he explained. “Thumbs up. But when we do another MRI, now it looks like, let’s say, the brain of a race car driver. So, we look inside and we find out what the difference was between the old MRI and the new MRI.” 

 

As ATLAS had already learned what the medical neural network should look like, it is able to calculate the smallest set of changes necessary, within the neural network, to return the system to its original task of medical assessment. 

“The agent has already launched, so the intervention has to be very small,” Jha explained. “Taking the MRI is like taking a snapshot, it doesn’t require retraining, so you don’t have to stop the agent. If the agent’s doing alright, it’s still a doctor. ATLAS doesn’t do anything. But if it’s now 99% a doctor, but 1% race car driver, we take the 1% out. We don’t wait until it’s a problem.” 

The paper has been accepted to the 2026 Conference on Empirical Methods in Natural Language Processing, or EMNLP, Oct. 24–29 in Budapest. 



New ‘four as’ framework for the governance of AI

Anglia Ruskin University academics unveil proposal to tackle key ethical challenges



Anglia Ruskin University






Experts have unveiled a new proposal for the governance of artificial intelligence as they argue the debate has moved beyond simply whether to slow the speed of AI development.

The new framework, published in the journal AI and Ethics, introduces the “Four As” – Action, Access, Authority and Allocation – and is the work of Dr Kirti Prakash and Dr Ian van der Linde of the Advanced Computing Research Centre at Anglia Ruskin University (ARU) in Cambridge, England.

The “Four As” framework is intended to preserve competition and innovation in frontier AI development while establishing safeguards around capabilities and uses for AI that could cause severe or irreversible harm.

The framework is based around the key ethical problems and draws lessons from previous attempts to govern powerful technologies, including nuclear arms control, CRISPR governance around genome editing and the 1975 Asilomar conference on recombinant DNA technology.

The researchers describe a slowdown on AI development as an “incomplete” governance objective, as AI capability is not confined to a single company or country, and existing models can be copied and improved without training new frontier models. Geopolitics and commercial incentives add further barriers. 

Instead, the researchers suggest proportionate safeguards, meaningful human control, independent assessment, clear responsibility and regular reassessment as AI capabilities change.

Through the “Four As”, their framework tackles whether humans can effectively constrain consequential AI action (Action), who should be permitted to obtain or exercise potentially harmful AI capabilities (Access), which consequential judgements may be delegated to AI (Authority), and how AI related benefits, burdens and power should be distributed (Allocation).

Rather than treating AI risk as a single problem, the framework identifies which “A” the ethical obligation arises under and matches it to the appropriate form of governance, with developers primarily implementing Action safeguards, regulators governing Access, deploying institutions governing Authority, and governments and public bodies addressing Allocation.

The framework also asks whether interventions have knock-on effects by simultaneously creating a different ethical obligation elsewhere. For example, restricting access to a powerful model may reduce misuse risk and security while concentrating capability and decision-making power among a small number of firms or institutions, creating a new allocation concern.

Dr Kirti Prakash, Senior Lecturer in AI and Applications at Anglia Ruskin University (ARU), said: “The debate around frontier AI is often framed as a choice between accelerating development and slowing it down. We argue that this is too blunt. Before deciding how strongly to intervene, we need to ask what exactly the ethical problem is.

“Losing control of an autonomous AI system is fundamentally different from deciding who should have access to a dangerous capability, whether AI should make a medical or military decision, or whether AI development concentrates power in a handful of organisations. Different problems require different forms of governance.”

Co-author Dr Ian van der Linde, Associate Professor in the School of Computing and Information Science at Anglia Ruskin University (ARU), said: “Effective AI governance will require more than a checklist of general ethical principles. The Four As provide a way of moving from identifying an ethical problem to asking what constraint is required, who should enforce it, what would signal that the safeguard has failed and what corrective action should follow.

“Recent UK policy provides a concrete example of this distinction. The newly announced AI Biosecurity Lab will use frontier AI to strengthen biological security while relying on trusted access to advanced models from companies including OpenAI, Anthropic and Google DeepMind, illustrating how governance may focus not simply on whether a capability exists, but on who can access and deploy it.”

The new “Four As” framework is published as international concern is growing about how rapidly frontier AI could develop. A recent study from the Cambridge Programme on AI Science & Policy and the Centre for the Governance of AI looked at whether increasing automation of AI research itself could trigger an “intelligence explosion”, compressing years of technological progress into months or less.

Dr Prakash added: “The possibility of AI increasingly contributing to the development of its own successors makes the governance question even more urgent. If capabilities begin advancing faster than institutions can adapt, simply asking whether AI is ‘safe’ will not be enough. We need to know who is in control, who has access, where decision-making authority lies and who ultimately gains or bears responsibility for the consequences.”

Journal

DOI

Article Title

Article Publication Date



Optimizing AI energy consumption throughout its entire lifecycle


Funded with $1.83M from DARPA, University of Michigan Engineering leads a project that aims to cut AI's energy needs by up to half




University of Michigan






RESEARCH TO WATCH

Photo of computer racks

WHAT'S THE STUDY: Data center energy consumption is expected to more than double by 2030 from roughly 420 terawatt-hours in 2024, with much of the anticipated growth driven by AI. But more efficient software design could cut AI energy use by up to half, according to University of Michigan Engineering researchers teaming up with colleagues at the University of Illinois and defense contractor BAE Systems.

Rather than relying on expensive, time-consuming trial-and-error testing across thousands of hardware and software configurations, the team's proposed system will enable developers to estimate energy usage across every stage of an AI workflow—from data processing and model training to real-world deployment. Over the course of the project, the researchers will build high-precision energy measurement tools that combine physical sensors with software analysis, releasing them as open-source documentation and software. The team calls the new mathematical framework underpinning these tools "energy calculus."

WHY IT MATTERS: While computer chips are getting faster, misalignments between software and hardware waste massive amounts of electricity—up to 30% during AI model training alone. More energy-efficient AI brings down electricity demand, reducing the cost of AI and extending battery life for AI-enabled devices. This is especially critical for national defense and military technology, where field equipment and remote computing systems must run complex AI algorithms under strict battery and power limits. 

HOW MUCH AND FOR HOW LONG: The $1.83 million contract from the Defense Advanced Research Projects Agency (DARPA) will fund the two-year project, "E-CALC: Energy Calculus for ML Lifecycle Optimization."

The initiative builds on previous efforts by the team lead. For AI model training, the Zeus energy optimization framework automatically tunes settings like batch size and power limits to minimize energy use. Exploring the energy cost of running AI models, the ML.ENERGY benchmark acts as a public leaderboard that measures and compares energy consumption across different open-weight AI models and open-source software frameworks. 

WHO'S INVOLVED: 

Mosharaf Chowdhury is an associate professor of computer science and engineering at the University of Michigan and principal investigator for E-CALC. He leads ML.ENERGY and Zeus projects.

"AI is becoming part of everyday life, business, science and national security, but it requires enormous and rapidly growing amounts of electricity," Chowdhury said. "If that energy use continues to rise unchecked, it could increase costs, strain power infrastructure, and make advanced AI harder to deploy. This project tackles a practical question: How can we get widespread benefits of AI with minimum energy?" 

The two co-principal investigators are Nishil Talati, assistant professor of electrical and computer engineering at the University of Illinois specializing in hardware-software co-design and execution bottleneck characterization, and David Nicholson, principal research engineer at BAE Systems who leads internal research and development on edge AI technologies for defense applications.

Approved for public release, distribution unlimited. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Defense Advanced Research Projects Agency (DARPA).

Navigating AI in transportation: New national research consortium aims to guide agencies



Three leading transportation research institutes are joining forces with agencies, universities, industry and the public so no one in transportation has to figure out AI alone




University of Michigan






Images  //  Video


Key takeaways:

  • AI is here to stay in transportation systems and three research institutes are coming together to determine best uses and practices.

  • The AI tools available to transportation agencies today didn't exist two years ago, and many will be outdated two years from now. 

  • Collaborating institutions, including University of Michigan Engineering, University of California Berkeley and Texas A&M.

 

Three of the nation's leading transportation research institutes have launched an effort to help public transportation agencies use AI effectively as the technology rapidly advances.

The National AI Consortium for Transportation, or NAICT, brings together the University of Michigan Transportation Research Institute, the University of California Berkeley Institute of Transportation Studies and the Texas A&M Transportation Institute into a national research initiative to give agencies a credible, noncommercial resource for evaluating, deploying and governing AI.

The AI tools available to transportation agencies today didn't exist two years ago, and many will be outdated two years from now. New models, products and vendor promises arrive every month, each claiming to make roads safer, freight move faster or infrastructure last longer. For the public agencies responsible for those systems, the hardest question is no longer whether to use AI. It's which tool fits which problem, how to use it well, and how to tell a proven solution from a polished sales pitch.

An inclusive approach

NAICT is not being built behind closed doors. The consortium is inviting state and local transportation agencies, industry, professional organizations and researchers to get involved now, starting with listening sessions this fall and a panel and workshop at the Transportation Research Board Annual Meeting in January 2027.

"NAICT is designed to give agencies and industry a single, credible, noncommercial resource for evaluating, deploying and governing AI in real-world transportation operations," said UMTRI Director Henry Liu. "We need to have a guided discussion to help shape what AI is and what it can do in safe, responsible and effective ways."

Liu co-leads NAICT with Scott Moura, director of Partners for Advanced Transportation Technology, an ITS Berkeley research center, and Greg Winfree, director of the Texas A&M Transportation Institute.

What's next?

NAICT will align its priorities with agencies and practitioners. Initial activities include:

  • UC Berkeley Listening Session (Nov. 5, 2026) : NAICT will hold a workshop and listening session. Transportation leaders and practitioners will have an opportunity to share the AI challenges they face and identify resources and support needed to move forward.

  • Transportation Research Board Annual Meeting (January 2027): NAICT will host a panel and interactive workshop in Washington, D.C., bringing together experts from research, industry and policy to explore safe and effective AI deployment. 

  • Education and Governance Resources (early 2027): NAICT plans to introduce its first educational offerings and a draft AI governance framework. Agencies and partners who engage early will have opportunities to review and help shape these resources before their broader release.

"We're moving fast, and we'd rather build this with agencies than for them," Liu said. "If you're responsible for putting AI to work on public roads, we want you in the room. Come find us at TRB."


Review: Perspectives from psychology could help quantify LLMs’ thinking capacity

Summary author: Abigail Eisenstadt



American Association for the Advancement of Science (AAAS)





A new Review discusses how concepts and methods borrowed from psychology can inform research to evaluate whether and how large language models (LLMs) can employ reasoning. “The ever-growing capabilities of these systems make them challenging but also increasingly important to understand and explain, especially since these expanding capabilities also bring greater potential for unforeseen harm,” Thilo Hagendorff and colleagues write. Their proposed approach – called “machine psychology” – advocates for seeking inspiration from the behavioral sciences. Since psychology’s protocols already assess emotional, cognitive, and social development, they could be modified to examine LLMs’ intelligence, biases, decision-making, social skills, and language. Hagendorff et al. urge caution, though: “Many psychological concepts are normatively laden and can foster mismatches in expectations between AI experts and the public regarding machine capability.”

Journal

DOI

Article Title

Article Publication Date

Nearly half of severely distressed generative AI users turn to chatbots for emotional support, study finds

Caption

Credit

Journal

DOI

Method of Research

Subject of Research

Article Title

Breakthrough from Brown University engineers could enable energy-efficient microchips for AI

Journal

DOI

Article Title

Article Publication Date

Improving AI-generated images, pixel by pixel



Researchers from URochester and NVIDIA develop new models that provide pixel-level control over AI image generation.




University of Rochester

Pix(els) or it didn’t happen

video: 

Developed by researchers at URochester and NVIDIA, PixelDiT2 efficiently implements pixel space diffusion to achieve almost the same performance as the state-of-the-art models—but provides controllability over individual pixels.

view more 

Credit: Available for re-use with credit to Yu et al.






If you have ever used an artificial intelligence text-to-image generator like ChatGPT or Google Gemini, you may have encountered a common but maddening problem: the image looks nearly perfect, but when you ask the AI to fix one small detail, it messes up something unrelated.

The issue is caused by the way the AI models are built. Most state-of-the-art AI image generators use a diffusion model in latent space, meaning they compress concepts such as a cat or a dog into their essential details and store them so they can be retrieved quickly and efficiently when it’s time to create an image. This is part of what gives the models their seemingly magical ability to generate the images you describe in seconds. But the method has its drawbacks.

“Using the latent space helps cut down the computation, but the problem is that you lose controllability because you are not adjusting individual pixels,” says Jiebo Luo, the Albert Arendt Hopeman Professor of Engineering at the University of Rochester. “As a result, you can’t easily control specific parts of the images you generate.”

Reimagining image generation at the pixel level

Luo’s lab and collaborators at the AI infrastructure company NVIDIA were one of the first teams to develop a model that trades the latent space for the pixel space, dubbed PixelDiT. They presented their first iteration at the 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), where it was nominated as a best paper finalist, given to just 15 of over 4,000 accepted papers at the conference. The model allows users to zero in on certain regions of an image, like fixing just the pixels used to illustrate a pair of glasses on a person’s face or adjusting the color of a car.

The team has since created an even more powerful model, PixelDiT2, that implements pixel space diffusion in an efficient way to achieve almost the same performance as the state-of-the-art latent space models but providing controllability over individual pixels. The team will present a paper showcasing PixelDiT2 at the Neural Information Processing Systems conference (NeurIPS) this December.

Luo and his collaborators, who include computer science PhD student and NVIDIA intern Yongsheng Yu as well as alumnus Wei Xiong ’22 PhD (computer science), a senior researcher at NVIDIA, now have turned their attention to an even more difficult problem: video generation.

“Video is a big challenge because you have to worry about smooth transitions from frame to frame; otherwise it looks funny to the human eye,” says Luo. “You also have a lot more data to work with.”

But Luo says the progress the team has made with PixelDiT2 leaves them in a strong position to pursue AI video generation in the pixel space, which would provide users powerful new levels of control over the videos they generate.