Monday, June 02, 2025

AI chatbots aren’t experts on psych med reactions — yet

Georgia Institute of Technology

Asking artificial intelligence for advice can be tempting. Powered by large language models (LLMs), AI chatbots are available 24/7, are often free to use, and draw on troves of data to answer questions. Now, people with mental health conditions are asking AI for advice when experiencing potential side effects of psychiatric medicines — a decidedly higher-risk situation than asking it to summarize a report.

One question puzzling the AI research community is how AI performs when asked about mental health emergencies. Globally, including in the U.S., there is a significant gap in mental health treatment, with many individuals having limited to no access to mental healthcare. It’s no surprise that people have started turning to AI chatbots with urgent health-related questions.

Now, researchers at the Georgia Institute of Technology have developed a new framework to evaluate how well AI chatbots can detect potential adverse drug reactions in chat conversations, and how closely their advice aligns with human experts. The study was led by Munmun De Choudhury, J.Z. Liang Associate Professor in the School of Interactive Computing, and Mohit Chandra, a third-year computer science Ph.D. student. De Choudhury is also a faculty member in the Georgia Tech Institute for People and Technology.

“People use AI chatbots for anything and everything,” said Chandra, the study’s first author. “When people have limited access to healthcare providers, they are increasingly likely to turn to AI agents to make sense of what’s happening to them and what they can do to address their problem. We were curious how these tools would fare, given that mental health scenarios can be very subjective and nuanced.”

De Choudhury, Chandra, and their colleagues introduced their new framework at the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics on April 29, 2025.

Putting AI to the Test

Going into their research, De Choudhury and Chandra wanted to answer two main questions: First, can AI chatbots accurately detect whether someone is having side effects or adverse reactions to medication? Second, if they can accurately detect these scenarios, can AI agents then recommend good strategies or action plans to mitigate or reduce harm?

The researchers collaborated with a team of psychiatrists and psychiatry students to establish clinically accurate answers from a human perspective and used those to analyze AI responses.

To build their dataset, they went to the internet’s public square, Reddit, where many have gone for years to ask questions about medication and side effects.

They evaluated nine LLMs, including general purpose models (such as GPT-4o and LLama-3.1), and specialized medical models trained on medical data. Using the evaluation criteria provided by the psychiatrists, they computed how precise the LLMs were in detecting adverse reactions and correctly categorizing the types of adverse reactions caused by psychiatric medications.

Additionally, they prompted LLMs to generate answers to queries posted on Reddit and compared the alignment of LLM answers with those provided by the clinicians over four criteria: (1) emotion and tone expressed, (2) answer readability, (3) proposed harm-reduction strategies, and (4) actionability of the proposed strategies.

The research team found that LLMs stumble when comprehending the nuances of an adverse drug reaction and distinguishing different types of side effects. They also discovered that while LLMs sounded like human psychiatrists in their tones and emotions — such as being helpful and polite — they had difficulty providing true, actionable advice aligned with the experts.

Better Bots, Better Outcomes

The team’s findings could help AI developers build safer, more effective chatbots. Chandra’s ultimate goals are to inform policymakers of the importance of accurate chatbots and help researchers and developers improve LLMs by making their advice more actionable and personalized.

Chandra notes that improving AI for psychiatric and mental health concerns would be particularly life-changing for communities that lack access to mental healthcare.

“When you look at populations with little or no access to mental healthcare, these models are incredible tools for people to use in their daily lives,” Chandra said. “They are always available, they can explain complex things in your native language, and they become a great option to go to for your queries.

“When the AI gives you incorrect information by mistake, it could have serious implications on real life,” Chandra added. “Studies like this are important, because they help reveal the shortcomings of LLMs and identify where we can improve.”

Citation: Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use, (Chandra et al., NAACL 2025).

Funding: National Science Foundation (NSF), American Foundation for Suicide Prevention (AFSP), Microsoft Accelerate Foundation Models Research grant program. The findings, interpretations, and conclusions of this paper are those of the authors and do not represent the official views of NSF, AFSP, or Microsoft.

Article Title

Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use

When less is more: How inhibition shapes learning

Georgia Institute of Technology

image:
This illustration titled, "Gateway to Memory," illustrates Dr. Singer's and Jeong's research and how interneurons act as gatekeepers that open specifically on paths to important locations to enable learning for those places. This art imaginatively represents this inhibitory gating as a monolithic gate opening and spilling light onto the crucial path, guiding the seeker toward their goal.
view more
Credit: Myriam Wares

Nuri Jeong remembers the feeling of surprise she felt during a trip back to South Korea, while visiting her grandmother, who’d been grappling with Alzheimer’s disease.

“I hadn’t seen her in six years, but she recognized me,” said Jeong, a former graduate researcher in the lab of Annabelle Singer in the Wallace H. Coulter Department of Biomedical Engineering at Georgia Tech and Emory University.

“I didn’t expect that. Even though my grandmother struggled to remember other family members that she saw all the time, she somehow remembered me,” Jeong added. “It made me wonder how the brain distinguishes between familiar and new experiences.”

That experience inspired Jeong to embark on a deep-dive exploration of spatial learning and memory, which has resulted in a new study published this month in the journal Nature.

In their article, Jeong, Singer, and a team of Georgia Tech researchers explain how the brain rapidly learns and remembers important locations.

“The brain relies on spatial learning to navigate the world, whether it’s finding a shortcut through a new neighborhood or remembering where you parked your car,” said Jeong, the paper’s lead author.

Clearing the Way for Learning

Most research focuses on excitatory neurons in spatial learning. Jeong, Singer, and team went in a different direction, probing the crucial role of inhibition.

Specifically, the team looked at the role of inhibitory neurons called parvalbumin interneurons, or PVs, in the hippocampus — a small part of the brain that helps with learning, memory, and spatial navigation. The researchers found that when PVs reduce their activity, it clears the way for excitatory neurons, which drive brain activity, to strengthen their bonds and reinforce learning.

“Think of PVs as a kind of circuit breaker that keeps our brain circuits from going haywire,” said Singer. “This research shows that inhibition isn’t static but plays a much more dynamic role in how we learn and remember. It isn’t just about putting the brakes on to keep our brains in check. It’s about precisely timing the release of inhibition to let the brain rapidly encode important information.”

The researchers used optogenetics — a technique for controlling neurons with light — to manipulate interneurons in real time. They monitored thousands of neurons in the mouse brain. As animals ran through a virtual reality maze and learned the location of a reward, PVs decreased activity before the award was even reached. Meanwhile, when the researchers prevented this decrease in inhibition, the animals failed to learn.

“We were surprised that PVs decreased their firing as animals approach a learned reward zone," Singer said. "The decrease actually predicted the reward. This challenges the traditional idea that more neural activity always equates to learning."

Selective Inhibition

The research has potential implications for Alzheimer’s research, according to Singer, because inhibition is impaired in the disease. She and her team want to understand why.

“When we think of Alzheimer’s, we often think about an overactive brain,” she said. “But it isn’t just a volume problem. It’s a timing and location problem. If inhibition isn’t decreasing in the right place in the right moments, the brain may struggle to form new memories. “

The study findings have potential applications for Alzheimer’s, learning disabilities, even techniques to enhance memory through non-invasive brain stimulation.

"By studying how the healthy brain learns, we’re uncovering fundamental principles that could have far-reaching implications,” said Singer, who plans to explore how PVs function in disease models, going forward. “If we can identify what goes wrong in these circuits, we might find ways to restore normal function and improve spatial learning.”

For Jeong, the research inspired by her grandmother remains personal. Before earning her Ph.D. in neuroscience from Emory in 2023, she got into an auto accident that set her back for a few months. She had plenty of time during her recovery to think of her future. Now she uses her neuroscience expertise as a corporate trainer and personal coach under the umbrella of her company, Goals Unhindered.

“But my first love was research, and this study is a reminder for me that inhibition in the brain, like setbacks in life, isn’t just about stopping activity,” she said. “It’s about learning and shaping new memories, and how we make our way in the world."

Dr. Annabelle Singer photographed in her lab.

Credit

Georgia Institute of Technology

Writer: Jerry Grillo

Journal

Nature

DOI

10.1038/s41586-025-08868-5

Method of Research

Experimental study

Subject of Research

Animals