New study: AI chatbots systematically violate mental health ethics standards

Brown University

FacebookXLinkedInWeChatBlueskyMessageWhatsAppEmail

 

image: 

Licensed psychologists reviewed simulated chats based on real chatbot responses revealing numerous ethical violations, including over-validation of user's beliefs. 

view more 

Credit: Zainab

PROVIDENCE, R.I. [Brown University] — As more people turn to ChatGPT and other large language models (LLMs) for mental health advice, a new study details how these chatbots — even when prompted to use evidence-based psychotherapy techniques — systematically violate ethical standards of practice established by organizations like the American Psychological Association. 

The research, led by Brown University computer scientists working side-by-side with mental health practitioners, showed that chatbots are prone to a variety of ethical violations. Those include inappropriately navigating crisis situations, providing misleading responses that reinforce users’ negative beliefs about themselves and others, and creating a false sense of empathy with users.

“In this work, we present a practitioner-informed framework of 15 ethical risks to demonstrate how LLM counselors violate ethical standards in mental health practice by mapping the model’s behavior to specific ethical violations,” the researchers wrote in their study. “We call on future work to create ethical, educational and legal standards for LLM counselors — standards that are reflective of the quality and rigor of care required for human-facilitated psychotherapy.”

The research will be presented on October 22, 2025 at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society. Members of the research team are affiliated with Brown’s Center for Technological Responsibility, Reimagination and Redesign.

Zainab Iftikhar, a Ph.D. candidate in computer science at Brown who led the work, was interested in how different prompts might impact the output of LLMs in mental health settings. Specifically, she aimed to determine whether such strategies could help models adhere to ethical principles for real-world deployment.

“Prompts are instructions that are given to the model to guide its behavior for achieving a specific task,” Iftikhar said. “You don’t change the underlying model or provide new data, but the prompt helps guide the model's output based on its pre-existing knowledge and learned patterns.

“For example, a user might prompt the model with: ‘Act as a cognitive behavioral therapist to help me reframe my thoughts,’ or ‘Use principles of dialectical behavior therapy to assist me in understanding and managing my emotions.’ While these models do not actually perform these therapeutic techniques like a human would, they rather use their learned patterns to generate responses that align with the concepts of CBT or DBT based on the input prompt provided.”

Individual users chatting directly with LLMs like ChatGPT can use such prompts and often do. Iftikhar says that users often share the prompts they use on TikTok and Instagram, and there are long Reddit threads dedicated discussing prompt strategies. But the problem potentially goes beyond individual users. Many of the mental health chatbots marketed to consumers are prompted versions of more general LLMs. So understanding how prompts specific to mental health affect the output of LLMs is critical.

For the study, Iftikhar and her colleagues observed a group of peer counselors working with an online mental health support platform. The researchers first observed seven peer counselors, all of whom were trained in cognitive behavioral therapy techniques, as they conducted self-counseling chats with CBT-prompted LLMs, including various versions of OpenAI’s GPT Series, Anthropic’s Claude and Meta’s Llama. Next, a subset of simulated chats based on original human counseling chats were evaluated by three licensed clinical psychologists who helped to identify potential ethics violations in the chat logs.

The study revealed 15 ethical risks falling into five general categories:

• Lack of contextual adaptation: Ignoring peoples’ lived experiences and recommending one-size-fits-all interventions.
• Poor therapeutic collaboration: Dominating the conversation and occasionally reinforcing a user’s false beliefs.
• Deceptive empathy: Using phrases like “I see you” or “I understand” to create a false connection between the user and the bot.
• Unfair discrimination: Exhibiting gender, cultural or religious bias.
• Lack of safety and crisis management: Denying service on sensitive topics, failing to refer users to appropriate resources or responding indifferently to crisis situations including suicide ideation.

Iftikhar acknowledges that while human therapists are also susceptible to these ethical risks, the key difference is accountability.

“For human therapists, there are governing boards and mechanisms for providers to be held professionally liable for mistreatment and malpractice,” Iftikhar said. “But when LLM counselors make these violations, there are no established regulatory frameworks.”

The findings do not necessarily mean that AI should not have a role in mental health treatment, Iftikhar says. She and her colleagues believe that AI has the potential to help reduce barriers to care arising from the cost of treatment or the availability of trained professionals. However, she says, the results underscore the need for thoughtful implementation of AI technologies as well as appropriate regulation and oversight.

For now, Iftikhar hopes the findings will make users more aware of the risks posed by current AI systems. 

“If you’re talking to a chatbot about mental health, these are some things that people should be looking out for,” she said.

Ellie Pavlick, a computer science professor at Brown who was not part of the research team, said the research highlights need for careful scientific study of AI systems deployed in mental health settings. Pavlick leads ARIA, a National Science Foundation AI research institute at Brown aimed at developing trustworthy AI assistants. 

“The reality of AI today is that it's far easier to build and deploy systems than to evaluate and understand them,” Pavlick said. “This paper required a team of clinical experts and a study that lasted for more than a year in order to demonstrate these risks. Most work in AI today is evaluated using automatic metrics which, by design, are static and lack a human in the loop.”

She says the work could provide a template for future research on making AI safe for mental health support.

“There is a real opportunity for AI to play a role in combating the mental health crisis that our society is facing, but it's of the utmost importance that we take the time to really critique and evaluate our systems every step of the way to avoid doing more harm than good,” Pavlick said. “This work offers a good example of what that can look like.”

---

DOI

10.1609/aies.v8i2.36632 

Article Publication Date

22-Oct-2025

 

This is how happy we need to be to have lower chronic disease mortality risk

Researchers used data from 123 countries to identify a ‘happiness threshold’ beyond which nations’ non-communicable disease mortality rate decreases, making happiness a public health resource

Frontiers

FacebookXLinkedInWeChatBlueskyMessageWhatsAppEmail

DON'T WORRY BE HAPPY

Heart disease, cancer, asthma, and diabetes: All are chronic or non-communicable diseases (NCD), which accounted for about 75% of non-pandemic related deaths in 2021. They may result from genetic, environmental, and behavioral factors, or a combination thereof. But can other factors also influence disease risk?

Now, a new Frontiers in Medicine study has investigated the relationship between happiness and health to find out if happier always means healthier and to determine if happiness and co-occurring health benefits are linear or follow a specific pattern.

“We show that subjective well-being, or happiness, appears to function as a population health asset only once a minimum threshold of approximately 2.7 on the Life Ladder scale is surpassed,” said first author Prof Iulia Iuga, a researcher at 1 Decembrie 1918 University of Alba Iulia. “Above this tipping point, increased happiness is associated with a decrease in NCD mortality.”

Happy equals healthy

“The life ladder can be imaged as a simple zero to 10 happiness ruler, where zero means the worst possible life and 10 means the best possible life,” explained Iuga. “People imagine where they currently stand on that ladder.” The team used data sourced from different health organizations, global development statistics, and public opinion polls. The data came from 123 countries and was collected between 2006 and 2021.

A score of 2.7 can be found towards the lower end of the ladder, and people or countries finding themselves there are generally considered unhappy or struggling. “An adjective that fits this level could be ‘barely coping’,” said Iuga. Nevertheless, already at this point, improvements in happiness begin to translate into measurable health benefits.

Once the threshold is surpassed and a country’s collective happiness rises above it, the study found that each 1% increase in subjective well-being is linked to an estimated 0.43% decrease in that country’s 30-to-70-year NCD mortality rate. This rate refers to the percentage of deaths due to NCDs among individuals aged between 30 and 70.

“Within the observed range, we found no evidence of adverse effects from ‘excessive’ happiness,” Iuga added. Below the 2.7-point threshold, small improvements in happiness (for example, from a score of 2 to 2.2) do not translate to measurable reduction in NCD deaths, the data indicated. Before measurable changes can be unlocked, very low well-being needs to be remedied, the study suggested.

Health unlocked

Countries that exceeded this threshold tend to have higher per person health spending, stronger social safety nets, and more stable governance as opposed to the countries falling below it. The average life ladder score across the examined countries during the study period was 5.45, with a minimum of 2.18 and a maximum of 7.97.

There are several ways that governments could raise countries above a score of 2.7, for example through promoting healthy living by expanding obesity prevention and tightening alcohol availability; improving the environment through stricter air-quality standard; and increasing their per capita health spending. The authors said their insights could help guide health and social policies and might aid to integrate well-being into nations’ agendas.

The authors pointed out that the life ladder scores making up their data were self-reported, which may have resulted in measurement errors, differences in cross-cultural response styles, or reporting bias. It is also possible that subnational differences between populations were captured inadequately. In the future, studies should include more measures, such as years lived with disability or hospital admission records, include subnational micro-data, and expand coverage to low-income or conflict states, which may have been overlooked in the data they used, the team pointed out.

Nevertheless, identifying the protective effects of happiness could be an important step towards healthier people. “Identifying this tipping point could provide more accurate evidence for health policy,” concluded Iuga. “Happiness is not just a personal feeling but also a measurable public health resource.”

---

Journal

Frontiers in Medicine

DOI

10.3389/fmed.2025.1667645 

Method of Research

Data/statistical analysis

Subject of Research

Not applicable

Article Title

How Happy is Healthy Enough? Uncovering the Happiness Threshold for Global Non-Communicable Disease Prevention

Article Publication Date

21-Oct-2025

Brainwave study sheds light on cause of ‘hearing voices’

University of New South Wales

FacebookXLinkedInWeChatBlueskyMessageWhatsAppEmail

A new study led by psychologists from UNSW Sydney has provided the strongest evidence yet that auditory verbal hallucinations – or hearing voices – in schizophrenia may stem from a disruption in the brain’s ability to recognise its own inner voice.

In a paper published today in the journal Schizophrenia Bulletin, the researchers say the finding could also be an important step towards finding biological indicators that point to the presence of schizophrenia. This is significant as there are currently no blood tests, brain scans, or lab-based biomarkers – signs in the body that can tell us something about our health – that are uniquely characteristic of schizophrenia.

Professor Thomas Whitford, with the UNSW School of Psychology, has been examining the role of inner speech in the cognition of healthy people and people living with schizophrenia spectrum disorders for some time.

“Inner speech is the voice in your head that silently narrates your thoughts – what you’re doing, planning, or noticing,” he says.

“Most people experience inner speech regularly, often without realising it, though there are some who don’t experience it at all.

“Our research shows that when we speak – even just in our heads – the part of the brain that processes sounds from the outside world becomes less active. This is because the brain predicts the sound of our own voice. But in people who hear voices, this prediction seems to go wrong, and the brain reacts as if the voice is coming from someone else.”

Brainwave analysis

Prof. Whitford says this confirms what mental health researchers have long theorised: that auditory hallucinations in schizophrenia may be due to the person's own inner speech being misattributed as external speech.

“This idea's been around for 50 years, but it's been very difficult to test because inner speech is inherently private,” he says.

“How do you measure it? One way is by using an EEG, which records the brain’s electrical activity. Even though we can’t hear inner speech, the brain still reacts to it – and in healthy people, using inner speech produces the same kind of reduction in brain activity as when they speak out loud.

“But in people who hear voices, that reduction of activity doesn’t happen. In fact, their brains react even more strongly to inner speech, as if it’s coming from someone else. That might help explain why the voices feel so real.”

Sound choices

The researchers divided participants into three groups. The first group included 55 people living with schizophrenia-spectrum disorders who had experienced auditory verbal hallucinations (AVH) in the past week. The second group of 44 participants also had schizophrenia, but either had no history of AVH or hadn’t experienced them recently. The third group was a control group of 43 healthy people with no history of schizophrenia.

Each participant was connected to an EEG (electroencephalography) device to measure brainwaves as they listened to audio over headphones. They were asked to imagine saying either ‘bah’ or ‘bih’ in their minds at the exact moment they heard recordings of one of those two sounds played through headphones. The participants had no way of knowing whether the sound they heard in the headphones would match the sound they made in their imagination.

In the healthy participants, when the sound that played in the headphones matched the syllable they imagined saying in their minds, the EEG showed reduced activity in the auditory cortex – the part of the brain that processes sound and speech. This suggests the brain was predicting the sound and dampening its response – similar to what happens when we speak out loud.

However, in the group of participants who had recently experienced AVH, the results were the reverse. In these individuals, instead of the expected suppression of brain activity when the imagined speech matched the sound heard, the EEG showed an enhanced response.

“Their brains reacted more strongly to inner speech that matched the external sound, which was the exact opposite of what we found in the healthy participants,” Prof. Whitford says.

“This reversal of the normal suppression effect suggests that the brain’s prediction mechanism may be disrupted in people currently experiencing auditory hallucinations, which may cause their own inner voice to be misinterpreted as external speech.”

Participants in the second group – people with a schizophrenia-spectrum disorder who hadn’t experienced AVH recently or at all – showed a pattern that was intermediate between the healthy participants and the hallucinating participants.

What this means

The researchers say this is the strongest confirmation to date that the brains of people living with schizophrenia are misperceiving imagined speech as speech that is produced externally.

“It was always a plausible theory – that people were hearing their own thoughts spoken out loud – but this new approach has provided the strongest and most direct test of this theory to date,” Prof. Whitford says.

He says the next thing he and his fellow researchers want to assess is if this measure can be used to predict who might transition to psychosis, with the potential to identify people with high risk of developing psychosis, which would allow for early intervention.

“This sort of measure has great potential to be a biomarker for the development of psychosis,” Prof. Whitford says.

“Ultimately, I think that understanding the biological causes of the symptoms of schizophrenia is a necessary first step if we hope to develop new and effective treatments.”

---

Journal

Schizophrenia Bulletin

DOI

10.1093/schbul/sbaf167 

Method of Research

Experimental study

Subject of Research

People

Article Title

Corollary discharge dysfunction to inner speech and its relationship to auditory verbal hallucinations in patients with schizophrenia spectrum disorders

Article Publication Date

21-Oct-2025

 

Life in the fast (and slow) lanes for salmon

Investigating life-history variation and climate adaptability

Kyoto University

FacebookXLinkedInWeChatBlueskyMessageWhatsAppEmail

 

image: 

Fast-life and slow-life Masu salmon tend to inhabit different parts of a river

view more 

Credit: KyotoU / Takeya Shida

Kyoto, Japan -- Life-history variation is fundamental to the long-term persistence of populations and species because it ensures their ability to adapt to changing environments. Many important studies have focused on life-history variation between habitats, but the variation maintained within a habitat has often been overlooked.

Unravelling this puzzle at the landscape level is critical for understanding the spatial scales at which adaption and population persistence operate in nature. This motivated a team of researchers at Kyoto University to investigate life-history variation in masu salmon.

"We wanted to understand how the variation in life-history is partitioned within and among habitats across heterogeneous landscapes," says first author Takeya Shida.

Previous research has demonstrated salmonid fish inhabiting large temperate watersheds to be an ideal subject for this type of study. In salmonids, fast-life individuals achieve accelerated growth early in the growing season -- initiating gonadal development in spring and maturing in autumn of a given year -- whereas slow-life individuals postpone the maturation decision until the following year.

Armed with this knowledge, the research team examined the hierarchical structure of life-history variation of the salmon in a large temperate watershed. They predicted that each habitat would maintain a certain degree of variation, but that the tendency for fast-life would be more prevalent in downstream reaches while slow-life would be more prevalent upstream.

The team established six study reaches, or segments, along a certain stream and investigated seasonal changes in water temperature and food resources, along with the seasonal growth of salmon juveniles and the ages of the mature fish. They then quantified the relative contributions of differences in age of mature fish between habitats and within habitats to the overall diversity of age of mature fish throughout the watershed, making separate calculations for males and females.

The results largely confirmed the team's hypothesis. Fast-life individuals tended to dominate downstream reaches -- with warmer temperatures and abundant aquatic invertebrate prey -- while slow-life individuals were more prevalent upstream, with colder temperatures and fewer aquatic prey but relatively abundant terrestrial invertebrate inputs into streams.

However, the scientists also found considerable variation in the age of mature fish in all areas, except in the farthest downstream and upstream reaches. This suggests the variation in life-history at the watershed scale to be more strongly dependent on within-habitat variation than on between-habitat.

"It's intriguing that the balance between within-habitat and between-habitat diversity may be influenced not only by environmental factors within the stream, but also by terrestrial invertebrate prey supplied from surrounding riparian forests," says Shida.

As habitat homogenization has continued, such within-habitat variation may become more important to support adaptation to environmental changes such as global climate change.

"Recently, we have been witnessing a loss of diversity within species far more quickly than species diversity," says team leader Takuya Sato. "This study reminded us of the importance of managing salmonid resources in a way that preserves this diversity."

###

The paper "Hierarchical organization of life-history variation in a salmonid fish across riverscape: relevance of seasonal growth opportunity and maturation decision-window" appeared on 21 October 2025 in the Journal of Animal Ecology, with doi: 10.1111/1365-2656.70141

About Kyoto University

Kyoto University is one of Japan and Asia's premier research institutions, founded in 1897 and responsible for producing numerous Nobel laureates and winners of other prestigious international prizes. A broad curriculum across the arts and sciences at undergraduate and graduate levels complements several research centers, facilities, and offices around Japan and the world. For more information, please see: http://www.kyoto-u.ac.jp/en

---

Journal

Journal of Animal Ecology

DOI

10.1111/1365-2656.70141 

Method of Research

Observational study

Subject of Research

Animals

Article Title

Hierarchical organization of life-history variation in a salmonid fish across riverscape: relevance of seasonal growth opportunity and maturation decision-window

Article Publication Date

21-Oct-2025