Tuesday, April 22, 2025

 

Facial expressions of avatars promote risky decision-making


The amygdala plays a key role in driving increased risk-taking



National Institute of Information and Communications Technology (NICT)

Figure 1 Example scenario 

image: 

Figure 1 Example scenario

How expressing opinions in an online meeting may feel different depending on whether a supervisor’s face is shown directly or represented as an avatar.

view more 

Credit: National Institute of Information and Communications Technology (NICT)




Highlights

- Humans take more risks when interacting with facial expressions shown on avatars rather than real human faces.

- This shift in risk-taking behavior is linked to activity in the amygdala.

- The findings offer new insights into both the advantages and the cautionary aspects of communication via avatars.

 

Abstract

A research team led by Dr. TANAKA Toshiko and Dr. HARUNO Masahiko at the National Institute of Information and Communications Technology (NICT, President: TOKUDA Hideyuki Ph.D.), investigated how avatar-mediated communication affects human decision-making. They discovered that participants were more likely to take risks when facial expressions (such as admiration or contempt) were displayed by avatars than when the same expressions were shown on real human faces. This increase in risk-taking was found to result from a more favorable valuation of the "uncertainty" of facial feedback in the avatar condition. Furthermore, fMRI analysis revealed that this valuation of uncertainty depends on activity in the amygdala.

These results offer important insights into how avatar-based social communication, such as in virtual and augmented reality, can affect human decision-making, and highlight the amygdala's critical role in this process. The findings were published in the April 22, 2025 issue of the journal PLOS Biology, which features high-impact research in the life sciences.

 

Background

In recent years, avatar-mediated communication has rapidly spread in contexts such as online meetings and customer service. However, scientific studies on how avatars influence human cognition and decision-making are still in their early stages. In particular, little is known about how behavior changes when a communication partner's facial expressions are shown via avatars rather than real faces. For example, situations such as expressing opinions during online meetings or shopping in virtual stores increasingly involve avatars as communication partners. Understanding how avatars' facial expressions influence risk-related decisions in such settings is crucial.

 

Achievements

In this study, participants performed a risk decision-making task while undergoing fMRI scanning. In each trial, they were presented with two options and asked to choose their preferred one:

  • A "safe" option: a guaranteed small reward (e.g., 80 yen with 100% probability)
  • A "risky" option: a larger reward with lower probability (e.g., 300 yen with 33% probability)

Before the task, participants had a face-to-face meeting with a same-gender observer. During the task, when participants selected the risky option:

  • Success was followed by the observer showing an admiring facial expression
  • Failure was followed by a contemptuous expression

These expressions were shown either using the observer's real face (Human condition) or an avatar face (Avatar condition), switching every 6 to 13 trials (see Figure 2, left).

Results showed that participants made more risky choices in the Avatar condition than in the Human condition, particularly when the risky option had only a slightly higher expected value (see Figure 2, right).

The increase in risky decision-making was explained by a more favorable valuation of the uncertainty of which facial expression would be shown. This valuation was more positive in the Avatar condition.

fMRI analysis further revealed that this valuation of uncertainty was associated with activity in the amygdala (see Figure 3).

 

Future Prospects

This study demonstrates that avatar-mediated communication influences human risk-taking behavior and that this effect is linked to the amygdala. The researchers plan to further investigate how factors such as the avatar's gender or age, individual personality traits, and other types of decision-making tasks are influenced by avatars.

They also aim to explore how avatars can be effectively used to support decision-making in real-world settings such as education and interpersonal support, while also identifying potential risks associated with avatar use.

 

Article information

Authors: Toshiko Tanaka, Masahiko Haruno

Title: Feedback from an avatar facilitates risk-taking by modulating the amygdala response to feedback uncertainty

Journal: PLOS Biology

DOI: 10.1371/journal.pbio.3003122

URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3003122

 

This research was supported in part by the following programs: JST Moonshot R&D Program “Realization of a society where people are free from limitations of body, brain, space, and time by 2050,” project: “Realizing a society where avatars coexist and empower all individuals”,  JST CREST Program “Exploration and application of multi-sensing systems in biological environments,” project: “Decoding multi-world predictive coding in cyber society” (Principal Investigator: HARUNO Masahiko) and Grant-in-Aid for Transformative Research Areas (A) “Decoding and manipulating brain dynamics that induce early behavioral changes, opening up multi-layered biology.”

 

All experiments in this study were approved by the NICT ethics committee. Informed consent was obtained from all participants after explaining the procedures of the study.

Illustrations in the background section were adapted from images generated using Microsoft Copilot.

Figure 2. (Left) Risk decision-making task. (Right) Increase in risk-taking under the avatar condition. 


Figure 3 Amygdala activity reflecting valuation of feedback uncertainty. The amygdala encoded individual differences in how uncertain facial feedback was interpreted, explaining variations in risk-taking behavior across participants.

Credit

National Institute of Information and Communications Technology (NICT)

 

Robot see, robot do: System learns after watching how-to videos



Cornell University







ITHACA, N.Y. – Cornell University researchers have developed a new robotic framework powered by artificial intelligence – called RHyME (Retrieval for Hybrid Imitation under Mismatched Execution) – that allows robots to learn tasks by watching a single how-to video.

Robots can be finicky learners. Historically, they’ve required precise, step-by-step directions to complete basic tasks and tend to call it quits when things go off-script, like after dropping a tool or losing a screw. RHyME, however, could fast-track the development and deployment of robotic systems by significantly reducing the time, energy and money needed to train them, the researchers said.

“One of the annoying things about working with robots is collecting so much data on the robot doing different tasks,” said Kushal Kedia, a doctoral student in the field of computer science. “That’s not how humans do tasks. We look at other people as inspiration.”

Kedia will present the paper, “One-Shot Imitation under Mismatched Execution,” in May at the Institute of Electrical and Electronics Engineers’ International Conference on Robotics and Automation, in Atlanta.

Home robot assistants are still a long way off because they lack the wits to navigate the physical world and its countless contingencies. To get robots up to speed, researchers like Kedia are training them with what amounts to how-to videos – human demonstrations of various tasks in a lab setting. The hope with this approach, a branch of machine learning called “imitation learning,” is that robots will learn a sequence of tasks faster and be able to adapt to real-world environments.

“Our work is like translating French to English – we’re translating any given task from human to robot,” said senior author Sanjiban Choudhury, assistant professor of computer science.

This translation task still faces a broader challenge, however: Humans move too fluidly for a robot to track and mimic, and training robots with video requires gobs of it. Further, video demonstrations – of, say, picking up a napkin or stacking dinner plates – must be performed slowly and flawlessly, since any mismatch in actions between the video and the robot has historically spelled doom for robot learning, the researchers said.

“If a human moves in a way that’s any different from how a robot moves, the method immediately falls apart,” Choudhury said. “Our thinking was, ‘Can we find a principled way to deal with this mismatch between how humans and robots do tasks?’”

RHyME is the team’s answer – a scalable approach that makes robots less finicky and more adaptive. It supercharges a robotic system to use its own memory and connect the dots when performing tasks it has viewed only once by drawing on videos it has seen. For example, a RHyME-equipped robot shown a video of a human fetching a mug from the counter and placing it in a nearby sink will comb its bank of videos and draw inspiration from similar actions – like grasping a cup and lowering a utensil.

RHyME paves the way for robots to learn multiple-step sequences while significantly lowering the amount of robot data needed for training, the researchers said. RHyME requires just 30 minutes of robot data; in a lab setting, robots trained using the system achieved a more than 50% increase in task success compared to previous methods, the researchers said.

Media note: Video and images can be viewed and downloaded here.

For additional information, see this Cornell Chronicle story.

-30-


Brain-inspired AI breakthrough: making computers see more like humans



IBS-Yonsei team unveils novel Lp-Convolution at ICLR 2025



Institute for Basic Science

Figure 1. Information Processing Structures of the Brain’s Visual Cortex and Artificial Neural Networks 

image: 

In the actual brain’s visual cortex, neurons are connected broadly and smoothly around a central point, with connection strength varying gradually with distance (a, b). This spatial connectivity follows a bell-shaped curve known as a ‘Gaussian distribution,’ enabling the brain to integrate visual information not only from the center but also from the surrounding areas.

In contrast, traditional Convolutional Neural Networks (CNNs) process information by having neurons focus on a fixed rectangular region (e.g., 3×3, 5×5, etc.) (c, d). CNN filters move across an image at regular intervals, extracting information in a uniform manner, which limits their ability to capture relationships between distant visual elements or respond selectively based on importance.

This study addresses the differences between these biological structures and CNNs, proposing a new filter structure called 'Lp-Convolution' that mimics the brain’s connectivity patterns. In this structure, the range and sensitivity of a neuron’s input are designed to naturally spread in a Gaussian-like form, allowing the system to self-adjust during training—emphasizing important information more strongly while downplaying less relevant details. This enables image processing that is more flexible and biologically aligned compared to traditional CNNs.

view more 

Credit: Institute for Basic Science





A team of researchers from the Institute for Basic Science (IBS), Yonsei University, and the Max Planck Institute have developed a new artificial intelligence (AI) technique that brings machine vision closer to how the human brain processes images. Called Lp-Convolution, this method improves the accuracy and efficiency of image recognition systems while reducing the computational burden of existing AI models.

Bridging the Gap Between CNNs and the Human Brain

The human brain is remarkably efficient at identifying key details in complex scenes, an ability that traditional AI systems have struggled to replicate. Convolutional Neural Networks (CNNs)—the most widely used AI model for image recognition—process images using small, square-shaped filters. While effective, this rigid approach limits their ability to capture broader patterns in fragmented data.

More recently, Vision Transformers (ViTs) have shown superior performance by analyzing entire images at once, but they require massive computational power and large datasets, making them impractical for many real-world applications.

Inspired by how the brain’s visual cortex processes information selectively through circular, sparse connections, the research team sought a middle ground: Could a brain-like approach make CNNs both efficient and powerful?

Introducing Lp-Convolution: A Smarter Way to See

To answer this, the team developed Lp-Convolution, a novel method that uses a multivariate p-generalized normal distribution (MPND) to reshape CNN filters dynamically. Unlike traditional CNNs, which use fixed square filters, Lp-Convolution allows AI models to adapt their filter shapes—stretching horizontally or vertically based on the task, much like how the human brain selectively focuses on relevant details.

This breakthrough solves a long-standing challenge in AI research, known as the large kernel problem. Simply increasing filter sizes in CNNs (e.g., using 7×7 or larger kernels) usually does not improve performance, despite adding more parameters. Lp-Convolution overcomes this limitation by introducing flexible, biologically inspired connectivity patterns.

Real-World Performance: Stronger, Smarter, and More Robust AI

In tests on standard image classification datasets (CIFAR-100, TinyImageNet), Lp-Convolution significantly improved accuracy on both classic models like AlexNet and modern architectures like RepLKNet. The method also proved to be highly robust against corrupted data, a major challenge in real-world AI applications.

Moreover, the researchers found that when the Lp-masks used in their method resembled a Gaussian distribution, the AI’s internal processing patterns closely matched biological neural activity, as confirmed through comparisons with mouse brain data.

“We humans quickly spot what matters in a crowded scene,” said Dr. C. Justin LEE, Director of the Center for Cognition and Sociality within the Institute for Basic Science. “Our Lp-Convolution mimics this ability, allowing AI to flexibly focus on the most relevant parts of an image—just like the brain does.”

Impact and Future Applications

Unlike previous efforts that either relied on small, rigid filters or required resource-heavy transformers, Lp-Convolution offers a practical, efficient alternative. This innovation could revolutionize fields such as:

- Autonomous driving, where AI must quickly detect obstacles in real time

- Medical imaging, improving AI-based diagnoses by highlighting subtle details

- Robotics, enabling smarter and more adaptable machine vision under changing conditions

“This work is a powerful contribution to both AI and neuroscience,” said Director C. Justin Lee. “By aligning AI more closely with the brain, we’ve unlocked new potential for CNNs, making them smarter, more adaptable, and more biologically realistic.”

Looking ahead, the team plans to refine this technology further, exploring its applications in complex reasoning tasks such as puzzle-solving (e.g., Sudoku) and real-time image processing.

The study will be presented at the International Conference on Learning Representations (ICLR) 2025, and the research team has made their code and models publicly available:

🔗 https://github.com/jeakwon/lpconv/



The brain processes visual information using a Gaussian-shaped connectivity structure that gradually spreads from the center outward, flexibly integrating a wide range of information. In contrast, traditional CNNs face issues where expanding the filter size dilutes information or reduces accuracy (d, e).

To overcome these structural limitations, the research team developed Lp-Convolution, inspired by the brain’s connectivity (a–c). This design spatially distributes weights to preserve key information even over large receptive fields, effectively addressing the shortcomings of conventional CNNs.

Credit

Institute for Basic Science