In new places, what we look at first could be as unique as a fingerprint
A Dartmouth study finds no two people take in a new scene the same way, instead focusing on objects with personal meaning
image:
Participants in a Dartmouth study explored real-world scenes in virtual reality while the headset tracked their gaze. Where each person looked, and for how long, was distinctive enough that an AI model could tell participants apart by connecting the objects they focused on thematically and determining the personal meaning they held. In follow-up tests, the AI model correctly predicted what would grab participants' attention in new settings.
view moreCredit: Caroline Robertson/Dartmouth
Walk into a crowded coffee shop, and what catches your eye as you take in the scene could say as much about you as the spirals on your fingertips or the mutations in your DNA.
Eye movements are so unique, in fact, that they could be used to identify you through the objects that have personal meaning, according to a new study by Dartmouth researchers in the Proceedings of the National Academy of Sciences. The findings reveal the depth to which we subjectively evaluate what's around us, while also suggesting that in a world of constant surveillance, we may be giving away more personal information than we realize.
Psychologists have long studied where people consciously or unconsciously focus their attention as they scan a new environment. While we usually come away with a similar understanding of the place itself, each person has distinct perception of how they got there, where they look, and for how long.
It's that variation that the study’s senior author, Caroline Robertson, and her team studied.
“From the earliest moments of taking in a new environment, we make radically different choices about what we pay attention to,” says Robertson, who is an associate professor of psychological and brain sciences at Dartmouth. “This work suggests that our latent conceptual priorities are embedded in the signatures of our gaze.”
A conceptual priority is a kind of personal bias that shapes what jumps out at us visually. A flag and a football, for example, look nothing alike physically, but they are connected by abstract ideas of identity such as patriotism or the United States. The study suggests that, in new environments, we spend more time seeking out information that is conceptually rich and personally meaningful to us.
And what people look for in an unfamiliar environment can distinguish one person from another by the objects and concepts that mean something to them, like a personality fingerprint, the researchers report.
Looking at where people look
Robertson studies visual attention and became focused on the individual differences that kept popping up in her experiments. With the study’s first author, Amanda (“AJ”) Haskins, who received her PhD from Dartmouth in 2024, and former research assistant Katherine Packard ’23, Robertson had about 60 study participants wearing VR headsets immerse themselves in a series of images of everyday scenes, including an auto repair shop, a public swimming pool, and an airport. Participants were free to turn their heads, move their bodies, and look where they pleased in the 16 seconds allotted for each image.
Meanwhile, the researchers used eye-tracking data recorded by the headsets to model each individual’s gaze pattern. They created a machine-learning model to recreate where participants looked within each space; a vision model to recreate the objects that held their attention; and a large language model, or LLM, to look at the conceptual themes that tie these objects together.
The LLM, which are the deep-learning systems that power artificial intelligence, generated captions for each image it processed that hinted at potential storylines. For example, one caption read, “a flag that is on the wall and could be signaling national identity,” while another read, “a missile that is for military equipment and could be part of an aeronautics display.”
When the researchers analyzed the results, they found that the vision model and the LLM could identify individuals by their unique eye movements, and that the LLM, which specifically encoded their conceptual preferences, could do this most accurately. The conceptual map, in other words, was most predictive.
Objects that don't look alike but are thematically linked could be used to tell participants apart when considered as a whole. For example, one person taking in an office scene first looked at writing-related items such as keyboards and notepads, while another individual focused on architectural elements like moldings and decorative backsplashes. These objects' conceptual similarities reveal each participant's unique preferences or interests.
These individual preferences were long lasting, the researchers find. When half the group returned a week later to explore a new set of scenes, the models built from their earlier eye-tracking data accurately predicted the visual features that would grab their attention.
“This suggests that individual differences in gaze patterns contain stable, personality-level preferences that extend beyond testing days,” Robertson says.
Though their gaze patterns varied, the participants had three main perceptual stages. In the first two seconds of taking in a new scene, their gaze focused on spatial dimensions, like the image’s horizon and center, before shifting to prominent visual elements, and after about eight seconds, to the meanings encoded in them.
That intuitively made sense to the researchers. We typically orient ourselves in space and then check out objects and people before transitioning to an interpretive mode that attempts to understand what it all means.
The richer the conceptual information that the LLM received, the more fine-grained distinctions it could pick up on. The researchers found that longer captions containing more context, such as “a hat that is on her head and could be keeping the sun from her eyes,” seemed to elicit more distinctive responses than just “a hat that is on her head.” The more distinctive the eye patterns, the easier they were for the LLM to pick up.
What the eyes give away
The researchers point out that eye-tracking data alone may not reveal our politics or personalities. But their findings suggest that VR and AR could be more intrusive than we realize, potentially giving away more personal data to advertisers than we do now with our clicks across the web.
The study may be the first to use an LLM to model visual gaze—and it will likely not be the last. The researchers are hopeful that the novel AI methods used in the study could have clinical applications.
“Individual gaze differences aren’t random, but rather, are consistent from place to place and stable over time,” says Haskins, now a postdoctoral researcher at the University of California, San Diego. “That’s important if we want to use gaze as a clinical marker of conditions like autism.”
One hallmark of autism is a reduced focus on faces, but it’s been unclear whether face avoidance is more visual than conceptual. The approach the researchers used could help to distinguish between the two.
It could also make earlier diagnosis of autism possible. Symptoms can show up as early as two years old, but currently the national average age at diagnosis is four. “The sooner you could know that a child is processing the world differently, the sooner you could augment the teaching environment,” Robertson says.
The team’s next steps include exploring whether multimodal models that track both visual and cognitive attention could improve predictions further. They also want to test whether the conceptual priorities they’ve identified vary systematically across cultures or clinical groups.
Journal
Proceedings of the National Academy of Sciences
Method of Research
Experimental study
Subject of Research
People
Article Title
Conceptual priorities shape individual gaze patterns during naturalistic visual attention
No comments:
Post a Comment