Experts urge complex systems approach to assess A.I. risks
The social context and its complex interactions must be considered and public engagement must be encouraged
Complexity Science Hub
image:
A: an illustration of the complex system of AI-infused social networks. B: a model of amplification of biases in networks due to feedback between algorithm and human decision over time. B (i), Simulations are performed on a synthetic network of size 2000 with 30% nodes being minorities and minority homophily and majority homophily are set to 0.7 each. In homophilic networks minorities are less presented in the top ranks compared to their size, 30%. In each time step, 5 random links are removed from the network. They are then rewired, with higher-ranked nodes having a greater probability of being chosen as targets for the new connection. The ranking is recalculated and this process of link rewiring is repeated over many feedback loops. The fraction of minorities in the upper ranks (eg: the top 10%) reduces over time. The results are averaged over 10 independent experiments. In part (ii), we can observe that the fraction of minorities in the top 10% goes from 23.4% down to 21.8% in 60 iterations. Part (iii) measures demographic parity where demonstrates how far in the rank 30% of minorities are present as we expect, all else equal. At the beginning of the process, we achieve 30% representation of the minorities when we arrive at the top 56% of the nodes. At the end of the process, we need to include the top 71% to get this fair representation.
view moreCredit: Complexity Science Hub
[Vienna, November 13, 2024] — With artificial intelligence increasingly permeating every aspect of our lives, experts are becoming more and more concerned about its dangers. In some cases, the risks are pressing, in others they won't emerge until many months or even years from now. Scientists point out in The Royal Society’s journal that a coherent approach to understanding these threats is still elusive. They call for a complex systems perspective to better assess and mitigate these risks, particularly in light of long-term uncertainties and complex interactions between A.I. and society.
"Understanding the risks of A.I. requires recognizing the intricate interplay between technology and society. It's about navigating the complex, co-evolving systems that shape our decisions and behaviors,” says Fariba Karimi, co-author of the article. Karimi leads the research team on Algorithmic Fairness at the Complexity Science Hub (CSH) and is professor of Social Data Science at TU Graz.
“We should not only discuss what technologies to deploy and how, but also how to adapt the social context to capitalize on positive possibilities. A.I. possibilities and risks should likely be taken into account in debates about, for instance, economic policy,” adds CSH scientist Dániel Kondor, first author of the study.
Broader and Long-Term Risks
Current risk assessment frameworks often focus on immediate, specific harms, such as bias and safety concerns, according to the authors of the article published in Philosophical Transactions A. “These frameworks often overlook broader, long-term systemic risks that could emerge from the widespread deployment of A.I. technologies and their interaction with the social context they are used,” says Kondor.
“In this paper, we tried to balance the short-term perspectives on algorithms with long-term views of how these technologies affect society. It's about making sense of both the immediate and systemic consequences of A.I.," adds Kondor.
What Happens in Real Life
As a case study to illustrate the potential risks of A.I. technologies, the scientists discuss how a predictive algorithm was used during the Covid-19 pandemic in the UK for school exams. The new solution was “presumed to be more objective and thus fairer [than asking teachers to predict their students’ performance], relying on a statistical analysis of students’ performance in previous years,” according to the study.
However, when the algorithm was put into practice, several issues emerged. “Once the grading algorithm was applied, inequities became glaringly obvious,” observes Valerie Hafez, an independent researcher and study co-author. “Pupils from disadvantaged communities bore the brunt of the futile effort to counter grading inflation, but even overall, 40% of students received lower marks than they would have reasonably expected.”
Hafez reports that many responses in the consultation report indicate that the risk perceived as significant by teachers—the long-term effect of grading lower than deserved—was different from the risk perceived by the designers of the algorithm. The latter were concerned about grade inflation, the resulting pressure on higher education, and a lack of trust in students’ actual abilities.
The Scale and the Scope
This case demonstrates several important issues that arise when deploying large-scale algorithmic solutions, emphasize the scientists. “One thing we believe one should be attentive to is the scale—and scope—because algorithms scale: they travel well from one context to the next, even though these contexts may be vastly different. The original context of creation does not simply disappear, rather it is superimposed on all these other contexts,” explains Hafez.
"Long-term risks are not the linear combination of short-term risks. They can escalate exponentially over time. However, with computational models and simulations, we can provide practical insights to better assess these dynamic risks,” adds Karimi.
Computational Models – and Public Participation
This is one of the directions proposed by the scientists for understanding and evaluating risk associated with A.I. technologies, both in the short- and long-term. “Computational models—like those assessing the effect of A.I. on minority representation in social networks—can demonstrate how biases in A.I. systems lead to feedback loops that reinforce societal inequalities,” explains Kondor. Such models can be used to simulate potential risks, offering insights that are difficult to glean from traditional assessment methods.
In addition, the study's authors emphasize the importance of involving laypeople and experts from various fields in the risk assessment process. Competency groups—small, heterogeneous teams that bring together varied perspectives—can be a key tool for fostering democratic participation and ensuring that risk assessments are informed by those most affected by AI technologies.
“A more general issue is the promotion of social resilience, which will help A.I.-related debates and decision-making function better and avoid pitfalls. In turn, social resilience may depend on many questions unrelated (or at least not directly related) to artificial intelligence,” ponders Kondor. Increasing participatory forms of decision-making can be one important component of raising resilience.
“I think that once you begin to see A.I. systems as sociotechnical, you cannot separate the people affected by the A.I. systems from the ‘technical’ aspects. Separating them from the A.I. system takes away their possibility to shape the infrastructures of classification imposed on them, denying affected persons the power to share in creating worlds attenuated to their needs,” says Hafez, who’s an A.I. policy officer at the Austrian Federal Chancellery.
About the Study
The study “Complex systems perspective in assessing risks in A.I.,” by Dániel Kondor, Valerie Hafez, Sudhang Shankar, Rania Wazir, and Fariba Karimi was published in Philosophical Transactions A and is available online.
About CSH
The Complexity Science Hub (CSH) is Europe’s research center for the study of complex systems. We derive meaning from data from a range of disciplines — economics, medicine, ecology, and the social sciences — as a basis for actionable solutions for a better world. Established in 2015, we have grown to over 70 researchers, driven by the increasing demand to gain a genuine understanding of the networks that underlie society, from healthcare to supply chains. Through our complexity science approaches linking physics, mathematics, and computational modeling with data and network science, we develop the capacity to address today’s and tomorrow’s challenges.
Journal
Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences
Method of Research
Case study
Subject of Research
People
Article Title
Complex systems perspective in assessing risks in AI
Article Publication Date
13-Nov-2024
COI Statement
The authors declare no competing interests. Valerie Hafez is a policy officer at the Austrian Federal Chancellery, but conducted this research independently. The views expressed in the paper do not necessarily reflect the views or positions of the Federal Chancellery.
AI needs to work on its conversation game
Researchers discover why AI does a poor job of knowing when to chime in on a conversation
Tufts University
When you have a conversation today, notice the natural points when the exchange leaves open the opportunity for the other person to chime in. If their timing is off, they might be taken as overly aggressive, too timid, or just plain awkward.
The back-and-forth is the social element to the exchange of information that occurs in a conversation, and while humans do this naturally—with some exceptions—AI language systems are universally bad at it.
Linguistics and computer science researchers at Tufts University have now discovered some of the root causes of this shortfall in AI conversational skills and point to possible ways to make them better conversational partners.
When humans interact verbally, for the most part they avoid speaking simultaneously, taking turns to speak and listen. Each person evaluates many input cues to determine what linguists call “transition relevant places” or TRPs. TRPs occur often in a conversation. Many times we will take a pass and let the speaker continue. Other times we will use the TRP to take our turn and share our thoughts.
JP de Ruiter, professor of psychology and computer science, says that for a long time it was thought that the “paraverbal” information in conversations—the intonations, lengthening of words and phrases, pauses, and some visual cues—were the most important signals for identifying a TRP.
“That helps a little bit,” says de Ruiter, “but if you take out the words and just give people the prosody—the melody and rhythm of speech that comes through as if you were talking through a sock—they can no longer detect appropriate TRPs.”
Do the reverse and just provide the linguistic content in a monotone speech, and study subjects will find most of the same TRPs they would find in natural speech.
“What we now know is that the most important cue for taking turns in conversation is the language content itself. The pauses and other cues don’t matter that much,” says de Ruiter.
AI is great at detecting patterns in content, but when de Ruiter, graduate student Muhammad Umair, and research assistant professor of computer science Vasanth Sarathy tested transcribed conversations against a large language model AI, the AI was not able to detect appropriate TRPs anywhere near the capability of humans.
The reason stems from what the AI is trained on. Large language models, including the most advanced ones such as ChatGPT, have been trained on a vast dataset of written content from the internet—Wikipedia entries, online discussion groups, company websites, news sites—just about everything. What is missing from that dataset is any significant amount of transcribed spoken conversational language, which is unscripted, uses simpler vocabulary and shorter sentences, and is structured differently than written language.
AI was not “raised” on conversation, so it does not have the ability to model or engage in conversation in a more natural, human-like manner.
The researchers thought that it might be possible to take a large language model trained on written content and fine-tune it with additional training on a smaller set of conversational content so it can engage more naturally in a novel conversation. When they tried this, they found that there were still some limitations to replicating human-like conversation.
The researchers caution that there may be a fundamental barrier to AI carrying on a natural conversation. “We are assuming that these large language models can understand the content correctly. That may not be the case,” said Sarathy. “They’re predicting the next word based on superficial statistical correlations, but turn taking involves drawing from context much deeper into the conversation.”
“It’s possible that the limitations can be overcome by pre-training large language models on a larger body of naturally occurring spoken language,” said Umair, whose PhD research focuses on human-robot interactions and is the lead author on the studies. “Although we have released a novel training dataset that helps AI identify opportunities for speech in naturally occurring dialogue, collecting such data at a scale required to train today’s AI models remains a significant challenge. There is just not nearly as much conversational recordings and transcripts available compared to written content on the internet.”
The study results were presented at the Empirical Methods in Natural Language Processing (EMNLP) 2024 conference, held in Miami from November 11 to 17 and posted on Arxiv.
Method of Research
Experimental study
Subject of Research
Not applicable
Article Title
Large Language Models Know What To Say But Not When To Speak
