Leading tech policy group calls for urgent review of speech recognition technology
Rigorous audit and governance practices proposed
image:
The overarching finding of “TechBrief: Automated Speech Recognition,” is that automated speech recognition (ASR) technologies require rigorous audit and governance practices, as well as guidelines to ensure they are inclusive and accessible.
view moreCredit: Association for Computing Machinery
The Association for Computing Machinery’s global Technology Policy Council has published “TechBrief: Automated Speech Recognition.” It is the latest in a series of short technical bulletins that present scientifically grounded perspectives on the impact and policy implications of specific technological developments in computing.
The overarching finding of “TechBrief: Automated Speech Recognition,” is that automated speech recognition (ASR) technologies require rigorous audit and governance practices, as well as guidelines to ensure they are inclusive and accessible. The authors note that automated speech recognition systems are quickly advancing by incorporating generative AI techniques similar to large language models (LLMs). They present an eye-opening list of ASR applications in daily life, including many uses the public may not have considered.
“The struggle is real. I can’t get AI customer support to understand me, and auto-captioning never matches what I said — it’s so frustrating,” says Shaomei Wu, a co-author of this Tech Brief, a person who stutters, and the founder and CEO of AImpower.org. “I feel voiceless and invalidated when the technology keeps interrupting or misunderstanding me.” Special attention is given to how ASR technologies can significantly impact people’s lives. For example, the brief points out that more than half a million doctors have used ASR tools to transcribe patient visits, and more than 60% of Fortune 100 companies use an ASR-based software package when hiring new employees. By providing this context, the co-authors hope readers will come to understand how important it is that ASR applications work properly.
“This is a story that has fallen under the media radar,” explained co-author Allison Koenecke, Assistant Professor, Cornell Tech. “There has been a good deal of focus on facial recognition and other new AI-based technologies, but relatively little discussion of speech recognition. Once we appreciate how critical it is that these applications function as intended, we come to realize the value of human oversight in how ASR is developed and deployed. Because the advances have been so rapid, and there has not been enough attention paid to speech recognition, we thought this was an opportune time to issue this report.”
The TechBrief authors contend that the first step to building safe and responsible oversight frameworks is to recognize that a “one size fits all” approach won’t work. Their key message is that the performance of an ASR system is highly variable depending on the setting. For example, the same medical patient's speech might reasonably be transcribed differently in varying circumstances (e.g., transcriptions used for diagnostic purposes may optimally include speech fragments and filler words, whereas transcriptions used for generating patient note summaries may not.)
In addition to the performance of speech recognition in high-stakes applications such as healthcare and employment, the co-authors raise concerns about the fairness, transparency, and accountability of ASR systems.
For example, in highlighting the Word Error Rate (WER) metric that is used to evaluate ASR programs, the co-authors relay several research findings to highlight how speech recognition can be inequitable. They cite statistics that: 1) The WER in ASR systems was 1.1 to 3.4 worse for Black American English speakers relative to White American English speakers, and 2) the WER in ASR systems was 2.8 to 4.2 times worse for Chicano English speakers relative to Standard American English speakers. The report contends that these statistics raise concerns about technological bias.
“As this technology becomes more pervasive, evaluating its impact on people of all backgrounds is a responsibility for its developers,” said co-author Jingjin Li, research fellow, AImpower.org.
"We shared emerging research on these questions because of its timely relevance," added Niranjan Sivakumar, a co-author of the ACM TechBrief, as well as Director and Head of Policy at AImpower.org. "Speech recognition is a field that has grown rapidly and is increasingly ubiquitous. We crafted this concise and accessible report to familiarize readers with how these tools are being used today and to draw attention to important issues to consider for the future. We hope our new report reaches a wide audience."
The brief’s co-authors include Allison Koenecke, Niranjan Sivakumar, Jingjin Li, and Shaomei Wu.
ACM’s TechBriefs are designed to complement ACM’s activities in the policy arena and to inform policymakers, the public, and others about the nature and implications of information technologies. Earlier ACM TechBriefs have covered topics such as tech abuse, generative artificial intelligence, climate change, facial recognition, smart cities, quantum simulation, and election auditing. Topics under consideration for future issues include AI development and considerations around open-source initiatives.
About the ACM Technology Policy Council
ACM’s global Technology Policy Council sets the agenda for global initiatives to address evolving technology policy issues and coordinates the activities of ACM's regional technology policy committees in the US and Europe. It serves as the central convening point for ACM's interactions with government organizations, the computing community, and the public in all matters of public policy related to computing and information technology. The Council’s members are drawn from ACM's global membership.
About ACM
ACM, the Association for Computing Machinery, is the world’s largest educational and scientific computing society, uniting computing educators, researchers, and professionals to inspire dialogue, share resources, and address the field’s challenges. ACM strengthens the computing profession’s collective voice through strong leadership, promotion of the highest standards, and recognition of technical excellence. ACM supports the professional growth of its members by providing opportunities for life-long learning, career development, and professional networking.
LLM use is reshaping scientific enterprise by increasing output, reducing quality and more
Summary author: Walter Beckwith
LLM-assisted manuscripts exhibit more complexity of the written word but are lower in research quality, according to a Policy Article by Keigo Kusumegi, Paul Ginsparg, and colleagues that sought to evaluate the impacts of widespread use of generative artificial intelligence (AI) technologies on scientific production. “As AI systems advance, they will challenge our fundamental assumptions about research quality, scholarly communication, and the nature of intellectual labor,” write the authors. “Science policymakers must consider how to evolve our scientific institutions to accommodate the rapidly changing scientific production process.” Despite enormous enthusiasm and growing concern surrounding the use of generative AI and large language models (LLMS) across research and academia, there has been little systemic evidence about how these technologies are reshaping scientific production. To address this gap, Kusumegi et al. assembled five large datasets spanning 2.1 million preprints, 28,000 peer-reviewed reports, and 246 million online views and downloads of scientific documents. Then, using text-based detectors to identify first-time LLM use, they conducted difference-in-differences analyses to compare researchers’ work before and after LLM adoption. They found that LLM adoption increases a researcher’s scientific output by 23.7 – 89.3%, with especially large boosts for authors facing higher writing and language barriers. Kusumegi et al. also discovered that LLM-assisted manuscripts show a reversal of the traditional positive relationship between writing complexity and research quality. After LLM adoption, more sophisticated language was used, but in substantively weak manuscripts. Lastly, the authors show that LLM adopters read and cite more diverse literature, referencing more books, younger works, and less-cited documents. “Our findings show that LLMs have begun to reshape scientific production. These changes portend an evolving research landscape in which the value of English fluency will recede, but the importance of robust quality-assessment frameworks and deep methodological scrutiny is paramount,” write the authors. “For peer reviewers and journal editors, and the community, more broadly, who create, consume, and apply this work, this represents a major issue."
Journal
Science
Article Title
Scientific production in the era of large language models
Article Publication Date
18-Dec-2025
No comments:
Post a Comment