Beyond ChatGPT: WVU researchers to study use and ethics of artificial intelligence across disciplines
image:
Erin Brock Carlson, assistant professor, English, WVU Eberly College of Arts and Sciences
view moreCredit: WVU Photo
Two West Virginia University researchers have designed a curriculum to engage liberal arts faculty in discussions on the social, ethical and technical aspects of artificial intelligence and its role in classrooms.
Through a grant from the National Endowment for the Humanities, Erin Brock Carlson, assistant professor of English, and Scott Davidson, professor of philosophy, both at the WVU Eberly College of Arts and Sciences, have designed an interdisciplinary, cross-institutional program to facilitate conversations among faculty about the benefits and drawbacks of AI, how it functions and the need for human interpretation.
The award will fund a summer workshop in which Carlson and Davidson will offer AI trainings for humanities faculty and guide them through creation and development of courses with an AI component. The researchers will then assist as faculty offer those courses to students, assess progress and help with the implementation of the projects that develop.
The researchers said they hope to challenge the notion that artificial intelligence research falls into the domain of STEM fields.
“The humanities gets overlooked and underappreciated so often,” Carlson said. “We are doing important, meaningful research, just like our colleagues in STEM and other fields. This is a chance to use a humanities lens to examine contemporary problems and developments like artificial intelligence and also to get conversations going between fields that oftentimes don’t talk to one another as much as we should.”
Co-directors Carlson and Davidson will be joined by a team of mentors and fellows — two from data science fields and two from the humanities perspective — that will serve and assist as resources in the interdisciplinary conversations. The seminar and summer workshops will support the creation or redesign of 10 courses. They plan to invite off-campus experts to help facilitate the workshops, work with the faculty and support their projects.
“It’s really about expanding capacity at the University and in the humanities to investigate the implications of AI or to actually use AI in humanities courses, whether it’s for writing, creating art or creating projects through the use of AI,” Davidson said. “There are a lot of different possibilities and directions that we hope these courses take. If we have 10 of them, it’s really going to have a big impact on humanities education here at the University.”
Carlson and Davidson acknowledge that attitudes about AI tend to be either extremely optimistic or extremely skeptical but that the reality is somewhere in the middle.
“AI is such a simplistic term to describe a whole suite of different technologies and developments that folks are dealing with every day, whether they know it or not,” Carlson said, noting that discussions could focus on personal, social and economic impacts of AI use, as well as how it affects character and intellectual values.
Davidson was inspired to focus on AI when he found an erroneous, AI-generated summary of one of his own articles.
“It was totally wrong,” he said. “I didn’t say those things, and it made me think about how somebody might look me up and find that summary of my article and get this false impression of me. That really highlighted that we need to build an understanding in students of the need to inquire deeper and to understand that you have to be able to evaluate AI’s accuracy and its reliability.”
Carlson and Davidson said the conversations need to consider AI’s drawbacks, as well. Using AI consumes large amounts of water and electricity resulting in greenhouse emissions. Data centers produce electronic waste that can contain mercury and lead.
They also intend to follow legal cases and precedents surrounding the use of AI.
“That’s another aspect of AI and the ways that it represents people,” Carlson said. “Because it has a very real, material impact on people in communities. It’s not just a super computer in a room. It’s a network that has a bunch of different implications for a bunch of different people, ranging from jobs to familial relationships. That’s the value of the humanities — to ask these tough questions because it’s increasingly difficult to avoid all of it.”
Conversations, as they expand, will need to keep up with the pace of AI’s rapidly developing landscape.
“There’s going to be a lot of people involved in this,” she said. “We put together an amazing team. We want it to be an open, honest and ethical conversation that brings in other folks and opens up further conversations across the College and the University at large.”
Trump clusters: How an English lit graduate used AI to make sense of Twitter bios
Analyzing social trends, disaster responses or customer insights using large language models to organize short text clusters just got easier.
University of Sydney
An English literature graduate turned data scientist has developed a new method for large language models (LLMs) used by AI chatbots to understand and analyse small chunks of text, such as those on social media profiles, in customer responses online or for understanding online posts responding to disaster events.
In today’s digital world, such use of short text has become central to online communication. However, analysing these snippets is challenging because they often lack shared words or context. This lack of context makes it difficult for AI to find patterns or group similar texts.
The new research addresses the problem by using large language models (LLMs) to group large datasets of short text into clusters. These clusters condense potentially millions of tweets or comments into easy-to-understand groups generated by the model.
PhD student Justin Miller has developed this method for use by AI programs that successfully produced coherent categories after analysing nearly 40,000 Twitter (X) user biographies from accounts tweeting about US President Donald Trump over two days in September 2020.
The language model developed by Mr Miller, an English literature graduate, clustered the biographies into 10 categories, and allocated scores within each of these categories to assist in analysing the likely occupation of the tweeters, their political leaning, or even their use of emojis.
The study is published in the Royal Society Open Science journal.
Mr Miller said: “What makes this study stand out is its focus on human-centred design. The clusters created by the large language models are not only computationally effective but also make sense to people.
“For instance, texts about family, work, or politics are grouped in ways that humans can intuitively name and understand. Furthermore, the research shows that generative AI, such as ChatGPT, can mimic how humans interpret these clusters.
“In some cases, the AI provided clearer and more consistent cluster names than human reviewers, particularly when distinguishing meaningful patterns from background noise.”
Mr Miller, a doctoral candidate in the School of Physics and a member of the Computational Social Sciences lab, said the tool he has developed could be used to simplify large datasets, gain insights for decision making and improve search and organisation.
Using large language models (LLMs), the authors created clusters using a methodology known as “Gaussian mixture modelling” that capture the essence of the text and are easier for humans to understand. They validated these clusters by comparing human interpretations with those from a generative LLM, which closely matched human reviews.
This approach not only improved clustering quality but also suggests that human reviews, while valuable, might not be the only standard for cluster validation.
Mr Miller said: “Large datasets, which would be impossible to manually read, can be reduced into meaningful, manageable groups.”
Applications include:
- Simplifying Large Datasets: Large datasets, which would be impossible to manually read, can be reduced into meaningful, manageable groups. For example, Mr Miller applied the same methods from this paper to another project on the Russia-Ukraine war. By clustering over 1 million social media posts, he identified 10 distinct topics, including Russian disinformation campaigns, the use of animals as symbols in humanitarian relief, and Azerbaijan’s attempts to showcase its support for Ukraine.
- Gain Insights for Decision-Making: Clusters provide actionable insights for organisations, governments and businesses. A business might use clustering to identify what customers like or dislike about their product, while governments could use it to condense wide ranging public sentiment into a few topics.
- Improve Search and Organisation: For platforms handling large volumes of user-generated content, clustering makes it easier to organise, filter and retrieve relevant information. This method can help users quickly find what they’re looking for and improve overall content management.
Mr Miller said: “This dual use of AI for clustering and interpretation opens up significant possibilities. By reducing reliance on costly and subjective human reviews, it offers a scalable way to make sense of massive amounts of text data. From social media trend analysis to crisis monitoring or customer insights, this approach combines machine efficiency with human understanding to organise and explain data effectively.”
-ENDS-
Interviews
Justin Miller | justin.k.miller@sydney.edu.au
Media enquiries
Marcus Strom | marcus.strom@sydney.edu.au | +61 474 269 459
Research
Miller, J. and Alexander, T. ‘Human-interpretable clustering of short text using large language models’ (Royal Society Open Science 2025) DOI: 10.1098/rsos.241692
Declaration: The researchers declare no conflicts.
Outside of work hours, please call +61 2 8627 0246 (this directs to a mobile number) or email media.office@sydney.edu.au.
Journal
Royal Society Open Science
Method of Research
Data/statistical analysis
Subject of Research
Not applicable
Article Title
Human-interpretable clustering of short text using large language models
Article Publication Date
21-Jan-2025
No comments:
Post a Comment