AI ‘hallucinations’ tackled by University of Bristol researchers

DO ANDROIDS DREAM OF ELECTRIC SHEEP

University of Bristol

Significant strides in addressing the issue of AI 'hallucinations' and improving the reliability of anomaly detection algorithms in Critical National Infrastructures (CNI) have been made by scientists based in Bristol’s School of Computer Science.

Recent advances in Artificial Intelligence (AI) have highlighted the technology's potential in anomaly detection, particularly within sensor and actuator data for CNIs. However, these AI algorithms often require extensive training times and struggle to pinpoint specific components in an anomalous state. Furthermore, AI's decision-making processes are frequently opaque, leading to concerns about trust and accountability.

To help combat this, the team brought in a number of measures to boost efficiency including:

1. Enhanced Anomaly Detection: Researchers employed two cutting-edge anomaly detection algorithms with significantly shorter training times and faster detection capabilities, while maintaining comparable efficiency rates. These algorithms were tested using a dataset from the operational water treatment testbed, SWaT, at the Singapore University of Technology and Design.
2. Explainable AI (XAI) Integration: To enhance transparency and trust, the team integrated eXplainable AI (XAI) models with the anomaly detectors. This approach allows for better interpretation of AI decisions, enabling human operators to understand and verify AI recommendations before making critical decisions. The effectiveness of various XAI models was also evaluated, providing insights into which models best aid human understanding.
3. Human-Centric Decision Making: The research emphasizes the importance of human oversight in AI-driven decision-making processes. By explaining AI recommendations to human operators, the team aims to ensure that AI acts as a decision-support tool rather than an unquestioned oracle. This methodology introduces accountability, as human operators make the final decisions based on AI insights, policy, rules, and regulations.
4. Scoring System Development: A meaningful scoring system is being developed to measure the perceived correctness and confidence of the AI's explanations. This score aims to assist human operators in gauging the reliability of AI-driven insights.

These advancements not only improve the efficiency and reliability of AI systems in CNIs but also ensure that human operators remain integral to the decision-making process, enhancing overall accountability and trust.

Dr Sarad Venugopalan, co-author of the study, explained: “Humans learn by repetition over a longer period of time and work for shorter hours without being error prone. This is why, in some cases, we use machines that can carry out the same tasks in a fraction of the time and at a reduced error rate.

“However, this automation, involving cyber and physical components, and subsequent use of AI to solve some of the issues brought by the automation, is treated as a black box.  This is detrimental because it is the personnel using the AI recommendation that is held accountable for the decisions made by them, and not the AI itself.

“In our work, we use explainable AI, to increase transparency and trust, so the personnel using the AI is informed why the AI made the recommendation (for our domain use case) before a decision is made."

This research is part of the MSc thesis of Mathuros Kornkamon, under the supervision of Dr Sridhar Adepu.

Dr Adepu added: “This work discovers how WaXAI is revolutionizing anomaly detection in industrial systems with explainable AI. By integrating XAI, human operators gain clear insights and enhanced confidence to handle security incidents in critical infrastructure."

The paper won Best Paper Award at the 10th ACM Cyber-Physical System Security Workshop at the ACM ASIACCS2024 conference.

Paper:

‘WaXAI: Explainable Anomaly Detection in Industrial Control Systems and Water Systems’ by Mathuros Kornkamon, Sarad Venugopalan and Sridhar Adepu in Proceedings of the 10th ACM Cyber-Physical System Security Workshop, pp. 3-15. 2024

---

Method of Research

Experimental study

Subject of Research

Not applicable

Article Title

WaXAI: Explainable Anomaly Detection in Industrial Control Systems and Water Systems

Article Publication Date

31-Jul-2024