TELL THE GOP
An ‘introspective’ AI finds diversity improves performance
An artificial intelligence with the ability to look inward and fine tune its own neural network performs better when it chooses diversity over lack of diversity, a new study finds. The resulting diverse neural networks were particularly effective at solving complex tasks.
“We created a test system with a non-human intelligence, an artificial intelligence (AI), to see if the AI would choose diversity over the lack of diversity and if its choice would improve the performance of the AI,” says William Ditto, professor of physics at North Carolina State University, director of NC State’s Nonlinear Artificial Intelligence Laboratory (NAIL) and co-corresponding author of the work. “The key was giving the AI the ability to look inward and learn how it learns.”
Neural networks are an advanced type of AI loosely based on the way that our brains work. Our natural neurons exchange electrical impulses according to the strengths of their connections. Artificial neural networks create similarly strong connections by adjusting numerical weights and biases during training sessions. For example, a neural network can be trained to identify photos of dogs by sifting through a large number of photos, making a guess about whether the photo is of a dog, seeing how far off it is and then adjusting its weights and biases until they are closer to reality.
Conventional AI uses neural networks to solve problems, but these networks are typically composed of large numbers of identical artificial neurons. The number and strength of connections between those identical neurons may change as it learns, but once the network is optimized, those static neurons are the network.
Ditto’s team, on the other hand, gave its AI the ability to choose the number, shape and connection strength between neurons in its neural network, creating sub-networks of different neuron types and connection strengths within the network as it learns.
“Our real brains have more than one type of neuron,” Ditto says. “So we gave our AI the ability to look inward and decide whether it needed to modify the composition of its neural network. Essentially, we gave it the control knob for its own brain. So it can solve the problem, look at the result, and change the type and mixture of artificial neurons until it finds the most advantageous one. It’s meta-learning for AI.
“Our AI could also decide between diverse or homogenous neurons,” Ditto says. “And we found that in every instance the AI chose diversity as a way to strengthen its performance.”
The team tested the AI’s accuracy by asking it to perform a standard numerical classifying exercise, and saw that its accuracy increased as the number of neurons and neuronal diversity increased. A standard, homogenous AI could identify the numbers with 57% accuracy, while the meta-learning, diverse AI was able to reach 70% accuracy.
According to Ditto, the diversity-based AI is up to 10 times more accurate than conventional AI in solving more complicated problems, such as predicting a pendulum’s swing or the motion of galaxies.
“We have shown that if you give an AI the ability to look inward and learn how it learns it will change its internal structure – the structure of its artificial neurons – to embrace diversity and improve its ability to learn and solve problems efficiently and more accurately,” Ditto says. “Indeed, we also observed that as the problems become more complex and chaotic the performance improves even more dramatically over an AI that does not embrace diversity.”
The research appears in Scientific Reports, and was supported by the Office of Naval Research (under grant N00014-16-1-3066) and by United Therapeutics. John Lindner, emeritus professor of physics at the College of Wooster and visiting professor at NAIL, is co-corresponding author. Former NC State graduate student Anshul Choudhary is first author. NC State graduate student Anil Radhakrishnan and Sudeshna Sinha, professor of physics at the Indian Institute of Science Education and Research Mohali, also contributed to the work.
-peake-
Note to editors: An abstract follows.
“Neuronal diversity can improve machine learning for physics and beyond”
DOI: 10.1038/s41598-023-40766-6
Authors: Anshul Choudhary, Anil Radhakrishnan, John F. Lindner, William L. Ditto, North Carolina State University Nonlinear Artificial Intelligence Laboratory; Sudeshna Sinha, Indian Institute of Science Education and Research Mohali
Published: Aug. 21, 2023 in Scientific Reports
Abstract:
Diversity conveys advantages in nature, yet homogeneous neurons typically comprise the layers of artificial neural networks. Here we construct neural networks from neurons that learn their own activation functions, quickly diversify, and subsequently outperform their homogeneous counterparts on image classification and nonlinear regression tasks. Sub-networks instantiate the neurons, which meta-learn especially efficient sets of nonlinear responses. Examples include conventional neural networks classifying digits and forecasting a van der Pol oscillator and physics-informed Hamiltonian neural networks learning Hénon–Heiles stellar orbits and the swing of a video recorded pendulum clock. Such learned diversity provides examples of dynamical systems selecting diversity over uniformity and elucidates the role of diversity in natural and artificial systems.
JOURNAL
Scientific Reports
METHOD OF RESEARCH
Computational simulation/modeling
SUBJECT OF RESEARCH
Not applicable
ARTICLE TITLE
Neuronal diversity can improve machine learning for physics and beyond
Amsterdam UMC is building models to enable greater use of AI in the health care system
80% of all patient data is unstructured. Notes from a conversation with a GP, the evaluation of a specialist in a university medical centre or even a recommendation from a pharmacist. While this 'unstructured’ data is no problem for the human eye, it presents an unsurmountable challenge to an AI-algorithm. One that is "preventing AI from reaching its full potential," in the view of Amsterdam UMC, Assistant Professor Iacer Calixto. To give AI the helping hand that it needs, Calixto is set to lead a project that will "tackle the important challenges that hinder its use in clinical practice,” thanks to funding from the NWO.
"We need to devise methods that are human-centred and responsible by design if we want these methods to be implemented in practice," says Calixto. The project will build on Natural Language Processing (NLP) techniques that already underpin the increasingly popular, ChatGPT. Currently, the unstructured nature of this data means that software such as ChatGPT cannot be easily used in the health care sector. However, the software itself offers plenty of opportunities for the sector. With promises to improve data entry, decision making and to free up crucial time that doctors and nurses can instead spend on patient care.
Ensuring Privacy is Maintained
"Protecting the privacy of our patients is a top priority at Amsterdam UMC, and that isn't different when we are developing, testing or using AI-algorithms," says Mat Daemen, vice-dean of Research at Amsterdam UMC.
To ensure that AI can also be used in a safe way, this project will also address issues relating to privacy. By developing new 'synthetic' patient records, based around simulated information. These records mimic real patient records, in order to facilitate healthcare and research, while protecting the information of the 'real' patients.
"One of the main bottlenecks of doing research in healthcare is access to high-quality data to train and validate machine learning models. Part of our project will generate synthetic patient records that include not only structured but also unstructured data such as free-text highlights from a consultation with a GP. These synthetic records, though not from real patients, can still be very useful to enable easier access to high-quality healthcare data for researchers and clinicians," says Calixto.
Responsibly Dutch
Another sticking point for the use of AI in the Dutch health sector, is a rather more self-evident one: language. Software such as ChatGPT are built on language databases, and these are predominantly in English. By building new models that are trained on Dutch medical records, the project will increase the reliability of existing tools as well as making them easier to use for professionals on the wards or in the treatment room.
This is a bold project that will ensure the Amsterdam UMC is one of the forces driving innovation in healthcare with artificial intelligence and natural language processing. Results obtained in this project, for instance, synthetic patient records will benefit the entire Dutch healthcare ecosystem, including other hospitals and university medical centres, says Calixto.
The responsibility of this AI project is not only limited to the important goal of maintaining patient privacy. The project will also seek to remove any aspects of discrimination and unfairness that may exist in existing AI models. For Daemen, this is an essential condition for the use of AI in Amsterdam UMC, and something that this project has at its core. "This project is an important addition to the efforts of many experts in Amsterdam UMC and in the Amsterdam region to introduce and use AI tools in a human centred and responsible way," he concludes.
No comments:
Post a Comment