AI NEWS
AI networks are more vulnerable to malicious attacks than previously thought
Artificial intelligence tools hold promise for applications ranging from autonomous vehicles to the interpretation of medical images. However, a new study finds these AI tools are more vulnerable than previously thought to targeted attacks that effectively force AI systems to make bad decisions.
At issue are so-called “adversarial attacks,” in which someone manipulates the data being fed into an AI system in order to confuse it. For example, someone might know that putting a specific type of sticker at a specific spot on a stop sign could effectively make the stop sign invisible to an AI system. Or a hacker could install code on an X-ray machine that alters the image data in a way that causes an AI system to make inaccurate diagnoses.
“For the most part, you can make all sorts of changes to a stop sign, and an AI that has been trained to identify stop signs will still know it’s a stop sign,” says Tianfu Wu, co-author of a paper on the new work and an associate professor of electrical and computer engineering at North Carolina State University. “However, if the AI has a vulnerability, and an attacker knows the vulnerability, the attacker could take advantage of the vulnerability and cause an accident.”
The new study from Wu and his collaborators focused on determining how common these sorts of adversarial vulnerabilities are in AI deep neural networks. They found that the vulnerabilities are much more common than previously thought.
“What’s more, we found that attackers can take advantage of these vulnerabilities to force the AI to interpret the data to be whatever they want,” Wu says. “Using the stop sign example, you could make the AI system think the stop sign is a mailbox, or a speed limit sign, or a green light, and so on, simply by using slightly different stickers – or whatever the vulnerability is.
“This is incredibly important, because if an AI system is not robust against these sorts of attacks, you don’t want to put the system into practical use – particularly for applications that can affect human lives.”
To test the vulnerability of deep neural networks to these adversarial attacks, the researchers developed a piece of software called QuadAttacK. The software can be used to test any deep neural network for adversarial vulnerabilities.
“Basically, if you have a trained AI system, and you test it with clean data, the AI system will behave as predicted. QuadAttacK watches these operations and learns how the AI is making decisions related to the data. This allows QuadAttacK to determine how the data could be manipulated to fool the AI. QuadAttacK then begins sending manipulated data to the AI system to see how the AI responds. If QuadAttacK has identified a vulnerability it can quickly make the AI see whatever QuadAttacK wants it to see.”
In proof-of-concept testing, the researchers used QuadAttacK to test four deep neural networks: two convolutional neural networks (ResNet-50 and DenseNet-121) and two vision transformers (ViT-B and DEiT-S). These four networks were chosen because they are in widespread use in AI systems around the world.
“We were surprised to find that all four of these networks were very vulnerable to adversarial attacks,” Wu says. “We were particularly surprised at the extent to which we could fine-tune the attacks to make the networks see what we wanted them to see.”
The research team has made QuadAttacK publicly available, so that the research community can use it themselves to test neural networks for vulnerabilities. The program can be found here: https://thomaspaniagua.github.io/quadattack_web/.
“Now that we can better identify these vulnerabilities, the next step is to find ways to minimize those vulnerabilities,” Wu says. “We already have some potential solutions – but the results of that work are still forthcoming.”
The paper, “QuadAttacK: A Quadratic Programming Approach to Learning Ordered Top-K Adversarial Attacks,” will be presented Dec. 16 at the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023), which is being held in New Orleans, La. First author of the paper is Thomas Paniagua, a Ph.D. student at NC State. The paper was co-authored by Ryan Grainger, a Ph.D. student at NC State.
The work was done with support from the U.S. Army Research Office, under grants W911NF1810295 and W911NF2210010; and from the National Science Foundation, under grants 1909644, 2024688 and 2013451.
METHOD OF RESEARCH
Computational simulation/modeling
SUBJECT OF RESEARCH
Not applicable
ARTICLE TITLE
QuadAttacK: A Quadratic Programming Approach to Learning Ordered Top-K Adversarial Attacks
AI helps us better understand (and protect) forests
The coordinated work of over 150 scientists, complemented by the substantial computational capabilities of AI, seeks to enhance our understanding of forests. The primary goal is to investigate the evolving nature of these ecosystems and develop effective measures for their conservation. The results of this challenging research – promoted by the researchers of the Global Forest Biodiversity Initiative – have been published in three scientific articles. Two of these articles have been published in Nature, while the third finds its place in Nature Plants.
“Through the collaborative efforts of hundreds of researchers science has achieved a significant advancement in the understanding of the world’s forest ecology and in emphasising the need to protect them. Without the supercomputer’s computing power and AI, we would have needed decades and the workforce of thousands, all without the certainty of obtaining reliable estimates.” says Roberto Cazzolla Gatti, professor at the Department of Biological, Geological, and Environmental Sciences of the University of Bologna and co-author of the three studies.
ALIEN SPECIES AND NATIVE DIVERSITY
The first topic that the researchers focused on was the invasion of non-native trees: this is an essential phenomenon to understand, to safeguard native ecosystems and limit the spreading of invasive species. Which factors are triggering and facilitating this process?
By analysing trees’ databases at a global level, researchers determined that the temperature and the precipitations serves as strong predictors of the invasion strategy: non-native species effectively invade an area when their environmental preferences align with those of the native community in conditions of extreme cold or drought.
However, there is another element which facilitates the diffusion of invasive species even more: human activity, particularly in environments such as managed forests or near roads and seaports.
“The invasion of a place by non-native trees is not only predicted by anthropic factors, but its gravity is also ruled by native diversity: a greater diversity reduces the gravity of the invasion. Rather than combating alien species when it's already too late, our focus should be on safeguarding the health of forests. This proactive approach, similar to other ecosystems, would pose greater challenges for alien species to spread and invade”, says Professor Cazzolla Gatti.
CARBON SINKS?
Nowadays we recognise the crucial role of protecting forests in preserving their capacity to capture carbon dioxide and serve as earth’s carbon sink. However, what is the global potential of carbon that forests can store?
The second study, published in Nature, focuses on that topic, and reveals that currently the global forest carbon is considerably below the natural potential. Almost 61% of this potential is located in areas hosting existing forests, where the safeguard of the ecosystem allows them to recover until reaching maturity. Instead, the remaining 39% is located outside urban and agricultural lands, but in regions in which forests have been either removed or fragmented.
“Forests alone cannot substitute the necessary reduction of CO2 emissions in the atmosphere. However, our results support the idea that the conservation, recovery, and sustainable management of different forests can offer precious contributions to reach the global goals for climate protection and biodiversity.
For the first time, we were able to verify that, despite the regional variations, predictions on global scale exhibit remarkable coherence, with only a 12% difference among the estimates obtained from the ground and those derived from the satellite. So, forests serve as a major earth’s carbon sink, however anthropogenic changes in climate and land use reduce their absorption capacity”, explains Professor Cazzolla Gatti.
LEAF TYPES AND CLIMATE CHANGE
While conducting research and analysis, researchers have gone even further. They have tried to understand in detail the factors that influence the global variation in tree leaves and the role of tree species in terrestrial ecosystems, including the cycle of carbon, water, and nutrients.
As a result, researchers have discovered that the global variation between evergreen and deciduous trees is mainly caused by isothermy and soil characteristics, while the type of leaves is determined by temperature. In particular, their estimate reveal that 38% of the world’s trees are evergreen with needle-shaped leaves, while 29% are broadleaved and 27% are broadleaved deciduous and 5% have needle-shaped leaves.
Professor Cazzolla Gatti explains, “Depending on future greenhouse gas emissions, by the end of the century, 17 to 34 percent of forest areas may undergo climate conditions that currently support a different type of forest: up to a third of the earth’s green areas will likely experience intense climate stress. The results of this study can improve the predictions on the functioning of forest ecosystems and the carbon cycle by quantifying the distribution of tree leaf types and corresponding biomass, identifying the areas in which climate change will exert the greater pressure on the current leaf types.”
JOURNAL
Nature
ARTICLE TITLE
Native diversity buffers against severity of non-native tree invasions
Using AI to find microplastics
Researchers use AI to identify toxic substances in wastewater with greater accuracy and speed
An interdisciplinary research team from the University of Waterloo is using artificial intelligence (AI) to identify microplastics faster and more accurately than ever before.
Microplastics are commonly found in food and are dangerous pollutants that cause severe environmental damage – finding them is the key to getting rid of them.
The research team’s advanced imaging identification system could help wastewater treatment plants and food production industries make informed decisions to mitigate the potential impact of microplastics on the environment and human health.
A comprehensive risk analysis and action plan requires quality information based on accurate identification. In search of a robust analytical tool that could enumerate, identify and describe the many microplastics that exist, project lead Dr. Wayne Parker and his team, employed an advanced spectroscopy method which exposes particles to a range of wavelengths of light. Different types of plastics produce different signals in response to the light exposure. These signals are like fingerprints that can also be employed to mark particles as microplastic or not.
The challenge researchers often find is that microplastics come in wide varieties due to the presence of manufacturing additives and fillers that can blur the “fingerprints” in a lab setting. This makes identifying microplastics from organic material, as well as the different types of microplastics, often difficult. Human intervention is usually required to dig out subtle patterns and cues, which is slow and prone to error.
“Microplastics are hydrophobic materials that can soak up other chemicals,” said Parker, a professor in Waterloo’s Department of Civil and Environmental Engineering. “Science is still evolving in terms of how bad the problem is, but it’s theoretically possible that microplastics are enhancing the accumulation of toxic substances in the food chain.”
Parker approached Dr. Alexander Wong, a professor in Waterloo’s Department of Systems Design Engineeringand the Canada Research Chair in Artificial Intelligence and Medical Imaging for assistance. With his help, the team developed an AI tool called PlasticNet that enables researchers to rapidly analyze large numbers of particles approximately 50 per cent faster than prior methods and with 20 per cent more accuracy.
The tool is the latest sustainable technology designed by Waterloo researchers to protect our environment and engage in research that will contribute to a sustainable future.
“We built a deep learning neural network to enhance microplastic identification from the spectroscopic signals,” said Wong. “We trained it on data from existing literature sources and our own generated images to understand the varied make-up of microplastics and spot the differences quickly and correctly— regardless of the fingerprint quality.”
Parker’s former PhD student, Frank Zhu, tested the system on microplastics isolated from a local wastewater treatment plant. Results show that it can identify microplastics with unprecedented speed and accuracy. This information can empower treatment plants to implement effective measures to control and eliminate these substances.
The next steps involve continued learning and testing, as well as feeding the PlasticNet system more data to increase the quality of its microplastics identification capabilities for application across a broad range of needs.
More information about this work can be found in the research paper, “Leveraging deep learning for automatic recognition of microplastics (MPs) via focal plane array (FPA) micro-FT-IR imaging”, published in Environmental Pollution.
JOURNAL
Environmental Pollution
Enhanced AI tracks neurons in moving animals
VIDEO:
TWO-DIMENSIONAL PROJECTION OF 3D VOLUMETRIC BRAIN ACTIVITY RECORDINGS IN C. ELEGANS. GREEN: GENETICALLY ENCODED CALCIUM INDICATOR, VARIOUS COLORS: SEGMENTED AND TRACKED NEURONS.
view moreCREDIT: MAHSA BARZEGAR-KESHTELI (EPFL)
Recent advances allow imaging of neurons inside freely moving animals. However, to decode circuit activity, these imaged neurons must be computationally identified and tracked. This becomes particularly challenging when the brain itself moves and deforms inside an organism’s flexible body, e.g. in a worm. Until now, the scientific community has lacked the tools to address the problem.
Now, a team of scientists from EPFL and Harvard have developed a pioneering AI method to track neurons inside moving and deforming animals. The study, now published in Nature Methods, was led by Sahand Jamal Rahi at EPFL’s School of Basic Sciences.
The new method is based on a convolutional neural network (CNN), which is a type of AI that has been trained to recognize and understand patterns in images. This involves a process called “convolution”, which looks at small parts of the picture – like edges, colors, or shapes – at a time and then combines all that information together to make sense of it and to identify objects or patterns.
The problem is that to identify and track neurons during a movie of an animal’s brain, many images have to be labeled by hand because the animal appears very differently across time due to the many different body deformations. Given the diversity of the animal’s postures, generating a sufficient number of annotations manually to train a CNN can be daunting.
To address this, the researchers developed an enhanced CNN featuring ‘targeted augmentation’. The innovative technique automatically synthesizes reliable annotations for reference out of only a limited set of manual annotations. The result is that the CNN effectively learns the internal deformations of the brain and then uses them to create annotations for new postures, drastically reducing the need for manual annotation and double-checking.
The new method is versatile, being able to identify neurons whether they are represented in images as individual points or as 3D volumes. The researchers tested it on the roundworm Caenorhabditis elegans, whose 302 neurons have made it a popular model organism in neuroscience.
Using the enhanced CNN, the scientists measured activity in some of the worm’s interneurons (neurons that bridge signals between neurons). They found that they exhibit complex behaviors, for example changing their response patterns when exposed to different stimuli, such as periodic bursts of odors.
The team have made their CNN accessible, providing a user-friendly graphical user interface that integrates targeted augmentation, streamlining the process into a comprehensive pipeline, from manual annotation to final proofreading.
“By significantly reducing the manual effort required for neuron segmentation and tracking, the new method increases analysis throughput three times compared to full manual annotation,” says Sahand Jamal Rahi. “The breakthrough has the potential to accelerate research in brain imaging and deepen our understanding of neural circuits and behaviors.”
Other contributors
Swiss Data Science Center
JOURNAL
Nature Methods
ARTICLE TITLE
Automated neuron tracking inside moving and deforming animals using deep learning and targeted augmentation.
ARTICLE PUBLICATION DATE
5-Dec-2023
