Liquid metal sensors and AI could help prosthetic hands to 'feel'

Study first to use liquid metal sensors and machine learning on a prosthetic hand

FLORIDA ATLANTIC UNIVERSITY

 VIDEO: RESEARCHERS USED INDIVIDUAL FINGERTIPS FITTED WITH STRETCHABLE TACTILE SENSORS WITH LIQUID METAL ON A PROSTHESIS TO DISTINGUISH BETWEEN DIFFERENT SPEEDS OF A SLIDING MOTION ALONG DIFFERENT TEXTURED SURFACES. THE FOUR... view more 

CREDIT: FLORIDA ATLANTIC UNIVERSITY/COLLEGE OF ENGINEERING AND COMPUTER SCIENCE

Each fingertip has more than 3,000 touch receptors, which largely respond to pressure. Humans rely heavily on sensation in their fingertips when manipulating an object. The lack of this sensation presents a unique challenge for individuals with upper limb amputations. While there are several high-tech, dexterous prosthetics available today - they all lack the sensation of "touch." The absence of this sensory feedback results in objects inadvertently being dropped or crushed by a prosthetic hand.

To enable a more natural feeling prosthetic hand interface, researchers from Florida Atlantic University's College of Engineering and Computer Science and collaborators are the first to incorporate stretchable tactile sensors using liquid metal on the fingertips of a prosthetic hand. Encapsulated within silicone-based elastomers, this technology provides key advantages over traditional sensors, including high conductivity, compliance, flexibility and stretchability. This hierarchical multi-finger tactile sensation integration could provide a higher level of intelligence for artificial hands.

For the study, published in the journal Sensors, researchers used individual fingertips on the prosthesis to distinguish between different speeds of a sliding motion along different textured surfaces. The four different textures had one variable parameter: the distance between the ridges. To detect the textures and speeds, researchers trained four machine learning algorithms. For each of the ten surfaces, 20 trials were collected to test the ability of the machine learning algorithms to distinguish between the ten different complex surfaces comprised of randomly generated permutations of four different textures.

Results showed that the integration of tactile information from liquid metal sensors on four prosthetic hand fingertips simultaneously distinguished between complex, multi-textured surfaces - demonstrating a new form of hierarchical intelligence. The machine learning algorithms were able to distinguish between all the speeds with each finger with high accuracy. This new technology could improve the control of prosthetic hands and provide haptic feedback, more commonly known as the experience of touch, for amputees to reconnect a previously severed sense of touch.

"Significant research has been done on tactile sensors for artificial hands, but there is still a need for advances in lightweight, low-cost, robust multimodal tactile sensors," said Erik Engeberg, Ph.D., senior author, an associate professor in the Department of Ocean and Mechanical Engineering and a member of the FAU Stiles-Nicholson Brain Institute and the FAU Institute for Sensing and Embedded Network Systems Engineering (I-SENSE), who conducted the study with first author and Ph.D. student Moaed A. Abd. "The tactile information from all the individual fingertips in our study provided the foundation for a higher hand-level of perception enabling the distinction between ten complex, multi-textured surfaces that would not have been possible using purely local information from an individual fingertip. We believe that these tactile details could be useful in the future to afford a more realistic experience for prosthetic hand users through an advanced haptic display, which could enrich the amputee-prosthesis interface and prevent amputees from abandoning their prosthetic hand."

Researchers compared four different machine learning algorithms for their successful classification capabilities: K-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), and neural network (NN). The time-frequency features of the liquid metal sensors were extracted to train and test the machine learning algorithms. The NN generally performed the best at the speed and texture detection with a single finger and had a 99.2 percent accuracy to distinguish between ten different multi-textured surfaces using four liquid metal sensors from four fingers simultaneously.

"The loss of an upper limb can be a daunting challenge for an individual who is trying to seamlessly engage in regular activities," said Stella Batalama, Ph.D., dean, College of Engineering and Computer Science. "Although advances in prosthetic limbs have been beneficial and allow amputees to better perform their daily duties, they do not provide them with sensory information such as touch. They also don't enable them to control the prosthetic limb naturally with their minds. With this latest technology from our research team, we are one step closer to providing people all over the world with a more natural prosthetic device that can 'feel' and respond to its environment."

CAPTION

Researchers used individual fingertips fitted with stretchable tactile sensors with liquid metal on a prosthesis attached to a robotic arm.

CREDIT

Alex Dolce, Florida Atlantic University 

Study co-authors are Rudy Paul, FAU Department of Ocean and Mechanical Engineering; Aparna Aravelli, Ph.D.; Ou Bai, Ph.D.; and Leonel Lagos, Ph.D., PMP, all with Florida International University; and Maohua Lin, Ph.D., FAU Department of Ocean and Mechanical Engineering.

The research was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (NIH) and the National Institute of Aging of the NIH, the National Science Foundation, the Department of Energy and pilot grants from the FAU Stiles-Nicholson Brain Institute and FAU I-SENSE.

About FAU's College of Engineering and Computer Science:

The FAU College of Engineering and Computer Science is internationally recognized for cutting edge research and education in the areas of computer science and artificial intelligence (AI), computer engineering, electrical engineering, bioengineering, civil, environmental and geomatics engineering, mechanical engineering, and ocean engineering. Research conducted by the faculty and their teams expose students to technology innovations that push the current state-of-the art of the disciplines. The College research efforts are supported by the National Science Foundation (NSF), the National Institutes of Health (NIH), the Department of Defense (DOD), the Department of Transportation (DOT), the Department of Education (DOEd), the State of Florida, and industry. The FAU College of Engineering and Computer Science offers degrees with a modern twist that bear specializations in areas of national priority such as AI, cybersecurity, internet-of-things, transportation and supply chain management, and data science. New degree programs include Masters of Science in AI (first in Florida), Masters of Science in Data Science and Analytics, and the new Professional Masters of Science degree in computer science for working professionals. For more information about the College, please visit eng.fau.edu.

About Florida Atlantic University:

Florida Atlantic University, established in 1961, officially opened its doors in 1964 as the fifth public university in Florida. Today, the University serves more than 30,000 undergraduate and graduate students across six campuses located along the southeast Florida coast. In recent years, the University has doubled its research expenditures and outpaced its peers in student achievement rates. Through the coexistence of access and excellence, FAU embodies an innovative model where traditional achievement gaps vanish. FAU is designated a Hispanic-serving institution, ranked as a top public university by U.S. News & World Report and a High Research Activity institution by the Carnegie Foundation for the Advancement of Teaching. For more information, visit http://www.fau.edu.