Robots can be more aware of human co-workers, with system that provides context

KTH, ROYAL INSTITUTE OF TECHNOLOGY

VIDEO: KTH RESEARCHER HONGYI LIU TESTS A ROBOT ARM BY PLACING HIS HAND IN ITS PATH. view more 

CREDIT: HONGYI LIU

Working safely is not only about processes, but context - understanding the work environment and circumstances, and being able to predict what other people will do next. A new system empowers robots with this level of context awareness, so they can work side-by-side with humans on assembly lines more efficiently and without unnecessary interruptions.

Instead of being able to only judge distance between itself and its human co-workers, the human-robot collaboration system can identify each worker it works with, as well as the person's skeleton model, which is an abstract of body volume, says Hongyi Liu, a researcher at KTH Royal Institute of Technology. Using this information, the context-aware robot system can recognize the worker's pose and even predict the next pose. These abilities provide the robot with a context to be aware of while interacting.

Liu says that the system operates with artificial intelligence that requires less computational power and smaller datasets than traditional machine learning methods. It relies instead on a form of machine learning called transfer learning - which reuses knowledge developed through training before being adapted into an operational model.

The research was published in the recent issue of Robotics and Computer-Integrated Manufacturing, and was co-authored by KTH Professor Lihui Wang.

Liu says that the technology is out ahead of today's International Organization for Standards (ISO) requirements for collaborative robot safety, so implementation of the technology would require industrial action. But the context awareness offers better efficiency than the one-dimensional interaction workers now experience with robots, he says.

"Under the ISO standard and technical specification, when a human approaches a robot it slows down, and if he or she comes close enough it will stop. If the person moves away it resumes. That's a pretty low level of context awareness," he says.

"It jeopardizes efficiency. Production is slowed and humans cannot work closely to robots."

Liu compares the context-aware robot system to a self-driving car that recognizes how long a stoplight has been red and anticipates moving again. Instead of braking or downshifting, it begins to adjust its speed by cruising toward the intersection, thereby sparing the brakes and transmission further wear.

Experiments with the system showed that with context, a robot can operate more safely and efficiently without slowing down production.

In one test performed with the system, a robot arm's path was blocked unexpectedly by someone's hand. But rather than stop, the robot adjusted - it predicted the future trajectory of the hand and the arm moved around the hand.

"This is safety not just from the technical point of view in avoiding collisions, but being able to recognize the context of the assembly line," he says. "This gives an additional layer of safety."

###

The research was an extension of the Symbiotic Human Robot Collaborative Assembly project, which was completed in 2019.