VR could help train employees working with robots
Virtual reality gives workers an immersive digital environment to practice complex tasks
University of Georgia
Working with robots is becoming more common in the recycling industry, helping automate tasks and making complicated work easier. But training human employees to work with robots can be difficult and time consuming.
Researchers from the University of Georgia created a new virtual reality space to help make training faster and easier.
One task that often requires robotic help is disassembly. It’s critical for recycling parts and valuable materials from electronics nearing the end of their lifespan. However, disassembly tasks often present their own challenges.
“Unlike assembly, which has a very standard procedure, disassembly is slightly more complicated,” said Beiwen Li, corresponding author of the study and an associate professor in UGA’s College of Engineering. “It may not work out the best if we just inverse the whole assembly procedure.”
The researchers developed VR Co-Lab to help ease confusion. By training employees digitally, not only can they practice disassembling recyclables without damaging materials, but they can also learn to avoid injury and collisions with the robot.
Virtual reality gives workers an immersive digital environment to practice complex tasks
University of Georgia
Working with robots is becoming more common in the recycling industry, helping automate tasks and making complicated work easier. But training human employees to work with robots can be difficult and time consuming.
Researchers from the University of Georgia created a new virtual reality space to help make training faster and easier.
One task that often requires robotic help is disassembly. It’s critical for recycling parts and valuable materials from electronics nearing the end of their lifespan. However, disassembly tasks often present their own challenges.
“Unlike assembly, which has a very standard procedure, disassembly is slightly more complicated,” said Beiwen Li, corresponding author of the study and an associate professor in UGA’s College of Engineering. “It may not work out the best if we just inverse the whole assembly procedure.”
The researchers developed VR Co-Lab to help ease confusion. By training employees digitally, not only can they practice disassembling recyclables without damaging materials, but they can also learn to avoid injury and collisions with the robot.
Using virtual reality to cut training time
In the virtual space, users practiced taking apart a hard disk with a robot’s help.
When users put on their headset, they found themselves in a virtual workstation like what they’d encounter on the job. They saw the various tools and machinery needed for disassembly, as well as the robot helping them.
During training, workers followed a step-by-step procedure for disassembling the hard disk. Human users handled more precise tasks like unscrewing or picking and placing small bolts, while the robot arm managed larger bolts and loose items. The program also provided feedback to the user, measuring how long it took to complete the session and how many mistakes users made.
“There are a lot of tasks. It requires a complicated training for workers, typically,” Li said. “So, if we have a VR system, that will be very helpful in shortening training time. It is much easier than having pages and pages of written documents to be read by the user.”
In the virtual space, users practiced taking apart a hard disk with a robot’s help.
When users put on their headset, they found themselves in a virtual workstation like what they’d encounter on the job. They saw the various tools and machinery needed for disassembly, as well as the robot helping them.
During training, workers followed a step-by-step procedure for disassembling the hard disk. Human users handled more precise tasks like unscrewing or picking and placing small bolts, while the robot arm managed larger bolts and loose items. The program also provided feedback to the user, measuring how long it took to complete the session and how many mistakes users made.
“There are a lot of tasks. It requires a complicated training for workers, typically,” Li said. “So, if we have a VR system, that will be very helpful in shortening training time. It is much easier than having pages and pages of written documents to be read by the user.”
Boosting performance while keeping workers safe
One key part of the VR system is body tracking.
VR Co-Lab uses Meta Quest Pro, which uses its cameras to track upper body movements in the wrists, elbows, shoulders and torso. This helps the program plan the robot’s movements based on the user’s actions, preventing collisions and improving interactivity between the human and robot.
The program warns of potential hazards that could lead to injury, such as collisions with the robot arm, while teaching users how to avoid them. The system can also be used to figure out how quickly the robot arm can work without overwhelming the employee.
"Robots are going to be important for the future of the recycling industry.” — Beiwen Li, College of Engineering
Li and his team are planning more extensive user testing in the future to ensure the system is helpful for a variety of skill levels and applicable to tasks outside of disassembling hard disks. Improving the training process will be valuable as using robots becomes more common.
“Robots are going to be important for the future of the recycling industry because they can do a lot of disassembly steps automatically. That can help reduce the labor shortage,” Li said. “Because disassembly is so complicated, it involves a human to work together with a robot. And that’s basically our motivation for developing this VR system for training.”
The study was published in Machines and funded by a grant from the National Science Foundation. It was co-authored by Yashwanth Maddipatla of Iowa State University, and Sibo Tian, Xiao Liang and Minghui Zheng of Texas A&M University.
One key part of the VR system is body tracking.
VR Co-Lab uses Meta Quest Pro, which uses its cameras to track upper body movements in the wrists, elbows, shoulders and torso. This helps the program plan the robot’s movements based on the user’s actions, preventing collisions and improving interactivity between the human and robot.
The program warns of potential hazards that could lead to injury, such as collisions with the robot arm, while teaching users how to avoid them. The system can also be used to figure out how quickly the robot arm can work without overwhelming the employee.
"Robots are going to be important for the future of the recycling industry.” — Beiwen Li, College of Engineering
Li and his team are planning more extensive user testing in the future to ensure the system is helpful for a variety of skill levels and applicable to tasks outside of disassembling hard disks. Improving the training process will be valuable as using robots becomes more common.
“Robots are going to be important for the future of the recycling industry because they can do a lot of disassembly steps automatically. That can help reduce the labor shortage,” Li said. “Because disassembly is so complicated, it involves a human to work together with a robot. And that’s basically our motivation for developing this VR system for training.”
The study was published in Machines and funded by a grant from the National Science Foundation. It was co-authored by Yashwanth Maddipatla of Iowa State University, and Sibo Tian, Xiao Liang and Minghui Zheng of Texas A&M University.
Journal
Machines
Machines
DOI
Article Title
VR Co-Lab: A Virtual Reality Platform for Human–Robot Disassembly Training and Synthetic Data Generation
VR Co-Lab: A Virtual Reality Platform for Human–Robot Disassembly Training and Synthetic Data Generation
Mid-air transformation helps flying, rolling robot to transition smoothly
New research from Caltech’s Center for Autonomous Systems and Technologies finds robots that morph before landing are more robust
California Institute of Technology
image:
The ATMO (aerially transforming morphobot) robot transforms from its flying, quadrotor configuration midair as it approaches the ground. It is then able to roll away in its driving configuration. Caltech researchers developed a sophisticated control system for ATMO that uses an advanced control method called model predictive control, which works by continuously predicting how the system will behave in the near future and adjusting its actions to stay on course.
view moreCredit: Ioannis Mandralis/Communications Engineering
Specialized robots that can both fly and drive typically touch down on land before attempting to transform and drive away. But when the landing terrain is rough, these robots sometimes get stuck and are unable to continue operating. Now a team of Caltech engineers has developed a real-life Transformer that has the "brains" to morph in midair, allowing the dronelike robot to smoothly roll away and begin its ground operations without pause. The increased agility and robustness of such robots could be particularly useful for commercial delivery systems and robotic explorers.
The new robot, dubbed ATMO (aerially transforming morphobot), uses four thrusters to fly, but the shrouds that protect them become the system's wheels in an alternative driving configuration. The whole transformation relies on a single motor to move a central joint that lifts ATMO's thrusters up into drone mode or down into drive mode.
The researchers describe the robot and the sophisticated control system that drives it in a paper recently published in the journal Communications Engineering.
"We designed and built a new robotic system that is inspired by nature—by the way that animals can use their bodies in different ways to achieve different types of locomotion," says Ioannis Mandralis (MS '22), a graduate student in aerospace at Caltech and lead author of the new paper. For example, he says, birds fly and then change their body morphology to slow themselves down and avoid obstacles. "Having the ability to transform in the air unlocks a lot of possibilities for improved autonomy and robustness," Mandralis says.
But midair transformation also poses challenges. Complex aerodynamic forces come into play both because the robot is close to the ground and because it is changing its shape as it morphs.
"Even though it seems simple when you watch a bird land and then run, in reality this is a problem that the aerospace industry has been struggling to deal with for probably more than 50 years," says Mory Gharib (PhD '83), the Hans W. Liepmann Professor of Aeronautics and Medical Engineering, director and Booth-Kresa Leadership Chair of Caltech's Center for Autonomous Systems and Technologies (CAST), and director of the Graduate Aerospace Laboratories of the California Institute of Technology (GALCIT). All flying vehicles experience complicated forces close to the ground. Think of a helicopter, as an example. As it comes in for a landing, its thrusters push lots of air downward. When that air hits the ground, some portion of it bounces back up; if the helicopter comes in too quickly, it can get sucked into a vortex formed by that reflected air, causing the vehicle to lose its lift.
In ATMO's case, the level of difficulty is even greater. Not only does the robot have to contend with complex near-ground forces, but it also has four jets that are constantly altering the extent to which they are shooting toward each other, creating additional turbulence and instability.
To better understand these complex aerodynamic forces, the researchers ran tests in CAST's drone lab. They used what are called load cell experiments to see how changing the robot's configuration as it came in for landing affected its thrust force. They also conducted smoke visualization experiments to reveal the underlying phenomena that lead to such changes in the dynamics.
The researchers then fed those insights into the algorithm behind a new control system they created for ATMO. The system uses an advanced control method called model predictive control, which works by continuously predicting how the system will behave in the near future and adjusting its actions to stay on course.
"The control algorithm is the biggest innovation in this paper," Mandralis says. "Quadrotors use particular controllers because of how their thrusters are placed and how they fly. Here we introduce a dynamic system that hasn't been studied before. As soon as the robot starts morphing, you get different dynamic couplings—different forces interacting with one another. And the control system has to be able to respond quickly to all of that."
Additional Caltech authors of the paper, "ATMO: An aerially transforming morphobot for dynamic ground-aerial transition" are Reza Nemovi, a design engineer in aerospace; and Richard M. Murray (BS '85), the Thomas E. and Doris Everhart Professor of Control and Dynamical Systems and Bioengineering. Co-author Alireza Ramezani, an associate professor of electrical and computer engineering at Northeastern University, is currently a visitor in aerospace at Caltech. The work was supported by funding from the Center for Autonomous Systems and Technologies at Caltech.
Caltech researchers used smoke visualization to better understand the complex forces at play during ATMO's midair transformation close to the ground.
Credit
Ioannis Mandralis/Communications Engineering
Journal
Communications Engineering
Article Title
ATMO: An aerially transforming morphobot for dynamic ground-aerial transition
