Deep-reinforcement-learning-based robot motion strategies for grabbing objects from human hands

Peer-Reviewed Publication

BEIJING ZHONGKE JOURNAL PUBLISING CO. LTD.

 

IMAGE: 

DEEPREINFORCEMENT LEARNING HAS DISADVANTAGES SUCH AS LOW SAMPLE UTILIZATION AND SLOW CONVERGENCE, AND THOUSANDSOF TRIAL-AND-ERROR ITERATIONS ARE REQUIRED TO PERFORM REINFORCEMENT LEARNING IN REALISTIC SCENARIOS, WHICH ISCOSTLY. TO ALLEVIATE THIS PROBLEM, RGRL FIRST SIMULATES A ROBOT GRASPING AN OBJECT FROM A USER IN A SIMULATEDSCENE IN WHICH TENS OF THOUSANDS OF LEARNING SESSIONS ARE PERFORMED. DOMAIN RANDOMIZATION IS USED TO NARROWTHE GAP BETWEEN THE SIMULATED AND REAL SCENES, AND A MULTI-OBJECTIVE REWARD FUNCTION IS USED TO EFFECTIVELYACCELERATE THE CONVERGENCE OF THE REINFORCEMENTLEARNING ALGORITHM.

view more 

CREDIT: BEIJING ZHONGKE JOURNAL PUBLISING CO. LTD.

Owing to the rapid development of artificial intelligence, sensors, and robot control technology in recent years,“machine-centered” human–machine collaboration has become increasingly incompetent in the case ofcomplex scenarios, and “human-centered” human–machine collaboration is gradually revealing its advantages. Common industrial and service robots attempt to avoid contact with users for safety. With theincreasing demand for robot intelligence, contact human–robot collaboration is unavoidable in the future.Common industrial and service robots can easily grasp stationary objects; however, robots that can graspobjects from the hands of users are rare. In most scenarios in which this function is implemented, physical,spatial, or motion constraints are employed to prevent the manipulator from harming the human hand duringthe grasping process. Unlike robots independently grasping stationary objects placed in a scene, graspingobjects from a user's hands requires more considerations, and the recognition, positioning, and real-timebehavioral actions of the user must be carefully taken into account. Robots must determine the correct positionand direction from which to grasp an object from a user's hand. This paper presents a robot grasping algorithmbased on deep reinforcement learning (RGRL) to solve the problem of robots safely grasping an object from auser's hand. Deep reinforcement learning is used in the RGRL so that the robot can actively learn how to pick

up an object from the user's hand without touching the user.

The contributions of this study are three-fold:

(1) A new algorithm, RGRL, for grasping objects from users is proposed, by incorporating domain

Randomizationand a multi-objective reward function.

(2) The RGRL has low computational and hardware costs. Because we use domain randomization in thetraining phase, we eliminate the need for manual labeling of the data. Only a 3D model of the object needs tobe imported into the simulation software, and the algorithm can automatically learn the appropriate motionpath. The only required addition to the robot is the Leap Motion sensor, which is used to satisfy the conditionsfor the algorithm to run.

(3) The RGRL is evaluated in simulated and real scenarios.

---

JOURNAL

Virtual Reality & Intelligent Hardware

DOI

10.1016/j.vrih.2022.12.001 

ARTICLE TITLE

Deep-reinforcement-learning-based robot motion strategies for grabbing objects from human hands