Saturday, August 10, 2024

HKUST develops groundbreaking artificial compound eye to revolutionize robotic vision at lower cost but higher sensitivity

Enhancing autonomous driving and emergency rescue

Hong Kong University of Science and Technology

Cover article in Science Robotics — image:
The research paper was featured as the cover article in Science Robotics (Volume 9, Issue 90) in May 2024. The cover shows a fusion of an image composite of a robber fly’s eye on the left and an illustration of the pinhole compound eye on the right.
view more
Credit: HKUST

A research team at the School of Engineering of the Hong Kong University of Science and Technology (HKUST) has recently developed a novel artificial compound eye system that is not only more cost-effective, but demonstrates a sensitivity at least twice that of existing market products in small areas. The system promises to revolutionize robotic vision, enhance robots' abilities in navigation, perception and decision-making, while promoting commercial application and further development in human-robot collaboration.

Mimicking the visual capabilities of compound eyes, this innovative system can be applied in a wide range of scenarios, such as installing on drones to improve their accuracy and efficiency in tasks like irrigation or emergency rescue in disaster sites. With its high sensitivity, the system can also enable closer collaboration among robots and other connected devices. In the long term, the compound eye system will enhance autonomous driving safety and accelerate the adoption of intelligent transport systems, fostering the development of smart cities.

Developed by the team led by Prof. FAN Zhiyong, Chair Professor at HKUST’s Department of Electronic & Computer Engineering and Department of Chemical & Biological Engineering, this groundbreaking technology represents a significant leap forward in the field of biomimetic vision systems.

Traditionally, roboticists have mainly focused on replicating the visual capabilities of insects, which offer a wide field of view and advanced motion-tracking capabilities. However, integrating compound eye systems into autonomous platforms like robots or drones has been challenging as these systems often suffer from issues related to complexity and stability during deformation, geometry constraints, as well as potential mismatches between optical and detector components.

To address these challenges, Prof. Fan’s team developed a pinhole compound vision system by adopting new materials and structures. This system features several key characteristics, including an inherent hemispherical perovskite nanowire array imager with high pixel density to enlarge the imaging field; and a 3D-printed lens-free pinhole array with a customizable layout to regulate incident light and eliminate the blind area between neighboring ommatidia (individual units within an insect’s compound eye). Owing to its good angular selectivity, a wide field of view, wide spectrum response in monocular and binocular configurations, as well as its dynamic motion tracking capability, the pinhole compound eye not only can accurately locate targets but can also track a moving quadruped robot after incorporated onto a drone.

Prof. Fan said, “This compound eye design is simple, light and cheap. Although it won’t fully replace traditional cameras, it could be a huge boost in certain robotics applications, such as in a swarm of drones flying in close formation. By further miniaturizing the device size and increasing the number of ommatidia, imaging resolution, and response speed, this type of device can find broad applications in optoelectronics and robotics.”

As a renowned researcher in biomimetic optoelectronics, Prof. Fan is keen on combining practical approach with daring imaginations to drive innovative research. This unique compound eye work marks another breakthrough in the field of vision and robotic systems following his development of the world’s first spherical artificial eye with 3D retina in 2020.

The research work was published and featured as a cover article in top-tier international journal Science Robotics. Dr. ZHOU Yu (postdoc), Dr. SUN Zhibo (postdoc), and DING Yucheng (PhD student) are the co-first authors while Prof. Fan is the corresponding author.