Innovative bird-eye-inspired camera developed for enhanced object detection

A new type of perovskite-based camera specializing in object detection is developed by mimicking bird's eye

INSTITUTE FOR BASIC SCIENCE

 

IMAGE: 

STRUCTURES AND FUNCTIONS OF BIRD’S EYE. (A) BIRD VISION. (B) DEEP CENTRAL FOVEA AND FOUR TYPES OF CONES. (C) FOVEATED VISION AND TETRACHROMATIC VISION.

view more 

CREDIT: INSTITUTE FOR BASIC SCIENCE

The eyes of raptors such as eagles can accurately perceive prey from kilometers away. Is it possible to model the camera technology after the bird’s eyes?

Researchers developed a new type of camera, which was inspired by the structures and functions of bird's eyes. A research team led by Prof. KIM Dae-Hyeong at the Center for Nanoparticle Research within the Institute for Basic Science (IBS), in collaboration with Prof. SONG Young Min at the Gwangju Institute of Science and Technology (GIST), has developed perovskite-based camera specializing in object detection.

The eyes of different organisms in the natural world have evolved and been optimized to suit their habitat and the environment in which they survive. As a result of countless years of evolutionary adaptation to the environment of living and flying at high altitudes, bird eyes also have unique structures and visual functions (Figure 1a).

In the retina of an animal's eye, there is a small pit called the fovea that refracts the light entering the eye. Unlike the shallow foveae found in human eyes, bird eyes have deep central foveae, which refract the incoming light to a large extent. The region of the highest cone density lies within the foveae (Figure 1b), allowing the birds to clearly perceive distant objects through magnification (Figure 1c). This specialized vision is known as foveated vision.

While human eyes can only see visible light, bird eyes have four cones that respond to ultraviolet (UV) as well as visible (red, green, blue; RGB) light. This tetrachromatic vision enables birds to acquire abundant visual information and effectively detect target objects in a dynamic environment (Figure 1c).

Inspired by these capabilities, the IBS research team designed a new type of camera that specializes in object detection, incorporating artificial fovea and a multispectral image sensor that responds to both UV and RGB (Figure 2a). First, the researchers fabricated the artificial fovea by mimicking the deep central foveae in the bird's eyes (Figure 2b) and optimized the design through the optical simulation. This allows for the camera to magnify distant target objects without image distortion.

The team then used perovskite, a material known for its excellent electrical and optical properties, to fabricate the multispectral image sensor. Four types of photodetectors were fabricated using different perovskite materials that absorb different wavelengths. The multispectral image sensor was finally fabricated by vertically stacking the four photodetectors (Figures 2c and 2d).

The first co-author Dr. PARK Jinhong states, “We also developed a new transfer process to vertically stack the photodetectors. By using the perovskite patterning method developed in our previous research, we were able to fabricate the multispectral image sensor that can detect UV and RGB without additional color filters.”

Conventional cameras that use a zoom lens to magnify objects have the disadvantage of focusing only on the target object and not its surroundings. On the other hand, the bird-eye-inspired camera provides both a magnified view of the foveal region along with the surrounding view of the peripheral region (Figures 3a and 3b). By comparing the two fields of vision, the bird-eye-inspired camera can achieve greater motion detection capabilities than the conventional camera (Figure 3c and 3d). In addition, the camera is more cost-effective and lightweight as it can distinguish UV and RGB light without additional color filters.

The research team verified the object recognition and motion detection capabilities of the developed camera through simulations. In terms of object recognition, the new camera demonstrated a confidence score of 0.76, which is about twice as high as the existing camera system's confidence score of 0.39. The motion detection rate also increased by 3.6 times compared to the existing camera system, indicating significantly enhanced sensitivity to motion.

"Birds’ eyes have evolved to quickly and accurately detect distant objects while in flight. Our camera can be used in areas that need to detect objects clearly, such as robots and autonomous vehicles. In particular, the camera has great potential for application to drones operating in environments similar to those in which birds live,” remarked Prof. Kim.

This innovative camera technology represents a significant advancement in object detection, offering numerous potential applications across various industries.

Bird-eye-inspired camera. (a) Schematic view of bird-eye-inspired camera. (b) Artificial fovea. (c) Schematic of a multispectral image sensor. (d) Multispectral image sensor.

Performance of the bird-eye-inspired camera. (a) Setup for measurement. (b) Bird-eye-inspired camera perceives both the distant object (star) through magnification in the foveal region and nearby objects (triangle, square, circle) in the peripheral region. (c, d) The multispectral image sensor can distinguish UV and RGB light without color filters and capture colored images.

CREDIT

Institute for Basic Science

---

JOURNAL

Science Robotics

DOI

10.1126/scirobotics.adk6903 

METHOD OF RESEARCH

Experimental study

SUBJECT OF RESEARCH

Not applicable

ARTICLE TITLE

Avian-eye-inspired perovskite artificial vision system for foveated and multispectral imaging

ARTICLE PUBLICATION DATE

29-May-2024