Brain-inspired acoustic-optical system recognizes voice commands and drone trajectories
A self-powered triboelectric acoustic sensor and an oxide neuromorphic transistor combine sensing, memory and reservoir computing in a single multimodal platform
image:
The acoustic-optical information reservoir system integrates a triboelectric acoustic sensor (TAS), a rectifier bridge and an indium-zinc-oxide photoelectronic neuromorphic transistor (IZO-PNT). Sound and light stimuli update synaptic weights, while the device maps voice commands and drone trajectory images into high-dimensional conductance states for classification.
view moreCredit: Si Yuan Zhou, Li Qiang Zhu* et al., Nano Research, Tsinghua University Press
As Internet of Things technologies continue to develop, human–machine interfaces are expected to perceive and process increasingly diverse environmental information. The human brain performs this task efficiently by integrating auditory and visual signals, whereas conventional artificial systems often use separate components for sensing, memory, and computation.
A research team at Ningbo University has developed an acoustic-optical information reservoir system (AOIRS) that integrates a triboelectric acoustic sensor (TAS) with an indium-zinc-oxide photoelectronic neuromorphic transistor (IZO-PNT). The system can perceive sound and light, reproduce synaptic-like memory behaviors, and process temporal information through reservoir computing.
The study, titled “Acoustic-optical information reservoir system based on triboelectric acoustic sensor tuned oxide photoelectronic neuromorphic transistor,” was published in Nano Research on June 29, 2026.
The TAS converts sound-induced mechanical vibrations into electrical signals through triboelectrification and electrostatic induction. It contains fluorinated ethylene-propylene and polyimide friction layers with copper-mesh electrodes. A three-dimensional acoustic coupling cavity was introduced to improve sound-wave collection compared with a planar device.
Under a sound wave signal of 110 dB and 200 Hz, the TAS generated an open-circuit voltage of ~20.4 V and a short-circuit current of ~3.8 μA. Its sensitivity reached ~1.1 V/dB, while the output power density reached ~30.3 mW/m² under a load resistance of 5 MΩ. The sensor maintained a stable response during a durability test of ~2000 s, and its open-circuit voltage decreased only slightly from ~20.4 V to ~19.2 V after ~170 days.
The TAS also captured voice commands with characteristic temporal and frequency information. The recorded voltage waveforms were similar to the original speech signals, while the corresponding frequency distributions remained highly similar below 2000 Hz.
After full-wave rectification, the acoustic signals were applied to the gate of the IZO-PNT. The transistor consists of an IZO channel and a chitosan-based solid-state electrolyte with a specific capacitance of ~5.1 μF/cm². Proton migration in the electrolyte and light-induced carrier processes in the IZO channel produced nonlinear conductance responses and short-term memory.
These properties allowed the device to reproduce synaptic functions, including excitatory postsynaptic current, paired-pulse facilitation, and spike-amplitude-, duration-, and number-dependent responses. The AOIRS also simulated learning–forgetting–relearning behavior. During three successive learning processes, the number of sound signals required to reach the same learning threshold decreased from 30 to 10 and then to 7, indicating that previous stimulation facilitated subsequent learning.
The device could be modulated by both sound and light. Under 20 consecutive sound wave signals of 90 dB, 200 Hz, and 0.5 s, the excitatory postsynaptic current peak increased from ~27.5 μA to ~51.9 μA as the optical power intensity increased from 0 to 57.2 mW/cm². Optical stimuli produced synaptic potentiation, while negatively rectified sound signals produced synaptic depression. The synaptic weights remained stable over repeated sound–light cycles.
The researchers then used the AOIRS as a physical reservoir. Its nonlinear dynamics and fading memory mapped temporal inputs into high-dimensional conductance states, which were classified by a multilayer perceptron.
For voice-command recognition, the team collected 270 samples covering nine commands spoken by 30 individuals. After 200 training epochs, the training and testing accuracies reached ~94.8% and ~89.8%, respectively.
The system was also tested using 1800 samples representing nine drone trajectories, including eight movement directions and a stationary state. Four successive 40 × 40 pixel images were converted into optical pulse trains and mapped into conductance states. After 200 epochs, the training and testing accuracies reached ~95.4% and ~93.6%, respectively.
The results demonstrate that sound and light information can be sensed, memorized, and processed through a shared neuromorphic hardware platform. The AOIRS shows potential for intelligent voice sensing, multimodal human–machine interaction, edge computing, and low-altitude applications such as drone trajectory recognition, formation control, and collision avoidance.
Other contributors include Wei Sheng Wang, Lin Feng Wu, Bo Bo Li, Wan Lin Zhang and Yu Fan Hu from the School of Physical Science and Technology at Ningbo University, and Wen Xiang Tao and Cong Shan Liu from the Center for Mechanics Plus Under Extreme Environments at Ningbo University.
This work was supported by the National Natural Science Foundation of China (U22A2075) and the Ningbo Key Scientific and Technological Project (2021Z116).
DOI Link:
https://doi.org/10.26599/NR.2026.94908978
About Nano Research
Nano Research is a peer-reviewed, open access, international and interdisciplinary research journal, sponsored by Tsinghua University and the Chinese Chemical Society, published by Tsinghua University Press on the platform SciOpen. It publishes original high-quality research and significant review articles on all aspects of nanoscience and nanotechnology, ranging from basic aspects of the science of nanoscale materials to practical applications of such materials. After 18 years of development, it has become one of the most influential academic journals in the nano field. Nano Research has published more than 1,000 papers every year from 2022, with its cumulative count surpassing 8,000 articles. In 2025 InCites Journal Citation Reports, its 2025 IF is 9.4 (8.3, 5 years), and it continues to be the Q1 area among the four subject classifications. Nano Research Award, established by Nano Research together with TUP and Springer Nature in 2013, and Nano Research Young Innovators (NR45) Awards, established by Nano Research in 2018, have become international academic awards with global influence.
Journal
Nano Research
Article Title
Brain-inspired acoustic-optical system recognizes voice commands and drone trajectories
No comments:
Post a Comment