Engineers develop device that merges sensing and computing functions for reconfigurable computing platform

In recent years, engineers have been trying to create hardware systems that better support the high computational demands of machine learning algorithms. These include systems that can perform multiple functions, acting as sensors, memories and computer processors all at once.

Researchers at Peking University recently developed a new reconfigurable neuromorphic computing platform that integrates sensing and computing functions in a single device. This system, outlined in a paper published in Nature Electronics, is comprised of an array of multiple phototransistors with one memristor (MP1R).

“The inspiration for this research stemmed from the limitations of traditional vision computing systems based on the CMOS von Neumann architecture,” Yuchao Yang, senior author of the paper, told Tech Xplore.

“These systems face significant challenges in real-time image processing due to the physical separation between image sensors, memory, and processors, resulting in data redundancy, high power consumption, and processing delays. In contrast, biological vision systems, such as the human eye, demonstrate remarkable efficiency and adaptability, motivating the development of bioinspired approaches to vision computing.”

While photonic memristors have been found to be promising devices to run algorithms for computer vision, their ability to encode and process optical data is limited. As a result, they often are not as well-suited for running other neural network architectures beyond those designed to complete computer vision tasks.

“This challenge motivated us to explore novel in-sensor processing solutions capable of unifying machine learning and biologically inspired vision computing paradigms,” said Yang.

The main objective of the recent study by Yang and his colleagues was to develop a universal and reconfigurable in-sensor processing platform. In contrast with previous systems based on photonic memristors, this platform should support both computer vision algorithms and other neural network architectures.

“We fabricated the MP1R array by integrating a 20×20 phototransistor array with 20 channels of reconfigurable Mott memristors,” explained Yang. “The process began with the fabrication of amorphous indium gallium zinc oxide (α-IGZO) thin-film transistors using silicon oxide-compatible processing, which allowed us to create back-gate phototransistors.”

The 20×20 phototransistor array fabricated by Yang and his colleagues can sense light and module its response based on its different wavelengths. Specifically, the array exhibits potentiation behavior when it is exposed to blue light and depression behavior when exposed to red light.

“Next, we integrated the Mott memristors, constructed from Ta/TaO_x/NbO_x/W heterostructures, which provide several key features,” said Yang.

“These include a linear resistive region, volatile memory, and threshold switching capabilities. These characteristics enable the system to support multiple types of encoding—analog and spike-based—and simulate both synaptic and neuronal functions effectively.”

The platform created by the researchers combines optical sensing with data processing and memory functions in a single system. It is highly versatile and can be used to run algorithms designed to tackle a variety of tasks, ranging from static and event-based image recognition tasks to the analysis of colored images.

“Our recent work has led to several notable achievements in the field of neuromorphic vision systems,” said Yang.

“One of the key innovations of this work is the integration of Mott oxide memristors with phototransistors to create a highly versatile hardware system. This integration allows the system to support multiple optical image encoding functions, including spatiotemporal, analog, and spike encoding, which were previously difficult to achieve in a single device.”

Notably, the system created by Yang and his colleagues is compatible with a wide range of neural network architectures, including convolutional neural networks (CNNs), recurrent neural networks (RNNs) and spiking neural networks (SNNs). This remarkable versatility could facilitate its future deployment in real-world settings.

“A significant achievement of our system is its ability to support both biological-inspired and machine-learning algorithms, bridging the gap between these two paradigms in vision computing,” said Yang.

“This reconfigurable hardware system simplifies circuit design by consolidating multiple neural processing principles into a single device. As a result, the system offers reduced network complexity, lower latency, and improved energy efficiency, making it particularly effective for real-time image processing applications.”

A further advantage of the platform created by Yang and his colleagues is its reliability across a wide range of tasks. This is due to the low-variability Ta/TaO_x/NbO_x/W memristor devices it is based on.

“In terms of practical implications, this work lays a crucial foundation for building large-scale, energy-efficient, and low-latency neuromorphic vision systems,” said Yang. “These systems could provide a powerful platform for advanced vision AI applications, offering significant benefits in terms of flexibility, performance, and scalability.”

The recent efforts by this team of researchers could pave the way for the development of other universal neuromorphic vision platform. This could help to improve the performance of machine learning algorithms on various tasks, while also reducing their power-consumption.

“Although we have successfully developed a memristor with rich dynamic characteristics and demonstrated its advantages in implementing neuromorphic vision hardware functions and architectures, contributing critical research advancements toward a universal neuromorphic vision computing platform, there is still significant work to be done,” added Yang.

“In the future, we plan to focus on achieving three-dimensional integration to enhance system density and computational efficiency.”

In their next studies, Yang and his colleagues will also try to optimize the power consumption of their platform and improve its sensitivity to changes in lighting. This would further boost the system’s versatility, allowing it to also collect quality data in natural light and low-lighting conditions.

© 2024 Science X Network