Multispectral and hyperspectral imaging are important optical inspection technologies. They collect the spatial and spectral information of the incidental light into 3D hypercubes, which can be post-processed into material and structural mapping of the scene. However, acquiring and analyzing the 3D hypercubes set great challenges in data collection, transportation, storage, and computation. The much higher energy, bandwidth, and memory budgets limit the implementation of high-speed, high-resolution hyperspectral imaging to achieve intelligent machine vision. This project introduces an intelligent sensor that directly computes the final spectral analysis result inside the sensors, thereby potentially lowering the energy consumption and analysis frame rate by more than two orders of magnitude for hyperspectral imaging. By integrating an electrically tunable black phosphorous (bP)/MoS2 photodiode with a spectrally encoded optoelectronic feedback loop, each pixel detects and analyzes visible-to-MIR spectral information with AI algorithms inside a single sensor. An in-situ training algorithm further enables a ‘sniff-and-seek’ operation mode: the nanodevice array learns from example objects shown to it without ground truth spectra to identify different materials. Such spectral machine vision sensor is ideal for field tasks that inspect complex, uncalibrated environments with high speed, good portability, and low power budget.
Project is currently funded by: Federal