Optical convolution operations with optical neural networks for incoherent color image recognition

Abstract

This study introduces optical neural networks (ONNs) designed to accelerate optical convolution operations using a red, green, and blue (RGB) pixel array integrated into conventional display technology. In a proof-of-concept demonstration, we initially employed a rank-4 kernel for a normal convolution network, which was integrated with a fully connected layer, to accurately classify color images across five fruit categories. Following seven epochs of on-system iterative training for a 3,000 training dataset, the ONN achieved 96% classification accuracy and maintained robust performance on an unseen 1,000 test dataset. Our analysis also showed its potential for efficient operation, with a classification accuracy exceeding 94% using an average less than 34 aJ of optical energy per MAC operation. Additionally, we demonstrated depth-wise convolution with a rank-3 kernel, recurring the system to spectrally resolve the signals into independent R, G, and B channels. This architecture enabled the successful classification of complex patterns containing three MNIST handwritten digits encoded in RGB. Our strategy contributes significantly to optical computing and neuromorphic vision, facilitating efficient recognition of real-world, multi-color, and incoherent light images.