Metalens-style image synthesis for metalens imaging via image-to-image translation

Abstract

Metalenses offer wafer-scale, ultra-thin optics for compact cameras, but strong chromatic and field-dependent aberrations still limit their practical use. Deep learning–based aberration correction can restore high-quality images from metalens captures, but current pipelines typically require hundreds to thousands of paired images per device. We address this data bottleneck by formulating metalens aberration synthesis as a deterministic, metalens-conditioned image-to-image translation problem. A generator is trained on a dataset of paired metalens and conventional images from a mass-producible metalens, then used to transform photographs into metalens-style outputs that reproduce realistic chromatic aberration, field-dependent blur, and spatial distortion. On a test set, the proposed translator reduces LPIPS(VGG) from 0.305 to 0.117 (62%) compared with a state-of-the-art transformer-based restoration baseline. Once trained, the translator can generate 600 synthetic metalens-style images in roughly 30 s on a single GPU, versus about 30 min for real metalens acquisition, a reduction in data-collection time. These synthetic pairs alone suffice to train a metalens image restoration model, suggesting that our approach can help alleviate the data bottleneck in future metalens imaging research.