Cyclic 2.5D Perceptual Loss for Cross-Modal 3D Medical Image Synthesis: T1w MRI to Tau PET

Abstract

Positron emission tomography (PET) provides an in vivo molecular marker for various diseases, including Alzheimer's disease and related dementias (ADRD). PET has become increasingly integrated into diagnostic decision-making, disease staging, and clinical trial enrichment. However, its widespread use remains constrained by high costs, government regulations, and the invasiveness of radiotracer injection. Modern diagnostic frameworks emphasize the importance of multimodal biomarker assessment, such as the "amyloid/tau/neurodegeneration" (A/T/N) framework for Alzheimer's disease; however, they are constrained by these barriers. Medical image synthesis or translation offers a potential solution by enabling the reconstruction of unavailable modalities. The clinical utility of PET depends on accurately capturing regional uptake patterns rather than exact voxel-wise intensities, motivating the use of perceptual loss functions to assess higher-level semantic features in generative models. While 2D, 3D, and 2.5D perceptual losses are utilized in 3D synthesis, each encounters challenges, including limited volumetric context, the scarcity of pretrained 3D models, and difficulty balancing optimization across anatomical planes. In this work, we address cross-modal synthesis of tau PET from structural magnetic resonance imaging (MRI), generating 3D pseudo-[F-18]flortaucipir standardized uptake value ratio (SUVR) maps from 3D T1-weighted MR images. We propose a cyclic 2.5D perceptual loss that cyclically optimizes the axial, coronal, and sagittal planes over training phases, thereby enhancing volumetric consistency. Furthermore, we standardize PET SUVRs by scanner manufacturer, reducing inter-manufacturer variability and better preserving high-uptake regions. We evaluate the proposed approach on cohorts spanning the ADRD spectrum using data from the Alzheimer's Disease Neuroimaging Initiative and the Standardized Centralized Alzheimer's Disease and Related Dementias Neuroimaging cohort. Our approach is broadly applicable across various generative frameworks and achieves high quantitative and qualitative performance on diverse architectures, including U-Net, UNETR, SwinUNETR, CycleGAN, and Pix2Pix. Notably, it achieves better agreement between synthesized SUVRs and measured PET scans in key brain regions relevant to Alzheimer-type tau pathology. The code is publicly available at https://github.com/labhai/Cyclic-2.5D-Perceptual-Loss.