Quantifying the repeatability and reproducibility of Dr. Noon CVD, AI software as medical device for cardiovascular risk assessment via retinal imaging

Abstract

Objective: To evaluate the repeatability and reproducibility of the Dr.Noon CVD, an artificial intelligence software as a medical device that assesses cardiovascular risk from retinal photographs by providing risk scores and classifying patients into three categories: category 0 (low risk), category 1 (intermediate risk), and category 2 (high risk). Methods: In this prospective, single-center study, participants underwent nonmydriatic fundus photography. For repeatability assessment, one examiner captured 3 consecutive images per eye. For reproducibility, a second examiner independently acquired 1 image per eye. Intraclass correlation coefficients (ICCs), within-subject standard deviations, and coefficients of variation were calculated. Differences by risk category and lens status were assessed using ANOVA and independent t tests. Results: Overall, Dr.Noon CVD demonstrated excellent reliability, with a repeatability ICC of 0.997 (95% confidence interval [CI]: 0.996–0.998) and a reproducibility ICC of 0.999 (95% CI: 0.998–0.999). When analyzed by risk categories, repeatability, and reproducibility ICCs were 0.985 (95% CI: 0.974–0.997) and 0.995 (95% CI: 0.990–0.997) for category 0, 0.960 (95% CI: 0.918–0.983) and 0.969 (95% CI: 0.921–0.988) for category 1, and 0.965 (95% CI: 0.943–0.980) and 0.984 (95% CI: 0.971–0.992) for category 2. In terms of lens status, phakic eyes showed repeatability and reproducibility ICCs of 0.998 (95% CI: 0.996–0.998) and 0.999 (95% CI: 0.998–0.999), respectively, while pseudophakic eyes showed slightly lower but still excellent values of 0.989 (95% CI: 0.980–0.995) and 0.994 (95% CI: 0.988–0.997). Conclusions: Dr.Noon CVD demonstrated high precision with excellent repeatability and reproducibility across all risk levels and lens statuses, supporting its reliability for cardiovascular risk screening using retinal images.