Accuracy of Fully Automated and Human-assisted Artificia Intelligence–based CT Quantification of Pleural Effusion Changes after Thoracentesis

Abstract

Quantifying pleural effusion change at chest CT is important for evaluating disease severity and treatment response. The purpose of this study was to assess the accuracy of artificial intelligence (AI)–based volume quantification of pleural effusion change on CT images, using the volume of drained fluid as the reference standard. Seventy-nine participants (mean age ± SD, 65 years ± 13; 47 male) undergoing thoracentesis were prospectively enrolled from October 2021 to September 2023. Chest CT scans were obtained just before and after thoracentesis. The volume of pleural fluid on each CT scan, with the difference representing the drained fluid volume, was measured by automated segmentation (fully automated measurement). An expert thoracic radiologist then manually corrected these automated volume measurements (human-assisted measurement). Both fully automated (median percentage error, 13.1%; maximum estimated 95% error, 708 mL) and human-assisted measurements (median percentage error, 10.9%; maximum estimated 95% error, 312 mL) systematically underestimated the volume of drained fluid, beyond the equivalence margin. The magnitude of underestimation increased proportionally to the drainage volume. Agreements between fully automated and human-assisted measurements (intraclass correlation coefficient [ICC], 0.99) and the test-retest reliability of fully automated (ICC, 0.995) and human-assisted (ICC, 0.997) measurements were excellent. These results highlight a potential systematic discrepancy between AI segmentation–based CT quantification of pleural effusion volume change and actual volume change.