Self-supervised multi-modal training from uncurated images and reports enables monitoring AI in radiology
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Park, Sangjoon | - |
dc.contributor.author | Lee, Eun Sun | - |
dc.contributor.author | Shin, Kyung Sook | - |
dc.contributor.author | Lee, Jeong Eun | - |
dc.contributor.author | Ye, Jong Chul | - |
dc.date.accessioned | 2024-01-09T14:36:06Z | - |
dc.date.available | 2024-01-09T14:36:06Z | - |
dc.date.issued | 2024-01 | - |
dc.identifier.issn | 1361-8415 | - |
dc.identifier.issn | 1361-8423 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/70503 | - |
dc.description.abstract | The escalating demand for artificial intelligence (AI) systems that can monitor and supervise human errors and abnormalities in healthcare presents unique challenges. Recent advances in vision-language models reveal the challenges of monitoring AI by understanding both visual and textual concepts and their semantic correspondences. However, there has been limited success in the application of vision-language models in the medical domain. Current vision-language models and learning strategies for photographic images and captions call for a web-scale data corpus of image and text pairs which is not often feasible in the medical domain. To address this, we present a model named medical cross-attention vision-language model (Medical X-VL), which leverages key components to be tailored for the medical domain. The model is based on the following components: self-supervised unimodal models in medical domain and a fusion encoder to bridge them, momentum distillation, sentencewise contrastive learning for medical reports, and sentence similarity-adjusted hard negative mining. We experimentally demonstrated that our model enables various zero-shot tasks for monitoring AI, ranging from the zero-shot classification to zero-shot error correction. Our model outperformed current state-of-the-art models in two medical image datasets, suggesting a novel clinical application of our monitoring AI model to alleviate human errors. Our method demonstrates a more specialized capacity for fine-grained understanding, which presents a distinct advantage particularly applicable to the medical domain. © 2023 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | Elsevier B.V. | - |
dc.title | Self-supervised multi-modal training from uncurated images and reports enables monitoring AI in radiology | - |
dc.type | Article | - |
dc.identifier.doi | 10.1016/j.media.2023.103021 | - |
dc.identifier.bibliographicCitation | Medical Image Analysis, v.91 | - |
dc.description.isOpenAccess | N | - |
dc.identifier.wosid | 001112184700001 | - |
dc.identifier.scopusid | 2-s2.0-85176503153 | - |
dc.citation.title | Medical Image Analysis | - |
dc.citation.volume | 91 | - |
dc.type.docType | Article | - |
dc.publisher.location | 네델란드 | - |
dc.subject.keywordAuthor | Error detection | - |
dc.subject.keywordAuthor | Monitoring AI | - |
dc.subject.keywordAuthor | Radiograph | - |
dc.subject.keywordAuthor | Vision-language model | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Radiology, Nuclear Medicine & Medical Imaging | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Interdisciplinary Applications | - |
dc.relation.journalWebOfScienceCategory | Engineering, Biomedical | - |
dc.relation.journalWebOfScienceCategory | Radiology, Nuclear Medicine & Medical Imaging | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.