Multi-task deep learning framework for enhancing Mayo endoscopic score classification in ulcerative colitis

Abstract

Objective: Ulcerative colitis (UC) endoscopic image classification presents challenges owing to imbalanced medical imaging data, particularly when the clinical importance of accurate positive predictions increases with disease severity. This study proposes a multi-task learning (MTL) framework inspired by the coarse-to-fine processing mechanism of the human brain to address these challenges. Methods: The proposed MTL framework was evaluated using endoscopic images of UC, focusing on its ability to classify disease stages with an emphasis on the accurate detection of advanced cases. Results: Our findings demonstrate that the proposed framework effectively mitigates the limitations posed by imbalanced datasets, particularly by enhancing classification performance in severe disease stages. Notably, DenseNet121 exhibited significantly superior performance compared to other backbones and achieved an additional performance gain in identifying Mayo endoscopic scores of 2 and 3 following joint-loss optimization. Additionally, MobileNet-v3-large, despite being a lightweight model, demonstrated notable gains under the proposed optimization scheme, highlighting the versatility of the framework across architectures with different computational complexities. Conclusion: MTL-based computer-aided diagnosis enables conservative and accurate identification of patients in critical stages, supporting timely and appropriate treatment decisions while reducing the risk of underdiagnosis and delayed care. Furthermore, our results highlight the potential of MTL in overcoming data imbalance issues. Future studies should explore integrating multiple convolutional neural network-based models to further boost classification accuracy.