CFP-AL: Combining Model Features and Prediction for Active Learning in Sentence Classification

Abstract

Active learning has been a research area conducted across various domains for a long time, from traditional machine learning to the latest deep learning research. Particularly, obtaining high-quality labeled datasets for supervised learning requires human annotation, and an effective active learning strategy can greatly reduce annotation costs. In this study, we propose a new insight, CFP-AL (Combining model Features and Prediction for Active Learning), from the perspective of feature space by analyzing and diagnosing methods that have shown good performance in NLP (Natural Language Processing) sentence classification. According to our analysis, while previous active learning strategies that focus on finding data near the decision boundary to facilitate classifier tuning are effective, there are very few data points near the decision boundary. Therefore, a more detailed active learning strategy is needed beyond simply finding data near the decision boundary or data with high uncertainty. Based on this analysis, we propose CFP-AL, which considers the model's feature space, and it demonstrated the best performance across six tasks and also outperformed others in three Out-Of-Domain (OOD) tasks. While suggesting that data sampling through CFP-AL is the most differential classification standard, it showed novelty in suggesting a method to overcome the anisotropy phenomenon of supervised models. Additionally, through various comparative experiments with basic methods, we analyzed which data are most beneficial or harmful for model training. Through our research, researchers will be able to expand into the area of considering features in active learning, which has been difficult so far.