Predict Unmatching Compositions for Compositional Zero-Shot Learning

Abstract

Compositional Zero-Shot Learning (CZSL) poses the challenge of predicting unseen attribute-object combinations in images. In this study, we focus on the open-world CZSL task, which presents a more realistic and comprehensive challenge by expanding the search space to include unmatching pairs. Through t-SNE visualization and convergence analysis, we observe that existing methods struggle to capture the interdependencies between labels, leading to the Plausible Unmatching Pair (PUP) problem, where models are prone to confusing matching and unmatching pairs. Inspired by label dependency modeling in multi-label classification, we propose a novel approach called Absence Modeling to address the PUP problem. Absence Modeling aims to predict unmatching compositions, allowing the model to learn irrelevant information between attributes and objects, thereby improving its ability to capture interdependencies. By applying Absence Modeling, we observe significant improvements in zero-shot performance and achieve state-of-the-art results. Our experimental results validate that our approach effectively addresses the PUP problem.