Discrepancy-guided Channel Dropout for Domain Generalization

Abstract

Deep Neural Networks (DNNs) tend to perform poorly on unseen domains due to domain shifts. Domain Generalization (DG) aims to improve the performance on such scenarios by minimizing the distribution discrepancy between source domains. Among many studies, dropout-based DG approaches which remove domain-specific features have gained attention. However, they are limited in minimizing the upper bound of generalization risk because they do not explicitly consider the distribution discrepancy when discarding features. In this paper, we propose a novel Discrepancy-guided Channel Dropout (DgCD) for DG that explicitly derives the discrepancy between domains and drops the channels with significant distribution discrepancy. Given a training batch, we perform two ways of standardization: (1) based on the variance/mean of the batch (i.e., sampled from all source domains) and (2) based on the variance/mean of domain-wise samples in the batch. With the two normal distributions, we explicitly derive the discrepancy using KL-divergence and backpropagate it towards each channel. A channel with a higher contribution to the discrepancy is more likely to be dropped. Experimental results show the superiority of DgCD over the state-of-the-art DG baselines, demonstrating the effectiveness of our dropout strategy which is directly coupled to reducing the domain discrepancy. Our code is available at: https://github.com/gyeomo/DgCD