What Matters for Out-of-Distribution Detectors using Pre-trained CNN?

Abstract

In many real-world applications, a trained neural network classifier may have inputs that do not belong to any classes of the dataset used for training Such inputs are called out-of-distribution (OOD) inputs. Obviously, OOD samples may cause the classifier to perform unreliably and inaccurately. Therefore, it is important to have the capability of distinguishing the OOD inputs from the in-distribution (ID) data. To improve the detection capability, quite a few methods using pre-trained convolutional neural networks(CNNs) with OOD samples have been proposed. Even though these methods show good performance in various applications, the OOD detection capabilities may vary depending on the implementation details and the methodology how to apply a set of detection methods. Thus, it is very important to choose both a good set of solutions and the methodology how to apply the set of solutions to maximize the effectiveness. In this paper, we carry out an extensive set of experiments to discuss various factors that may affect the OOD detection performance. Four different OOD detectors are tested with various implementation settings to find the configuration to achieve practically solid results.