LossDistillNet: 3D Object Detection in Point Cloud Under Harsh Weather Conditions

Abstract

Recently, 3D object detection models have achieved very good performance under normal weather conditions, with the SE-SSD model having produced the highest performance by exchanging features between the teacher and student models. However, the performance of this model is significantly reduced by adverse weather conditions. Therefore, instead of training the teacher and student models simultaneously, we applied the knowledge distillation algorithm. In this algorithm, the teacher model is trained first by normal input, and the student model is then trained with distillation and student loss by adverse weather condition input. Although recent research has focused on combining different types of sensor inputs to enhance the original model's performance in inclement weather, there are no studies that directly address the problem of missing points for point clouds. Accordingly, we applied a probability estimation, which includes a Deep Mixture of Factor Analyzers (DMFA) network and loss-convolution layer, to recover lost points. We conducted a model evaluation in both fog and snow environments at three levels of density - light, medium, and heavy - and compared the proposed model's performance with that of two state-of-the-art models: one with normal weather condition, and the other with harsh weather conditions. Consequently, our proposed method was shown to significantly outperform the two existing models.