Convolutional Network with Densely Backward Attention for Facial Expression Recognition

Abstract

The emergence of convolutional neural network (CNN) has enabled facial expression recognition to accomplish significant outcomes nowadays. However, while existing multi-stream networks are subject to costly computation, the attention-embedded approaches do not involve multiple levels of semantic context in the predefined CNN. Based on the observation that emotions via a person's face are fusion of various muscular modalities, relying upon the outputs and corresponding attentional features of the deepest layer in the CNN is insufficient due to loss of informative details through multiple sub-sampling stages. Therefore, this paper introduces a CNN with densely backward attention to leverage the aggregation of channel-wise attention at multi-level features in a backbone network for reaching high recognition performance with cost-effective resource consumption. Particularly, cross-channel semantic information in high-level features are exploited densely to recalibrate fine-grained details in low-level versions. Then, a step of multi-level aggregation is further executed for thorougly involving spatial representations of important facial modalities. As a consequence, the proposed approach gains highest mean class accuracy of 79.37% on RAF-DB, which is competitive with the state-of-the-arts.