Multi-Structure KELM with Attention Fusion Strategy for Hyperspectral Image Classification

Abstract

Hyperspectral image (HSI) classification refers to accurately corresponding each pixel in an HSI to a land-cover label. Recently, the successful application of multi-scale and multi-feature methods has greatly improved the performance of HSI classification due to their enhanced utilization of the available spectral-spatial information. However, as the number of scales and the number of features increases, it becomes more difficult to achieve an optimal degree of fusion for multiple classifiers (e.g., Kernel Extreme Learning Machine, KELM). On the other hand, a limited sample size of the HSI may cause overfitting problems, which seriously affects the classification accuracy. Therefore, in this paper, a novel Multi-Structure KELM with Attention Fusion Strategy (MSAF-KELM) is proposed to achieve accurate fusion of multiple classifiers for effective HSI classification with ultra-small sample rates. First, a multi-structure network is built that combines multiple scales and multiple features to extract abundant spectral-spatial information. Second, a fast and efficient KELM is employed to enable rapid classification. Finally, a Weighted Self Attention Fusion Strategy (WSAFS) is introduced, which combines the output weights of each KELM sub-branch and the self-attention mechanism to achieve an efficient fusion result on multi-structure networks. We conducted experiments on four types of HSI datasets with different evaluation methods and compared them with several classical and state-of-the-art methods, which demonstrate the excellent performance of our method on ultra-small sample rates. The code is available at https://github.com/Fang666666/MSAF-KELM for reproducibility. IEEE