Interleaved incremental association Markov blanket as a potential feature selection method for improving accuracy in near-infrared spectroscopic analysis

Abstract

The interleaved Incremental Association Markov Blanket (inter-IAMB) is described herein as a feature selection method for the NIR. spectroscopic analysis of several samples (diesel, gasoline, and etchant solutions). Although the Markov blanket (MB) has been proven to be the minimal optimal set of features (variables) that does not change the original target distribution, variables selected by the existing IAMB algorithm could be redundant and/or misleading as the IAMB requires an unnecessarily large amount of learning data to identify the MB. Use of the inter-IAMB interleaving the grow phase with the shrink phase to maintain the size of the MB as small as possible by immediately eliminating invalid candidates could overcome this drawback. In this report, a likelihood-ratio (LR)-based conditional independence test, able to handle spectroscopic data normally comprising a large number of continuous variables in a small number of samples, was uniquely embedded in the inter-IAMB and its utility was evaluated. The variables selected by the inter-IAMB in complexly overlapped and feature indistinct NIR spectra were used to determine the corresponding sample properties. For comparison, the properties were also determined using the IAMB-selected variables as well as the whole variables. The inter IAMB was more effective in the selection of variables than the IAMB and thus able to improve the accuracy in the determination of the sample properties, even though a smaller number of variables was used. The proposed LR-embedded inter-IAMB could be a potential feature selection method for vibrational spectroscopic analysis, especially when the obtained spectral features are specificity-deficient and extensively overlapped.