Researchers develop new feature selection method for limited-sample industrial data

A research team from the Ningbo Institute of Materials Technology and Engineering of the Chinese Academy of Sciences has introduced a novel feature selection method by removing noise entropy within mutual information. This study was published in IEEE Transactions on Industrial Informatics.

Feature selection, a critical step in machine learning and data mining, aims to reduce dimensionality by eliminating irrelevant or redundant features, thereby improving model performance. However, industrial data, often characterized by small sample sizes and high dimensionality, pose significant challenges, including high computational costs and the risk of overfitting.

Traditional methods struggle to maintain accuracy when dealing with such data, particularly in the presence of sensor noise, which can distort mutual information metrics and degrade classification performance.

To overcome these limitations, the research team proposed an approach by modeling feature noise as a censored normal distribution. Leveraging the principle of maximum entropy, they determined the entropy of noise by solving the variance equation in transmission.

Additionally, the researchers developed a noise-free mutual information metric to assess the relevance of a label and noise-corrupted features. Thus, the entropy of unknown feature noise within mutual information was removed while retaining noisy samples, eliminating the impact of noise in classification with limited samples.

The proposed method outperforms conventional techniques by providing a more reliable assessment of noise across all noisy samples. Building on this, the researchers introduced a novel criterion called Maximal Noise-Free Relevance and Minimal Redundancy (MNFR-MR), which ensures robust feature selection.

This approach addresses a critical bottleneck in processing industrial data, particularly in scenarios where sample sizes are constrained. As industries increasingly adopt data-driven technologies such as the Industrial Internet of Things (IIoT) and digital twins, this method holds significant promise for unlocking actionable insights and improving decision-making across various domains.

This study not only advances the theoretical understanding of feature selection in noisy, high-dimensional datasets but also offers practical solutions for real-world industrial applications, paving the way for more accurate and efficient data-driven intelligence.