Feature Selection in Machine Learning Book - Code Repository

• Published: August, 2022
• Updated: February, 2025

Links

• Book

Table of Contents

1. Basic Selection Methods

   1. Removing Constant Features
   2. Removing Quasi-Constant Features
   3. Removing Duplicated Features

2. Correlation Feature Selection

   1. Removing Correlated Features
   2. Smart Correlation

3. Statistical methods

   1. Chi-square distribution
   2. Anova
   3. Correlation
   4. Mutual information

4. Univariate Methods

   1. Single feature classifiers / regressors
   2. Target mean encoding

5. Wrapper Methods

   1. Exhaustive Feature Selection
   2. Step Forward Feature Selection
   3. Step Backward Feature Selection

6. Embedded Methods: Linear Model Coefficients

   1. Lasso
   2. Decision tree feature importance

7. Recursive Feature Elimination

   1. RFE - embedded importance
   2. RFE - model performance

8. Alternative Feature Selection Methods

   1. Feature Shuffling
   2. Recursive Feature Addition
   3. Probe Features
   4. MRMR

• Book