3
$\begingroup$

My question is should you use the same algorithm in feature selection as your model?

If I'm using a KNN model for classification should I also use a KNN algo when running feature selection? Or lightGBM algo for feature selection and lightGBM model?

Or is there a best algo for feature selection that you can use those features with any model?

I know there are several different feature engineering techniques I just don't know which techniques should be used with different machine learning algorithms.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ there is no theoretical reason not to use same (class of) algorithms both for feature selection and as model $\endgroup$ Commented Jan 5, 2021 at 18:27
  • 1
    $\begingroup$ In theory, a drawback of using different models could be that features being important to your model being used as a filter to select features might not have the same importance for your model applied during inference $\endgroup$ Commented Jan 5, 2021 at 20:34

1 Answer 1

Reset to default
2
$\begingroup$

Yes - You can use the same algorithm for both feature selection and prediction. Common examples are L1 regression and tree-based algorithms. Those algorithms both find the most important features and predict targets using the same fitting mechanism.

The difference between those algorithms and your examples is that those algorithms do both steps concurrently and your examples are sequential.

There is no single best algorithm for feature selection that can also be used with other models for prediction. Different algorithms will perform better on different datasets.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.