I read about PCA online and the way it computes a covariance matrix, computes eigenvalues, and then transforms the matrix to reduce the dimensionality of the matrix to a certain number k <= p. After I do this, can i just use this new matrix with any classifier I want, e.g. SVM and random_forest. The way i interpreted it, PCA just performs a transformation that tries to reduce the dimensionality while loosing as little as info as possible. Therefore to me it seems like it should be fine. Are there any reasons you can't?
$\begingroup$ $\endgroup$
Add a comment |
1 Answer
$\begingroup$ $\endgroup$
1 Yes, that's fine to do in many situations, and is a somewhat common practice when the original dimension is high.
You do lose some level of interpretability for methods that care about the actual feature dimensions: sparsity-inducing models now likely don't make any sense, and trees and forests suddenly become harder to inspect and understand.
- $\begingroup$ thanks for the help. luckily I don't find myself needing interpretability that much these days... $\endgroup$mt88– mt882015-08-14 05:45:46 +00:00Commented Aug 14, 2015 at 5:45