3
$\begingroup$

I'm performing a principal component analysis (PCA) using some economic variables of a region. I have six variables and I want to reduce them to two principal components. Most of the variables have a larger value in either one of the first two principal components' eigenvectors. For example, they either have a large value in the first principal component [eigenvector] and near zero in the second, or vice versa.

However, I have one variable that has almost the same value in the eigenvectors of the first and the second principal components.

Does this tell me anything about this variable? Should I keep it or should I remove it from the analysis?

Improve this question
$\endgroup$
1
  • $\begingroup$ I have one variable that has almost the same value... Eigenvector entry is the cosine of the angle between the component and the variable axes in space. Should I keep it or... We don't know the aims and nuances of your particular analysis. What makes you think you have to get rid of such a variable? $\endgroup$ Commented Jun 26, 2015 at 9:11

1 Answer 1

Reset to default
1
$\begingroup$

Eigenvectors are just giving you the "directions" of the principal component axes; typically, those are unit vectors. In PCA, you order the eigenvectors by decreasing eigenvalues; the eigenvalues tell you about how much "variance is explained" by the eigenvectors (you principal component axes). E.g., if you use PCA for dimensionality reduction on a linear task, you'd want to choose the top k eigenvectors that explain most of the variance (contain the most information).

As mentioned above, you can calculate the "variance explained" based on the magnitude of the eigenvalues; I plotted the "variance explained" for the Iris dataset below:

enter image description here

In this plot, you can see that the first two principal components (the eigenvectors that correspond to the 2 largest eigenvalues) explain almost all of the variance in this dataset (>95 %).

I have a short tutorial and code examples here if you want to reproduce the results.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I don't see how this answers the original question at all. The question was about one variable entering two PCs with similar weights; what does this mean and what should be done about it. You were replying to something else. $\endgroup$ Commented Jun 28, 2015 at 12:36

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.