Skip to main content
Cross Validated

Return to Answer

added 104 characters in body
Source Link
amoeba
amoeba
  • 109.1k
  • 37
  • 325
  • 350

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix is equivalent to standardisesstandardizing each of the datavariables (to mean 0 and standard deviation 1). In general they, PCA with and without standardizing will give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon[,-8] # look at heptathlon data (excluding 'score' variable)

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon[,-8], scale=TRUE) hep.PC.cov = prcomp(heptathlon[,-8], scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $82\%$ of the variance) and PC2 is almost equal to javelin (together they explain $97\%$). PCA on correlation is much more informative and reveals some structure in the data and relationships between variables (but note that the explained variances drop to $64\%$ and $71\%$).

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix standardises the data. In general they give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon[,-8] # look at heptathlon data (excluding 'score' variable)

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon[,-8], scale=TRUE) hep.PC.cov = prcomp(heptathlon[,-8], scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $82\%$ of the variance) and PC2 is almost equal to javelin (together they explain $97\%$). PCA on correlation is much more informative and reveals some structure in the data and relationships between variables (but note that the explained variances drop to $64\%$ and $71\%$).

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix is equivalent to standardizing each of the variables (to mean 0 and standard deviation 1). In general, PCA with and without standardizing will give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon[,-8] # look at heptathlon data (excluding 'score' variable)

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon[,-8], scale=TRUE) hep.PC.cov = prcomp(heptathlon[,-8], scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $82\%$ of the variance) and PC2 is almost equal to javelin (together they explain $97\%$). PCA on correlation is much more informative and reveals some structure in the data and relationships between variables (but note that the explained variances drop to $64\%$ and $71\%$).

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

fixed my mistake in the previous edit (inserted [,-8] back)
Source Link
amoeba
amoeba
  • 109.1k
  • 37
  • 325
  • 350

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix standardises the data. In general they give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlonheptathlon[,-8] # look at heptathlon data (excluding 'score' variable)

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# PCA # scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlonheptathlon[,-8], scale=TRUE) hep.PC.cov = prcomp(heptathlonheptathlon[,-8], scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $98\%$$82\%$ of the variance) and PC2 is almost equal to javelin (together they explain $97\%$). PCA on correlation looks much more informativePCA on correlation is much more informative and reveals some structure in the data and relationships between variables (but note that the explained variances drop to $64\%$ and $71\%$).

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix standardises the data. In general they give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon # look at heptathlon data

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# PCA # scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon, scale=TRUE) hep.PC.cov = prcomp(heptathlon, scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $98\%$ of the variance) and PC2 is almost equal to javelin. PCA on correlation looks much more informative and reveals some structure in the data and relationships between variables.

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix standardises the data. In general they give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon[,-8] # look at heptathlon data (excluding 'score' variable)

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon[,-8], scale=TRUE) hep.PC.cov = prcomp(heptathlon[,-8], scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $82\%$ of the variance) and PC2 is almost equal to javelin (together they explain $97\%$). PCA on correlation is much more informative and reveals some structure in the data and relationships between variables (but note that the explained variances drop to $64\%$ and $71\%$).

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

note about the explained variance when PCA is done on covariance
Source Link
amoeba
amoeba
  • 109.1k
  • 37
  • 325
  • 350

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix standardises the data. In general they give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon # look at heptathlon data

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# PCA # scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon, scale=TRUE) hep.PC.cov = prcomp(heptathlon, scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $98\%$ of the variance) and PC2 is almost equal to javelin. PCA on correlation looks much more informative and reveals some structure in the data and relationships between variables.

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix standardises the data. In general they give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon # look at heptathlon data

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# PCA # scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon, scale=TRUE) hep.PC.cov = prcomp(heptathlon, scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m and PC2 is almost equal to javelin. PCA on correlation looks much more informative and reveals some structure in the data and relationships between variables.

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales.

Using the correlation matrix standardises the data. In general they give different results. Especially when the scales are different.

As an example, take a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (run 800m) are around 120.

library(HSAUR) heptathlon # look at heptathlon data

This outputs:

 hurdles highjump shot run200m longjump javelin run800m Joyner-Kersee (USA) 12.69 1.86 15.80 22.56 7.27 45.66 128.51 John (GDR) 12.85 1.80 16.23 23.65 6.71 42.56 126.12 Behmer (GDR) 13.20 1.83 14.20 23.10 6.68 44.54 124.20 Sablovskaite (URS) 13.61 1.80 15.23 23.92 6.25 42.78 132.24 Choubenkova (URS) 13.51 1.74 14.76 23.93 6.32 47.46 127.90 ...

Now let's do PCA on covariance and on correlation:

# PCA # scale=T bases the PCA on the correlation matrix hep.PC.cor = prcomp(heptathlon, scale=TRUE) hep.PC.cov = prcomp(heptathlon, scale=FALSE) biplot(hep.PC.cov) biplot(hep.PC.cor)

PCA on correlation or covariance

Notice that PCA on covariance is dominated by run800m and javelin: PC1 is almost equal to run800m (and explains $98\%$ of the variance) and PC2 is almost equal to javelin. PCA on correlation looks much more informative and reveals some structure in the data and relationships between variables.

Notice also that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

massive edit to simplify the code and insert the biplots. Trying to improve this well-upvoted and accepted answer to make it more useful
Source Link
amoeba
amoeba
  • 109.1k
  • 37
  • 325
  • 350
added output image
Source Link
edit approved Dec 31, 2014 at 14:33
mreq
edit approved Dec 31, 2014 at 14:33
mreq
  • 215
  • 2
  • 8
added 1 characters in body
Source Link
csgillespie
csgillespie
  • 13k
  • 9
  • 63
  • 91
Added R code
Source Link
csgillespie
csgillespie
  • 13k
  • 9
  • 63
  • 91
Source Link
csgillespie
csgillespie
  • 13k
  • 9
  • 63
  • 91