Factor Analysis does not completely mitigate the singular covariance matrix problem!

I created a Jupyter notebook to gather some empirical evidence. The notebook uses Factor Analysis to generate a model and then finds the minimum training set size needed to avoid the singular covariance matrix problem. Here are the results:

As you can see, the training set size still needs to be greater than the number of factors to avoid the singular covariance matrix problem.

I don’t know the mathematical reason but I guess it makes sense conceptually. Factor Analysis reduces the dimensionality to $k$ factors, so the training set still needs to have more than $k$ examples to avoid Perfect Multicollinearity.

Factor Analysis does not completely mitigate the singular covariance matrix problem!

I created a Jupyter notebook to gather some empirical evidence. The notebook uses Factor Analysis to generate a model and then finds the minimum training set size needed to avoid the singular covariance matrix problem. Here are the results:

As you can see, the training set size still needs to be greater than the number of factors to avoid the singular covariance matrix problem.

I don’t know the mathematical reason but I guess it makes sense conceptually. Factor Analysis reduces the dimensionality to $k$ factors, so the training set still needs to have more than $k$ examples to avoid Perfect Multicollinearity. That being said, Factor Analysis is still useful for mitigating the singular covariance matrix problem in small training sets because one can choose $k$ to be less than the training set size.