Timeline for Behavior of Lasso Estimator with More Predictors Than Observations (p > n) and Identical Correlations?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 12, 2024 at 10:42 | comment | added | Sextus Empiricus | The Lasso path is not easily predictable. That makes it difficult to say with a simple formula which parameters are gonna end up in the final minimal L1 norm solution (for two highly correlating features that are very much the same it can be that one of them ends up in the solution and the other not). It is related to saying how many zeros you are gonna get as function of the regularisation parameter stats.stackexchange.com/questions/639928/… | |
| Jun 11, 2024 at 12:40 | comment | added | whuber♦ | One does not need a post for that: the conclusion follows directly from the standard formulation of the Lasso objective function along with the basic theorem of linear algebra that in a vector space of dimension $n,$ any $p\gt n$ vectors are linearly dependent. I therefore provided a link to a collection of relevant posts so you can select explanations you find most helpful. | |
| Jun 11, 2024 at 8:04 | comment | added | Joe94 | @whuber thanks. Do you have a link to the post that described that the variable selection would be arbitrary in this case? | |
| Jun 10, 2024 at 20:29 | comment | added | Lukas Lohse | @RichardHardy check out my answer here: stats.stackexchange.com/a/631944/341520 1) because it answers your and also kind of this question and 2) because I'm very proud of it :) | |
| Jun 10, 2024 at 19:29 | comment | added | Richard Hardy | Why would the lasso select at most $n$ predictors? | |
| Jun 10, 2024 at 15:25 | comment | added | Frank Harrell | I'm glad to see this compilation of lasso issues. When $p > n$ no one should accept the results without a simulation showing stability and reliability of the approach. The results of such simulations are typically quite disconcerting. | |
| Jun 10, 2024 at 15:21 | comment | added | whuber♦ | The answer will likely depend on floating point rounding error and be algorithm dependent, because the mathematical answer is "it's perfectly arbitrary." See our posts about the Lasso for an explanation. | |
| Jun 10, 2024 at 15:04 | history | asked | Joe94 | CC BY-SA 4.0 |