0
$\begingroup$

What is the behavior of a Lasso estimator if it is used in a dataset with more predictors (p) than observations (n), where all predictors are uncorrelated but highly relevant to 𝑦 y with exactly the same correlation with 𝑦 y? Which predictors does the Lasso estimator shrink to zero and which does it retain?

A consistent estimator would not reduce any of the 𝑝 p variables to zero. However, as I understand, the Lasso estimator would select at most 𝑛 n predictors. My question is: given these conditions, which predictors does Lasso select and why?

Improve this question
$\endgroup$
7
  • $\begingroup$ The answer will likely depend on floating point rounding error and be algorithm dependent, because the mathematical answer is "it's perfectly arbitrary." See our posts about the Lasso for an explanation. $\endgroup$ Commented Jun 10, 2024 at 15:21
  • 3
    $\begingroup$ I'm glad to see this compilation of lasso issues. When $p > n$ no one should accept the results without a simulation showing stability and reliability of the approach. The results of such simulations are typically quite disconcerting. $\endgroup$ Commented Jun 10, 2024 at 15:25
  • $\begingroup$ Why would the lasso select at most $n$ predictors? $\endgroup$ Commented Jun 10, 2024 at 19:29
  • 1
    $\begingroup$ @RichardHardy check out my answer here: stats.stackexchange.com/a/631944/341520 1) because it answers your and also kind of this question and 2) because I'm very proud of it :) $\endgroup$ Commented Jun 10, 2024 at 20:29
  • $\begingroup$ @whuber thanks. Do you have a link to the post that described that the variable selection would be arbitrary in this case? $\endgroup$ Commented Jun 11, 2024 at 8:04

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.