There are simple relationships for the bounds of $\lambda$ where you get all coefficients zero and where you get none of the coefficients zero.
LASSO: Deriving the smallest lambda at which all coefficient are zero $$\lambda_{\text{all zero}} = \sqrt{k} \vert \vert X^T y\vert \vert_\infty$$
What is the smallest $\lambda$ that gives a 0 component in lasso? $$\lambda_{\text{non zero}} = \frac{d}{n} \vert \vert X \vec{\beta}_{last,normalized} \vert \vert_2^2 $$ with $$\vec{\beta}_{last,normalized} = \frac{\vec{\beta}_{last}}{\sum \vec{\beta}_{last} \cdot sign(\hat{\beta})} $$ and $$\vec{\beta}_{last} = - (X^TX)^{-1} \text{sign} (\hat{\beta}) $$
A demonstration of the above formula's with some code
library(lars) ### generate some data set.seed(1) n = 50 m = 20 X = matrix(rnorm(m*n),n) y = rnorm(n) ### perform lasso mod = lars(X,y, intercept = 0, normalize = 0) nonzero = rowSums(mod$beta^2 > 0) lambda = mod$lambda lambda = c(lambda[1]+1,lambda) # add one additional point ### plot the number of nonzero estimated coefficients ### as function of lambda plot(lambda,nonzero, log ="x", type = "s") ### upper bound above which all coefficients are zero l_max = max(abs(t(X) %*% y)) lines(l_max*c(1,1), c(-1,m+1), lty = 2, col = 2) ### lower bound below which all coefficients are non-zero mod = lm(y~0+X) beta = mod$coefficients bl = - solve(t(X)%*%X) %*% sign(beta) bln = bl/sum(bl*sign(beta)) br = beta/bln d = min(br[br>0]) l_min = sum((X %*% bln)^2)*d lines(l_min*c(1,1), c(-1,m+1), lty = 2, col = 2)