3
$\begingroup$

I am trying to fit a multinomial logit model using glmnet. I have a few questions:

  1. How is the baseline category specified?

  2. Looking at the model coefficients using coef.glmnet, I'm thinking that many are given by a dot. I assume this means the coefficient was set to zero by LASSO, so the variable was dropped. However, I'm finding that variables are dropped this way even when lambda=0, so there is no regularizing term. Can someone explain what's going on here?

  3. Sometimes I find that a variable is dropped in some but not all of the logits. How should this be interpreted?

Here is a working example: 

library(glmnet) data(iris) X <- model.matrix(Species ~., data=iris) y <- iris$Species fit <- glmnet (X, y, family="multinomial") coef(fit, s=0)
Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Please ask only one question per post ( you can link between them for common background) $\endgroup$ Commented Sep 4, 2018 at 19:59

1 Answer 1

Reset to default
2
$\begingroup$

Q1: glmnet do not use (or need) a baseline category. The penalization takes care of the linear dependence, no matrix is inverted during the fitting! Furthermore, it would be an error to use a baseline category, because different choices would lead to fitted models giving different predictions. See discussion in Dropping one of the columns when using one-hot encoding

Q2: Not sure about this, probably has to do with details about the algorithm (which is iterative, not exact). Leaving that one to others.

Q3: The usual lasso do not know about different dummys belonging to the same categorical variable, so it might choose to set different variables to zero. To avoid that, use the group lasso. See Why use group lasso instead of lasso?

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.