Timeline for Intuition behind logistic regression
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 27, 2013 at 19:42 | comment | added | Bill | @AdamO This is a manifestation of the usual economist/statistician divide, but . . . I don't think logistic regression is semi-parametric. The statistical model is $P(Y_i=1)=\frac{exp(X_i\beta)}{1+exp(X_i\beta)}$. That's parametric. One can (and I do) interpret it as coming from a threshold model with logistic error. If I get worried about making too many assumptions on the error term, I am going to drop logistic regression, not the threshold model. Threshold models can be estimated with much weaker assumptions on the error terms using maximum score and related estimators, for example. | |
| Sep 27, 2013 at 18:56 | comment | added | AdamO | But why not use the estimating equation approach? With semiparametric inference, you have much fewer assumptions about the distribution of error terms. The parameters are consistent, estimable, and have real scientific interpretation. When I think of estimation, I think statisticians are interested in estimating trends and not errors. | |
| Sep 27, 2013 at 18:35 | comment | added | Bill | @AdamO It's really not that sensitive an assumption. There is an old paper (Amemiya, 1981, J of Economic Literature) surveying discrete response models which discusses the rule of thumb that the coefficients on the parameters in a linear probability model, a probit, and a logit on the same data were in proportion 1: 2.5 : 4. This has certainly been my experience fitting these models. There was a bit of theoretical work done subsequently on why this was so. | |
| Sep 27, 2013 at 17:22 | comment | added | Macro | @AdamO, however you motivate logistic regression, it's still mathematically equivalent to a thresholded linear regression model where the errors have a logistic distribution. I agree that this assumption may be hard to test but it's there regardless of how you motivate the problem. I recall a previous answer on CV (I can't place it right now) that showed with a simulation study that trying to tell whether a logistic or probit model "fit better" was basically a coin flip, regardless of the true data generating model. I suspect logistic is more popular because of the convenient interpretation. | |
| Sep 27, 2013 at 16:52 | comment | added | AdamO | That seems like a very sensitive assumption and one that would be difficult to test. I think logistic regression can be motivated when such error distributions don't hold. | |
| Sep 26, 2013 at 22:50 | comment | added | Macro | @AdamO, if the $\epsilon_i$ have a logistic distribution, then this describes logistic regression. | |
| Sep 26, 2013 at 22:39 | comment | added | AdamO | What you described is exactly the motivation for the probit model, not logistic regression. | |
| Sep 26, 2013 at 21:48 | history | edited | Bill | CC BY-SA 3.0 | insert caveat |
| Sep 26, 2013 at 21:41 | history | answered | Bill | CC BY-SA 3.0 |