13
$\begingroup$

I recently fit 4 multiple regression models for the same predictor/response data. Two of the models I fit with Poisson regression.

model.pois <- glm(Response ~ P1 + P2 +...+ P5, family=poisson(), ...) model.pois.inter <- glm(Response ~ (P1 + P2 +...+ P5)^2, family=poisson(), ...)

Two of the models I fit with negative binomial regression.

library(MASS) model.nb <- glm.nb(Response ~ P1 + P2 +...+ P5, ...) model.nb.inter <- glm.nb(Response ~ (P1 + P2 +...+ P5)^2, ...)

Is there a statistical test I can use to compare these models? I've been using the AIC as a measure of the fit, but AFAIK this doesn't represent an actual test.

Improve this question
$\endgroup$
7
  • $\begingroup$ You want to compare the models' fit using a statistical test, right? What kind of hypothesis would you like to test? $\endgroup$ Commented Dec 16, 2010 at 20:56
  • $\begingroup$ @Firefeather For example, I would like to test whether the fit of model.nb.inter is significantly better than that of model.pois.inter. Yes, the AIC is lower, but how much lower constitutes significantly better? $\endgroup$ Commented Dec 16, 2010 at 21:00
  • $\begingroup$ Note: the answer to this question need not actually include the AIC. $\endgroup$ Commented Dec 16, 2010 at 21:07
  • $\begingroup$ I don't know the answer to this question, but I can give a start. I know you can use an $F$ test to compare model.pois against model.pois.inter (and similarly compare model.nb against model.nb.inter), but I can't guarantee that comparisons between a Poisson model and a negative binomial model would work. I wonder if an $F$ test to compare the variances of each pair would be reliable. $\endgroup$ Commented Dec 16, 2010 at 21:16
  • 1
    $\begingroup$ @Firefeather, yes I'm aware of the need to control the familywise confidence level. Would Scheffe be more appropriate here than, say, Bonferroni? $\endgroup$ Commented Dec 16, 2010 at 21:40

3 Answers 3

Reset to default
17
$\begingroup$

You can compare the negative binomial model to the corresponding Poisson model with a likelihood ratio test. A Poisson model is equivalent to a negative binomial model with an overdispersion parameter of zero. Therefore they are nested models and likelihood ratios are valid. The complication is that the overdispersion parameter is restricted to be non-negative, i.e. it logically can't be less than zero, so the null hypothesis is on the boundary of the parameter space. This means that instead of comparing twice the log-likelihood to a chi-squared distribution with one degree of freedom, you need to compare it to a mixture distribution consisting of equal parts of a chi-squared with 1 d.f. and a point mass at zero (a chi-squared distribution with zero degrees of freedom). What that means in practice is that you can calculate the p-value using the chi-squared with 1 d.f. and then halve it. For more details and background, see Case 5 of Self & Liang JASA 1987; 82:605-610..

Note that some statistical software packages, such as Stata, will do this all for you automatically when you fit a negative binomial model. In fact I've shamelessly cribbed much of the above from the Stata help system -- if you have Stata see help j_chibar.

Improve this answer
$\endgroup$
5
$\begingroup$

I believe anova() in R can be used for this. Despite its name, it's a likelihood ratio test. Crawley in his The R Book has some examples of usage.

Improve this answer
$\endgroup$
1
$\begingroup$

As onestop notes, because the models are nested you can perform a likelihood ratio test.

In general though that isn't true, so if you want to compare non-nested models you could use Vuong's test.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.