I have two alternative hierachical bayesian models that were designed to the describe the same process (from a high-level point-of-view). Both model provides comparable (but not identical) inferences on test data-sets and both exhibits satisfactory behaviours. So from a practical point-of-view, I have no reason to prefer one model over the other (while they rely on different views). One important difference between them consists in the number of auxiliary latent rv composing the model: one model is composed $N+1$ (scalar) latent rv whereas the other is composed of (scalar) $2N+1$ latent rvs($N$ being the nber of observed sample). Is there any reason to prefer a model with less auxiliary variables (regarless of the computational time)?
Add a comment |
1 Answer
$\begingroup$ $\endgroup$
1 Consider the following pointers:
- Adding parameters to a model is not cost-free, both from a classical and a Bayesian point of view.
Taylor, J. M. G., Siqueira, A. L. and Weiss, R. T. (2000). The cost of adding parameters to a model. Journal of the Royal Statistical Society, Series B 58: 593–607.
The Occam's razor can also be used to give an argument in favour of models with less parameters.
Conduct a formal model comparison (either by using AIC, BIC, DIC, Bayes factors or your favourite scoring rule) in order to obtain more believable evidence in favour of one of the models in question.
In vague words. Small or even moderate samples usually contain little information about hidden variables or hierarchical variables. For this reason, the less parameters you use, the more information you are likely to obtain from the sample.
- $\begingroup$ Thanks Ulysses, this helps me a lot. Concerning point 3; I tried DIC and penalized Loss functions (using jags) but they give me largely different results. DIC gives a difference of about 0.3 (in favour of the MORE complex model) whereas penalized Loss functions gives a difference of about 70 (in favour of the LESS complex model)! I am a bit confused about what to do with these results, can you give me some insights/references about the way to proceed to compare properly the model? (Inference is based on MCMC) $\endgroup$beuhbbb– beuhbbb2013-02-27 12:51:19 +00:00Commented Feb 27, 2013 at 12:51