0
$\begingroup$

My main background knowledge about Bayesian analysis comes from Doing Bayesian Data Analysis by John K. Kruschke.

I have a dataset with observations y (success, fail) and several categorical variables A (treatment), B (age, in 5 groups), C, D, E, etc.

Firstly, I want to answer a question if the treatment has nonzero effect in improving the success rate. So I fit a model

$f = \beta_0 + \beta_1[\text{A's Index}]$

$y \sim \text{bernoulli}(\text{inverse_logit}(f))$,

then I made a group comparison of A (A0, A1, A2) to check if the the difference between each group of A is nonzero or not.

Then I add another variable B and the interaction of A and B and fit

$f = \beta_0 + \beta_1[\text{A's Index}]+\beta2[\text{B's Index}] + \beta_{A \times B}$

$y \sim \text{bernoulli}(\text{inverse_logit}(f))$,

and I analyzed the group differences of A, B, interaction term.

Then I want to put all the categorical variables together and interaction term $\beta_{A\times B}$ to fit a full model. Then I find out these variables show a significance in between-group difference, which then I claim it's a contribution.

I want to know if this is Bayesian at all. As I have too many categorical variables, I also read this article. It reviewed several methods for variable selection; however, does it still not have very popular way for variable selection of categorical variables. My major task is to answer question of A correctly. Then I want to find out other variables also has a contribution on the success rate or improvement of success. Is there a standard way of doing this? Could anyone share a reference?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

Whether the "difference is significant or not" is not part of Bayesian logic. It would be good to edit that part of the post.

It is not clear at all why you entertained more than one model. It's better to formulate a complete model, then to assess evidence for nonzero or large effects on parameters in that one model. And logistic model assumptions (here, additivity on the log odds scale) cannot be simultaneously true for a model and a sub-model.

So formulate a subject-matter-driven model, choose priors, and draw inferences from that model. From the multivariate posterior parameter samples you can easily make marginal assessments by concentrating on a single parameter at a time, or a derived parameter (combination of other parameters). For example you can compute $\Pr(\theta > 0)$ or $\Pr(|\theta| > 0.2)$.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for pointing the error. Should I concern the collinearity introduced by all my variables (around 100)? My first thought for 3 models is because answering the treatment effect is the most important task and finding the age group that's more prone to be cured. If I formulate just one model, should I concern about the correlation between variables? Or is there a way to avoid the impact of correlation and guarantee correct analysis on single variable? $\endgroup$ Commented Jan 16, 2022 at 13:49
0
$\begingroup$

I am not sure you need variable selection here, how many obs do you have?

You can just iteratively add variables to the model and see what changes, as you have started to do.

You can assume 0-centered slim priors for your betas to test whether posterior is different from 0.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I have 2000 observations and around 100 categorical variables. $\endgroup$ Commented Jan 16, 2022 at 13:50
  • $\begingroup$ How many successes? failures? Your effective sample size (the minimum of these two frequencies) probably does not support an analysis of 100 variables. You need some structure and possibly data reduction (unsupervised learning). Model age as a continuous variable allowing for nonlinearity. $\endgroup$ Commented Jan 17, 2022 at 13:06

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.