I need help adjusting for p-values. I have a cohort of 200 patients. I am using 3 metrics to divide this group into different categories. Here is how they are distributed.
| Metric A Category | Number of patients |
|---|---|
| None | 120 |
| Mild | 55 |
| Moderate | 20 |
| Severe | 5 |
| Metric B Category | Number of patients |
|---|---|
| Low risk | 125 |
| Medium risk | 55 |
| High risk | 20 |
| Metric C Category | Number of patients |
|---|---|
| Disease | 160 |
| Without disease | 40 |
I am interested in whether there are significant differences in the body mass index and apolipoprotein A levels between groups of Metric A. I am also interested in the same variables but for Metric B. And also for Metric C. I decided to use Kruskal-Wallis tests since none of the metrics had distributions that satisfied the assumptions of ANOVA.
I originally used the Bonferroni method with an alpha of 0.05 to correct the p-values yielded by the Kruskal-Wallis tests. Since I was interested in two variables, apolipoprotein A and body mass index, I thought I should use a m =2. This yielded me an adjusted p-value of 0.025. I used this p-value threshold to determine which results from The Kruskal-Wallis tests were significant.
Only Metric A had p-values < 0.025. Afterward, I used a Conover-Iman test with an alpha of 0.05 and a p-adjustment with Holm-Bonferroni method to determine whether apolipoprotein A and BMI were different between groups in Metric A.
However, I am not sure if this is the right approach. From posts in this site, some suggest using all the number of hypothesis testing to adjust for p. Using the example of the post, this would be (4 * 2) + (3 * 2) + (2 * 2) = 18 in my case. So m = 18, which would give me a threshold of p < 0.002. The other approach I saw is to just do the p-value adjustment after a post-hoc test without any correction beforehand. Which is the correct approach?
