I often see the claim that post-hoc power is nonsense. However, it is unclear what the post-hoc power they criticize is.
My Question
What is the post-hoc power in the following experiment?
Experiment:
We randomly divide 20 animals into two groups, Group A and Group B. After that, for Group A, Foods A are fed, and for Group B, Foods B are fed. After a certain period, bodyweight was measured, and the data were as follows.Group_A :40.2, 40.4, 40.6, 40.8, 41.0, 41.2, 41.4, 41.6, 41.8
Group_B :30.1, 30.3, 30.5, 30.7, 30.9, 31.1, 31.3, 31.5, 31.7, 31.9, 32.1I would like to conduct a two-sided test with a significance level of 0.05 to see if there is a significant difference between the two groups.
I think it is one of the following ones. Both codes are written in "R". R source codes can be downloaded from the following link.
Method 1
#Load data Group_A = c(40.2, 40.4, 40.6, 40.8, 41.0, 41.2, 41.4, 41.6, 41.8) Group_B = c(30.1, 30.3, 30.5, 30.7, 30.9, 31.1, 31.3, 31.5, 31.7, 31.9, 32.1) # Welch Two Sample t-test t.test(Group_A,Group_B) library(effsize) library(pwr) cd = cohen.d(Group_A, Group_B) cd pwr.t2n.test(n1 = 9, n2= 11, d = cd$estimate, sig.level = 0.05, power = NULL, alternative = c("two.sided")) Method 2
# Load data Group_A = c(40.2, 40.4, 40.6, 40.8, 41.0, 41.2, 41.4, 41.6, 41.8) Group_B = c(30.1, 30.3, 30.5, 30.7, 30.9, 31.1, 31.3, 31.5, 31.7, 31.9, 32.1) # Welch Two Sample t-test twel=t.test(Group_A,Group_B) twel pwel=twel$p.value library(effsize) library(pwr) cd = cohen.d(Group_A, Group_B) cd pwr.t2n.test(n1 = 9, n2= 11, d = cd$estimate, sig.level = pwel, power = NULL, alternative = c("two.sided")) Which is the “correct” post-hoc power calculation code?
Notes:
If your "R" environment does not have packages named "effsize" and "pwr", you need to install them previously. If the following command is executed on R while connected to the Internet, installation should start automatically.
install.packages("effsize") install.packages("pwr") P.S. I'm not very good at English, so I'm sorry if I have some impolite or unclear expressions. I welcome any corrections and English review. (You can edit my question and description to improve them)