In my experiment design, I will be testing subjects multiple times under two conditions: in a controlled lab setting and in real-life situations. Testing subjects repeatedly is necessary as performance may vary, and I intend to conduct tests on them in groups. Therefore, I will utilize a linear mixed model for analysis:
lmer(milliseconds ~ condition + (1 + condition | subject / testgroup), data = .)
The dependent variable (Y) ranges from 100 ms to 1500 ms. However, the scale of this variable lacks inherent meaning for communication purposes. Consequently, I am less interested in raw differences, as they may not be practically meaningful for the research question.
My hypothesis Is that that the subject will perform equivalent in the different conditions.
Lakens et al. (2018) suggest considering both raw equivalence bounds and standardized equivalence bounds, depending on the theoretical importance of the raw mean differences. In this case, since the dependent variable lacks theoretical importance, and standardized effect size differences are easier to communicate, I opt for standardized differences.
As recommended by Lakens et al. (2018), I plan to conduct both a null-hypothesis significance test and an equivalence test on the data, provided I can justify a SESOI.
I will use Cohen’s d of 0.5 as the SESOI and the bounds. However, I am unsure about conducting the equivalence test and require further guidance on this step.
However, my question in this post is whether I can perform a power analysis solely on the effect size differences. Additionally, can I solely focus on analyzing the effect size differences and similarities in my analysis? This is because setting a SESOI of 0.5 is an easier way to communicate. Since it's not common to do this type of research, I struggle to formulate a raw SESOI, and it also doesn't make much sense to use one.
reference: DOI: 10.1177/2515245918770963