3
$\begingroup$

I'd like to start off by saying I am very much a beginner to statistical analyses, so I might be missing some key ideas here.

Context: My study involves an experimental group and control group that received different exercises. Within the groups, some participants identified as male, some as female. I measured participant satisfaction in completing the exercises.

Question: I want to know if the experiences of female students within the two groups differed in terms of how satisfied they were.

Tests run: (1) A two-way ANOVA with female/male + group (experimental or control) as my two independent variables, and satisfaction measure (based on survey of scale questions) as dependent variable. (2) I conducted t-test of just female students and compared their motivation ratings in experimental vs. control group, and another t-test of just male students and compared their satisfaction ratings in experimental vs. control group.

Results: (1) The ANOVA showed non-significant effects (p > 0.05) for both gender, and gender * group interaction. (2) The t-tests showed that female students had significantly higher satisfaction in experimental group, and male students had no difference in satisfaction among the two groups.

Question: Doesn't the results of (2) suggest an interaction effect? Unless I'm misunderstanding what ANOVA results mean completely.. I checked all assumptions of normality and equality of variances before running each of the tests using SPSS.

Improve this question
$\endgroup$
2
  • $\begingroup$ How many participants were in each group for your IVs? Also, what were the p-values when you ran the ANOVA? $\endgroup$ Commented Dec 14, 2019 at 22:38
  • $\begingroup$ Yes, please give us the ANOVA and t-test results. It seems that the group effect was significant? But that was mainly driven by females? $\endgroup$ Commented Jul 15, 2023 at 11:34

2 Answers 2

Reset to default
1
$\begingroup$

A better way to approach this analysis could be to:

  1. Fit the ANOVA model to test the overall significance and/or;
  2. Perform pairwise comparisons on the ANOVA model to test individual effects for significance while controlling the Family Wise Error Rate (FWER), e.g. Tukey's HSD test, Dunnett's test. Your t-test results apparently were not adjusted for multiple comparisons?

You might also find the following posts helpful:

Improve this answer
$\endgroup$
0
$\begingroup$

The basic problem here is interpreting the main effect when there's a significant interaction. This is generally bad practice.

A secondary issue is comparing an ANOVA with a t-test on a much smaller sample size. The degrees of freedom in the models are very different.

Using the output of the ANOVA makes most sense here.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.