5
$\begingroup$

I have a study that is designed such that it seems to straddle a within- and between-subject design. I suspect there is a rich literature on how to analyze data such as these, but I simply lack the terminology to find it.

My study is interested in attitudes toward the homeless and in particular whether ethnicity interacts with homelessness to increase stigma. To study this, participants read vignettes about someone and make ratings related to stigma. My independent variables are whether the character in the vignette has a home or not (homed, homeless) and ethnicity (white, black). In a perfect world, everyone would receive all four resulting conditions (homed-white, homed-black, homeless-white, homeless-black). However, due to time, each participant can receive only two. As such, each participant receives one of the following:

homeless-black / homed-white

homeless-white / homed-black

(the order of the two vignettes counterbalanced for each but that isn't important here)

This looks like a 2 x 2 repeated measures design in that each participant receives both levels of homelessness and both levels of ethnicity, but it isn't quite because they don't receive all four conditions, only two.

This sounds at surface level like a fractional factorial design (https://en.wikipedia.org/wiki/Fractional_factorial_design; see also, Mixed factor both "within" and "between" subjects) but in the examples I've read of those, the same portion of the design appears to be of a particular type (e.g., there is a portion of the design that is fully crossed and can be specified as within or between – something like one variable with two levels within-subjects and one level between).

It could be I'm thinking about it the wrong way, but in short I am seeking advice as follows:

1.) Is this properly a fractional design?

2.) How can I analyze these data using something akin to an ANOVA?

3.) Although I am not limited to R, I would prefer to use it, so recommendations for functions or packages appreciated.

Also, if I am simply turned around and this is really very basic, knowing that would also be most helpful! Also happy to do self-guided reading. I just need to know what this is called!

Improve this question
$\endgroup$
6
  • $\begingroup$ Not sure if I understand correctly but if the study has not been done yet, I'd consider changing the design so that you compare homeless black people to homeless white people and homed black people to homed white people. Now, you are manipulating 2 characteristics at the same time (aka for the same respondents) and therefore cannot say which characteristic relates to the possible attitude differences. $\endgroup$ Commented Jan 29, 2024 at 9:31
  • $\begingroup$ I think it isn't helpful to think of this as a fraction of a larger design. You have two main factors, home or not and race. These are assigned to blocks of size 2 but only two kinds of blocks out of a possible 6. You can estimate the main effects and the interaction between them. The issue (not problem) is that the estimates of the main effects are made using all of the respondents and the variation of the estimates uses all of that variation. The interaction effect is estimated using two different groups of respondents with repeated measures on each one. $\endgroup$ Commented Jan 29, 2024 at 16:17
  • $\begingroup$ Thanks both! @Sointu, at present we are locked into using the same name for each black vignette and each white vignette and using the same vignette for each homed and homeless condition, which is how we can to the present design. $\endgroup$ Commented Jan 30, 2024 at 14:31
  • $\begingroup$ @DavidSmith, I'll abandon the idea of a fractional design... it didn't seem to fit... we'd like to estimate the main effects and interactions between them. What type of model would you advise we fit? I did find one paper with something like this that treated everything as between-subjects (ignoring any within-subject variance). Is that defensible? $\endgroup$ Commented Jan 30, 2024 at 14:34
  • $\begingroup$ Still not sure I understand correctly, but I guess if you're willing to assume that all vignette readers are entirely interchangeable with each other (like they were "assessment machines" or something like that), you can use race and homelessness as between-person predictors of attitudes. $\endgroup$ Commented Jan 31, 2024 at 8:32

1 Answer 1

Reset to default
2
$\begingroup$

This is more of a comment, but doesn't fit in comment space. In my opinion the problem is not the repeated nature of the data. You can run the study according to your design and use one of many analyses that handle the fact that you have 2 observations per participant. You can use multilevel regression with random intercept, RM-ANOVA, regular regression with clustered / sandwiched standard errors, or something else. This is not a problem. However, if your design is such that

Group 1 participants read a vignette about a homeless black person and a vignette about a white person with a home.

Group 2 participants read a vignette about a black person with a home and a vignette about a homeless white person.

Then you can't estimate the effect of race or the effect of homelessness on attitudes, because you have perfectly confounded the two in your design, unless you are willing to consider all participants as completely interchangeable, "assessment machines", as I mentioned.

You can only assess how attitudes towards homeless black people differ from attitudes towards white people with homes, and, separately, how attitudes towards black people with homes differ from attitudes towards homeless white people (actually, my intuition is that you could/maybe even should analyze the groups separately, but I'm not sure).

This is a problem that is separate from having a repeated measures / clustered data. If you want to estimate the "pure" effects of race and homelessness, you have to ignore the within-person nature of the data. But usually, ignoring the within-person element is not good. That's why I was questioning your design. However maybe I still haven't understood the design correctly...

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks, Soitu. I do understand your point. They cannot be compared with simple techniques unless the repeated measures element is discarded. I think I am going to explore a multilevel modelling solution and have posted a new version of my question to that group. A multilevel model would permit a random intercept, but also random slopes for everything except for the interaction. It at least accounts for more of the within-subject variability, but assumes a common, fixed interaction between the two variables across subjects. $\endgroup$ Commented Feb 3, 2024 at 17:23

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.