I have three groups. Both have small sample sizes (<10). And we can't check the assumption of normality. I want to compare any two of them.
What to do in this case( unequal small sizes, non-normality)
6
- $\begingroup$ How about a permutation test? You may not have much here with such small samples, but on the plus side you can do full enumeration. $\endgroup$soakley– soakley2013-12-07 23:27:34 +00:00Commented Dec 7, 2013 at 23:27
- $\begingroup$ I don't understand. How to carry out a permutation test ？ What's its assumptions? $\endgroup$OMG– OMG2013-12-07 23:32:33 +00:00Commented Dec 7, 2013 at 23:32
- $\begingroup$ So much I want to say about this, but first some questions -- when you say 'I want to compare them' - what is it you're trying to compare? Do you mean to compare locations? Is there a specific alternative? Are you talking about 3 pairwise comparisons or are you after something notionally more like a one-way ANOVA? (The suitability of permutation tests will be affected by how small the sample sizes are -- if they're small enough, you won't be able to get below some specific significance levels; on the other hand, outside specific parametric assumptions you may not have much choice about that) $\endgroup$Glen_b– Glen_b2013-12-07 23:55:52 +00:00Commented Dec 7, 2013 at 23:55
- $\begingroup$ A good description of a permutation test will not fit in a comment. But check Wikipedia under "Resampling" for a decent start. $\endgroup$soakley– soakley2013-12-08 00:13:14 +00:00Commented Dec 8, 2013 at 0:13
- 1$\begingroup$ So does that mean you want to change your question to 'how do I do multiple comparisons after an ANOVA?' or do you actually want to know something like 'how do I do something like an ANOVA and multiple comparisons but without assuming normality?' ...? (Incidentally, what are your sample sizes?) $\endgroup$Glen_b– Glen_b2013-12-08 01:13:45 +00:00Commented Dec 8, 2013 at 1:13
| Show 1 more comment
2 Answers
$\begingroup$ $\endgroup$
0 (This is too long for a comment; if the question is sufficiently clarified - see the comments - it may be edited into a better answer.)
On permutation tests:
The basic idea of a permutation test is that if the null hypothesis is true, the group/category labels are arbitrary and could be reassigned without changing the distribution of the groups. So they are reassigned, in every fashion and a test statistic sensitive to the alternative is calculated; if the null is true, the sample should be a randomly chosen member of that permutation distribution. If the alternative is true, the sample should tend toward being 'extreme' in that distribution (extreme in a sense defined by the ordering resulting from what is most consistent with the alternative).
There's some discussion of permutation tests here, discussion of why they might be used here, discussion of the relationship between permutation tests and randomization tests here, and a little on the choice between permutation tests and t-tests here.
There are examples of different permutation tests done here and here and here and here (see near the end).
$\begingroup$ $\endgroup$
2 No matter what you use your power to detect differences will be low. Another possible approach would be using non-parametric tests. If you have three groups you could use Kruskal-Wallis (followed by a multiple comparison procedure if you want to compare them pairwise).
In R you can use kruskal.test for the Kruskall Wallis test and kruskalmc in the pgirmess package for multiple comparisons
- $\begingroup$
No matter what you use your power to detect differences will be lowHow to understand this? $\endgroup$OMG– OMG2013-12-11 20:01:38 +00:00Commented Dec 11, 2013 at 20:01 - $\begingroup$ OMG: With sample size so small, the power will be low. $\endgroup$Glen_b– Glen_b2013-12-11 21:45:05 +00:00Commented Dec 11, 2013 at 21:45