4
$\begingroup$

I have 7 groups of data. I want to test if there is a significant difference between these groups of data.

However, some of this data is overlapping. ie. some data in Group 1 might be in group 3. Some data in group 6 might also be in group 7 for example. The groups are therefore not perfectly matched.

Secondly, because of this reason, the number of entries in each group are different. For example, group 1 has 54 entries, whereas group 4 has 94 entries.

Is there a particular test I can use to test if there is a significant difference between the groups?

I hope I have made this clear. Many thanks for your help.

Improve this question
$\endgroup$
4
  • $\begingroup$ I'm sorry but I think you have to clarify whether you want to test if there is a pair of groups among which a difference is found, or if there is a difference simultaneously among the groups as a whole. Moreover, what is the nature of the data in each entry? Is is a value? And are you testing if these values are different? $\endgroup$ Commented Jul 5, 2014 at 18:38
  • $\begingroup$ I am not super familiar with Statistics, so I don't know the correct terms for describing the data. But I will try and describe it: I am looking at the page length of letters. The 7 groups are different 'themes'. If a letter includes information on one of the 7 themes, then I include that letters page length under that theme group. For example, one letter is 8 pages, and includes information on Theme 1, 3 and 4. So under 1, 3 and 4 I would have '8' as a data entry. Therefore the sample is not perfectly matched. I hope that is kind of clear. $\endgroup$ Commented Jul 6, 2014 at 21:45
  • $\begingroup$ So I would suppose you want to check if the average length page between themes is different. Is that it? $\endgroup$ Commented Jul 6, 2014 at 22:01
  • $\begingroup$ I suppose that is it. Is there a particular test I can use to achieve this considering the nature of the data? (ie. Uneven sample size, and overlapping data) $\endgroup$ Commented Jul 7, 2014 at 11:34

1 Answer 1

Reset to default
1
$\begingroup$

I am not sure the following test and procedure are the best for overlapping data, but let us come back to it in the end.

The first thing to do is to chose a test to compare the mean of two groups, and a natural choice would be the Students's t test. A more adequate approach to your specific problem might be the Hotelling's T-square distribution, but I do not know much about it so I'll leave it to those more acquainted with it than me.

If you choose to perform several t-tests (one for each pair of samples), or any kind of test that deliver a p-value, you will have to correct for the fact that you are doing multiple testing. What you'll have to correct is the level at which you will reject the null hypothesis, and how to correct for it is explained in this answer.

At this point is important to note that if you choose to walk down this path then you are formally correcting for multiple independent tests. Your tests are not independent, because the data overlap, so if you choose m as the number of tests you are being too conservative. It would be preferable to estimate the number of independent tests from your data m' and correct only for it (but it might not be obvious at all).

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the detailed reply. I followed your advice and did multiple t-tests for each pairing. This gave me 21 different results (this is what you meant right? 7 different groups = 21 pairings?) I also tried to make use of "the Bonferroni correction". As I understand it, this will always make the p-value higher. In all of the 21 t-tests, I wasn't able to reject the Null hypothesis even without making "the Bonferroni correction". Does this mean the problem with correcting for independent tests is not important, as it is clear that I cant reject H0 anyway? Thanks "the Bonferroni correction $\endgroup$ Commented Jul 7, 2014 at 21:10
  • $\begingroup$ One other thing: I did find this link which talks about t-tests for overlapping samples. But I have no idea how credible (if at all) it is. nces.ed.gov/nationsreportcard/tdw/print_page.aspx Is it of any value? Thanks $\endgroup$ Commented Jul 7, 2014 at 21:34
  • $\begingroup$ Your interpretation is correct. If you cannot reject the null hypothesis in the individual tests from the beginning, and since the correction only increases the p-value, you will not be able to reject the null hypothesis after correction. As for your link I cannot load it; I am not an expect on this subject but I upvoted the question to draw other's attention. $\endgroup$ Commented Jul 7, 2014 at 23:06

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.