2
$\begingroup$

As part of my research I have developed a software and I wanted to test it with 2 other softwares that does similar work. The problem is I could find only few users of these two softwares (20 users for software 1 and 10 users for software 2 and 5 users used both software 1 and software 2). This is what I have done

  1. Defined a 4 questions that covers different aspects of the software. These questions can be answered in a Likert scale of 1-5

  2. These 30 users are asked to answer these questions based on my software and other software (this 30 included users of software 1 or software 2. So I Considered software 1 and 2 as a single set)

  3. Done wilcoxon test to find the statistical significance

I want to know whether the procedure followed by me is good. Or is there any better procedure for my task at hand ? is it ok to consider software 1 and 2 as a single group to compare with my software ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Did the 5 users who are familiar with both software 1 and 2 form part of the 30 total? In other words, is the total number, $n$, of answers for the ratings $30$ or $35$? If there were $35$ binary comparisons, then you could use $35$ as the $n$ for comparison, with the result being a comparison of your software with "other" software, where "other" is weighted by frequency of use of "other."

Edit: OK, please confirm that you asked about each users experience with "any and all other software." In that case the comparison is paired between your software and the sum total experience for each user with other software, and, Wilcoxon signed-rank test is appropriate for $n=30$.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Yes. both software 1 and 2 form part of the 30 total. That is we have $30$ users in total for the study. My question is, is it ok to consider software 1 and 2 as a single group to compare with my software ? Another question is whether my testing procedure is ok ? $\endgroup$ Commented Feb 28, 2018 at 0:52
  • $\begingroup$ From a research perspective is it ok to do a test like that to prove that my software is better than other softwares ? Or i need to do the test separately for each software ? $\endgroup$ Commented Feb 28, 2018 at 4:24
  • $\begingroup$ Bit of an apples and oranges problem. The data you have does not have the same ability to discriminate between better and worse for each software comparison, yours with 1, yours with 2, and yours with 1 & 2. If you had equal numbers in each category you could contemplate running such a comparison with the same power, but, you do not have that. Moreover, each test would in that ideal case still have $\frac{n}{3}$ comparisons, not as powerful as with $n$ comparisons. What you do have is a comparison with software experience of users, not with any particular alternative software. $\endgroup$ Commented Feb 28, 2018 at 4:40
  • $\begingroup$ Ok. I am going with my software vs Other. Which plot is best suited for this ? $\endgroup$ Commented Feb 28, 2018 at 8:11
  • $\begingroup$ Wilcoxon result is a probability does not need plotting. What else is worth plotting depends on things I have not been shown. $\endgroup$ Commented Feb 28, 2018 at 14:04

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.