Questions tagged [reliability]
A measure is said to have a high reliability if it produces similar results under consistent conditions. DO NOT confuse reliability with validity (see tag wiki). DO NOT use for inter-rater reliability which has its own tag inter-rater
582 questions
0 votes
0 answers
17 views
Example of Distribution which is a Base Distribution for the Proportional Hazard family as well as Reversed Hazard rate family of Distribution
Let $X$ and $Y$ be two random variables one belongs to proportional hazard rate family and another is proportional reverse hazard rate family of distribution. Now my concern is whether there exists ...
0 votes
1 answer
43 views
How to choose between subscale scores and a combined score when reliability and inter-correlations vary across time points?
I am using a part of a questionnaire that could be scored in two ways: a Total Score (all items) and two Subscale Scores (each based on a subset of items). Internal consistency (McDonald’s ω) and the ...
1 vote
1 answer
46 views
Unexpected behavior of reverse-coded item: positive correlation and reliability issues
I encountered issues with reverse-coded items in two different Likert-type questionnaires. In the first questionnaire, a theoretically reverse-scored item initially showed positive correlations with ...
2 votes
2 answers
104 views
Can I use unstandardised factor loadings to compute communality?
For a larger problem I am trying to solve, I have been reading about communality/item-level reliability for a single-factor model. From what I have read, I see that the formula for communality ($c$) ...
9 votes
3 answers
391 views
What makes a metric representative and reliable when comparing different processes?
I’d like to ask a general question about metrics. Recently, while working as an intern, I was asked to evaluate the success of different marketing journeys (in a CRM/Marketing Cloud context). The data ...
0 votes
1 answer
62 views
Spearman-Brown Index Cut-Off
For a project, we developed a two-item scale that we averaged after calculating the Spearman-Brown coefficient to assess its reliability. I am wondering what would be an appropriate cutoff for ...
1 vote
0 answers
42 views
Understanding terms of a composite reliability equation
I'm trying to compute composite reliability using the equation below (based on Wang & Stanley, 1970, and described here). $$ r_c = \frac{\sum_{i=1}^{k} r_i + \sum_{i=1}^{k} \sum_{j (\ne i)=1}^{k} ...
1 vote
0 answers
51 views
Repeated measures, reliability and validity tests
I conducted a pre-test consisting of an experiment in which a sample of 50 participants was exposed to 16 images. While viewing each image, participants responded to a Likert-type fear scale composed ...
0 votes
0 answers
129 views
DGP for X_t and Y_t with specified correlation over time?
Edited for clarity. I am trying to set up a DGP for two vectors of normally distributed random variables $\mathbf{X}=(X_1, ..., X_{50})$ and $\mathbf{Y}=(Y_1, ..., Y_{50})$, with the following two ...
2 votes
1 answer
76 views
Which one is more reliable between two items given their average rating and number of reviews?
It is a situation when we browse something sorted by rating. The sorting shows list of items according to their rating start from highest rating to lowest. We, the searcher, suppose to pick the ...
3 votes
1 answer
106 views
Measuring a test's inter-rater reliability: split-half or Krippendorff?
A few hundred people took a test made up of 20 questions. The questions were randomly chosen for each participant from a pool of around 1,000. Each question was scored as either 1 for a correct answer ...
1 vote
0 answers
75 views
Which factor score estimation method for Rasch/2PL itemfit and reliability?
I want to calculate the item-fit values (infit and outfit) and the person separation reliability (PSR) for a Rasch and 2PL model using R. My goal is developing a reliable and good fitting scale that ...
1 vote
0 answers
49 views
How to write the renewal equation in Python? [closed]
I have been reading about the renewal equation for a research project. I was amazed that despite plenty of research, the renewal equation does not seem to be implemented in any Python library (or is ...
3 votes
3 answers
129 views
Poor reliability for difference scores in behavioural tasks: implications for ANCOVA, LMM where the raw RT data is used in analysis
I have measured RT in two conditions: condition A and condition B and I am interested in how RT in these conditions interacts with anxiety score. As is common, the split-half reliability is excellent ...
0 votes
1 answer
90 views
How to compare two components' reliability?
I have two different components in service, an old one and a new one. I want to know if the new component is more reliable than the old one (ie: higher MTBF) Group1 Old component: contains 80% of the ...