Questions tagged [statistical-significance]
"Statistical significance" is a term used in classical hypothesis testing. We say that the evidence in the test is "statistical significant" if it is sufficiently in favour of the alternative hypothesis that we reject the null hypothesis in the test (at a given significance level).
6,531 questions
6 votes
2 answers
833 views
Why does the p-value increase when I add more observations to my t-test?
I am comparing the means of two groups using an independent two-sample t-test in R. Initially, I had the following samples: Group A: n = 15, mean = 52.3, sd = 4.8 Group B: n = 15, mean = 48.1, sd = 5....
- 86
2 votes
1 answer
52 views
How to design a minimal test (sample size + combinations) when two components both affect accuracy?
I’m trying to design an experiment for a system where two different components both contribute to the final accuracy, and I can only test a small subset of all possible combinations. A concrete ...
4 votes
4 answers
331 views
Borderline interaction p value
I’m working on a logistic regression model where I want to examine whether the effect of one continuous predictor (X1) on a binary outcome depends on another ...
1 vote
0 answers
12 views
Significance testing between smooths in GAM (bs = 'fs') using emmeans [closed]
I have a GAM model built as: ...
0 votes
0 answers
17 views
Variable with complete correlation giving large standard error binary data GLM [duplicate]
I am trying out some analysis on some self-generated artificial data in R about modelling passing and failing based on being assigned two "labels" A-G (see code below) and I am running into ...
- 125
1 vote
0 answers
52 views
How do you study between-groups and within-groups effects together?
I have the following experiment design: Treatment variables A: ['x', 'y', 'z'], between groups B: ['low', 'high'], within groups C: ['low', 'original', 'high'], between groups Setup: I conducted ...
- 111
1 vote
0 answers
90 views
Comparing significant differences between GAM predictions along a continuous dimension
I am working with a GAM model to study how land cover (LC_L1_factor) affects bulk density (BD) with depth (DEPTH_M): ...
2 votes
3 answers
280 views
Non-significant result for Chi Square test followed by a significant post-hoc test
I have some data - I have 3 groups of participants (33 people in group 1, 26 in group 2, and 22 in group 3), and I want to compare the proportion of married vs. non-married people in each of the ...
5 votes
3 answers
327 views
Analyzing test result using test statistic vs p-value?
I am trying to understand how/when to use a test-statistic and/or p-value to draw a conclusion on a test. As far as I understand, when running a statistical test we must decide whether to use the test ...
- 75
2 votes
1 answer
67 views
standardized indirect effects on SEM [closed]
I have 4 exogenous variables, 3 mediators, and 3 endogenous variables. AMOS has calculated direct and indirect effects, and I have relied on standardized indirect effects to report in my dissertation. ...
- 21
0 votes
0 answers
53 views
Testing if aggregated statistics have different distribution for each group in time series data
This post is continuation of these (Test for significant difference between the groups of the time series based on the aggregated statistics), but with more detailed explanation (a lot of changes were ...
2 votes
1 answer
100 views
Test for significant difference between the groups of the time series based on the aggregated statistics
Imagine I have a time series measurement of the 1 tool. Each measurement is labeled by the ordinal category (3 categories) and measurements are non-overlapping. What I want to do is to test a ...
12 votes
3 answers
525 views
Sign test: p and CI contradiction (significant but CI includes 0 AND not significant but CI doesn't include 0)
I run a sign test in R and get repeatedly contradicting p-values and confidence interval (CI). See for example these results of BSDA::SIGN.test() with two dummy ...
- 566
5 votes
2 answers
439 views
Exact Wilcoxon signed rank test: p >.05 but CI (95%) does not include 0
I run an Exact Wilcoxon signed rank test using exactRankTests::wilcox.exact() in R. The test is not significant (p>.05) but the 95% CI does not include zero. How ...
- 566
1 vote
0 answers
81 views
Minimal conditions to define rejection regions in hypothesis testing
I have a conceptual question about hypothesis testing. I would like to know what are the minimal conditions that a statistic and its rejection regions must satisfy in order for a test of a null ...
- 210