Skip to main content
Cross Validated

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

5
  • 11
    $\begingroup$ (+1) An easy way to hack a credible interval is to adopt just the right prior :-). Of course no competent practitioner would do this--Gelman emphasizes using sensitivity assessments, uninformative hyperpriors, etc.--but then again no competent user of hypothesis tests would do p-value hacking, would they? On the other hand, in a Bayesian analysis it might be more difficult to hide what one is doing--assuming the prior is clearly disclosed--compared to all the undocumented analyses that may be involved in p-value hacking. $\endgroup$ Commented Jan 25, 2015 at 21:32
  • 1
    $\begingroup$ @whuber, that's true, but I think we can set aside any issues w/ the inappropriateness or subjectivity of the prior. If the true effect isn't exactly 0, w/ enough data the credible interval will eventually not include 0, just as the p will be <.05 (cf, the last quote), so you can just keep collecting data until you get the result you want irrespective of the prior. $\endgroup$ Commented Jan 25, 2015 at 21:38
  • 4
    $\begingroup$ Good points. I am reminded of a recent question about predicting failures in 10,000 products after observing no failures in 100,000 of them. The answer is pretty sensitive to the prior because failures are so rare. This may be the kind of exceptional situation that "proves the rule"; it shows that in reality it can be impracticable to collect enough data to obtain a desired result. That's exactly when some clients start imploring the statistician to "do their magic" to achieve the desired outcome! Probably many readers have felt that pressure before ... . $\endgroup$ Commented Jan 25, 2015 at 21:43
  • 1
    $\begingroup$ @gung, in practical clinic trials, there are always stopping criteria at different phases for recruiting more subjects for experiments. In that sense, would Bayesian approach sound less likely to manipulate the credible interval thus the research conclusions? $\endgroup$ Commented Jan 25, 2015 at 23:25
  • 2
    $\begingroup$ @AaronZeng, it seems to me that explicit stopping criteria apply equally to Frequentist & Bayesian perspectives. I don't see any net advantage / disadvantage here. $\endgroup$ Commented Jan 25, 2015 at 23:53