3
$\begingroup$

The documentation clearly states that for the symmetrical normal distribution RarerProbability[distribution,x] gives the two-sided p-value for x.

Using direct integration on an example:

nd = NormalDistribution[0, 1]; RarerProbability[nd, 1] == 2 Integrate[PDF[nd][x], {x, 1, Infinity}] (*True*)

What is the functionality of RarerProbability when applied to asymmetrical distributions like Weibull or SkewNormal distributions? And are there useful applications?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ This definition of a P-value gives the probability of getting a more extreme value than what is observed based on the estimated value of the pdf at that potential extreme value. So rather than placing equal probabilities in each "tail", one bases the calculation on equal values of the probability density function. It's used more often in Bayesian analyses. The documentation seems to focus on finding outliers and one must remember that if you have too many outliers, you have the wrong data generation model rather than bad data. $\endgroup$ Commented Nov 17 at 19:39
  • $\begingroup$ Thank you @JimB. It took some thinking, but I understand. I confirmed my understanding in a notebook which I will post as an answer. Thanks to you! $\endgroup$ Commented Nov 17 at 23:10

1 Answer 1

Reset to default
3
$\begingroup$

Thanks to the comment by JimB, I have an answer to my question, which I will post here.

(*make an asymmetrical distribution*) sd = SkewNormalDistribution[0, 1, 1]; (*and its PDF*) sdpdf = PDF[sd]; Plot[PDF[sd][x], {x, -3, 3}]

enter image description here

(*calculate tail probabilities using RarerProbability*) RarerProbability[sd, 1] // N (*0.5527262395650943`*) (*calculate right tail probability by integration*) rightTail = NIntegrate[sdpdf[x], {x, 1, Infinity}]; (*calculate probability density at right tail edge*) pdf1 = sdpdf[1] // N; (*find the x value corresponding to the same value*) leftEdge = x /. FindRoot[{sdpdf[x] == pdf1}, {x, -1, 0}]; (*calculate the left tail probability*) leftTail = Integrate[PDF[sd, x], {x, -Infinity, leftEdge}]; (*the sum of the probabilites of the two tails is what RarerProbility reported*) leftTail + rightTail (*0.5527262395649778`*)
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1 Essentially the usual approach to characterize "extremeness" is to "rank" the distribution by the value of the random variable. RarerProbability characterizes extremeness by the density (i.e., the mode - where the largest density is - is the least extreme). $\endgroup$ Commented Nov 17 at 23:33
  • 1
    $\begingroup$ I would like to add that the main purpose of RarerProbability is in the fact that it applies to multi-variate distributions as well, where the notion of a "tail" is much less well-defined. $\endgroup$ Commented Nov 18 at 11:17

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.