2
$\begingroup$

I was trying to understand the difference between the concepts of probability distribution and probability space for what concerns the assignment of probability over the sample space. Wikipedia gives the following definition for probability distribution:

A probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events (subsets of the sample space).

Which actually is the same definition which it gives also for a probability space, even if for this last one more formally introduces the concepts of a sample space $Ω$, an event space F (a σ-algebra) and a probability function P giving measures of probability for the events. Also, I would like to point out that a probability function is indeed one of the possible ways to uniquely define a probability distribution.

I would like to know if there is some difference between the two terms, if the difference is more of a contextual convention nuance, or if this is the actual formality and indeed which in case is the actual difference.

$\endgroup$
3
  • 1
    $\begingroup$ To begin with, Wikipedia's characterization -- it's not a definition -- is limited to probability spaces with only countable numbers of events of nonzero probability: essentially, discrete spaces. It would be far better to characterize a distribution as a function assigning probabilities to events rather than outcomes. Thus, your question really seems to be a matter of distinguishing colloquial descriptions from rigorous definitions. $\endgroup$ Commented Nov 28, 2023 at 17:14
  • $\begingroup$ @whuber why you say it's limited to discrete variable spaces? It seems to me that is more of your assumption than a limitation of Wikipedia's reported characterization (however you want to call it, I would say definition as it defines it, but in case tell why). What you have reported subsequently it's more the definition of a probability function than of a probability distribution, which can be described, by the way between the others, from a probability function. Also, in case, are you stating that they are the same for what concerns probability assignment over the sample space? $\endgroup$ Commented Nov 29, 2023 at 18:28
  • 1
    $\begingroup$ I assume nothing: I am applying a standard definition. When a probability measure is defined on all individual outcomes, that immediately implies the discrete sigma algebra is in use. Please consult any advanced textbook on probability theory. The reason why the Wikipedia quotation is not a definition is that it uses mathematically meaningless qualitative words to explain what kind of function a probability is, without supplying any criteria one can actually use to distinguish probability functions from any other kind of function. $\endgroup$ Commented Nov 29, 2023 at 19:19

1 Answer 1

1
$\begingroup$

These two concepts are so strikingly different that it is really difficult for me to understand how they can be confused. The confusion may stem from the ambiguous colloquial description you quoted (link?) which can be easily eliminated by presenting them with rigorous mathematical notations.

Short Answer: A probability space is a triple $(\Omega, \mathscr{F}, P)$, while a probability distribution is a (set) function. So they are very different.

Longer Answer: A probability space is a triple $(\Omega, \mathscr{F}, P)$ that contains a sample space $\Omega$, a $\sigma$-field $\mathscr{F}$, and a probability measure $P$, while a probability distribution (of a random variable $X$ that is a measurable function from $\Omega$ to $\mathbb{R}$) is a probability measure (i.e., a set function) defined on another measurable space $(\mathbb{R}^1, \mathscr{R}^1)$ where the random variable $X$ takes value on.

More Elaborations: As I made it clear above, to make sense of the term "probability distribution" (let's denoted it by $\mu$), a random variable $X$ must be defined in advance on the probability space $(\Omega, \mathscr{F}, P)$. In particular, $\mu$ is different from the original probability measure $P$, but it is a probability measure induced by $P$ (sometimes called "push-forward measure") as follows: \begin{align*} \mu(A) = P(X^{-1}(A)) = P(\{\omega \in \Omega: X(\omega) \in A\}), \quad A \in \mathscr{R}^1. \end{align*} Here $\mathscr{R}^1$ stands for the Borel field generated by open sets in $\mathbb{R}^1$. For this reason, $\mu$ is usually denoted as $P\circ X^{-1}$ or $P_X$. By any means, $\mu$ is a set function, which is obviously a different object compared to the probability space.

For completeness, I am quoting the formal definitions of these two concepts from an authoritative mathematics textbook (Probability and Measure (3rd edition) by Patrick Billingsley):

(Probability space) If $\mathscr{F}$ is a $\sigma$-field in $\Omega$ and $P$ is a probability measure on $\mathscr{F}$, the triple $(\Omega, \mathscr{F}, P)$ is called a probability measure space, or simply a probability space.

(Probability distribution) The distribution or law of the random variable is the probability measure $\mu$ on $(\mathbb{R}^1, \mathscr{R}^1)$ defined by \begin{align*} \mu(A) = P[X \in A], \quad A \in \mathscr{R}^1. \end{align*}

Comments on your Quotation:

A probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment.

In my opinion, this sounds more like a description of $P$ in the original probability space, instead of $\mu$ on $(\mathbb{R}^1, \mathscr{R}^1)$.

It is a mathematical description of a random phenomenon in terms of its sample space and the probabilities of events (subsets of the sample space).

Again, this "It" is better (if I have to) interpreted as $P$ -- accordingly the sample space and subsets of the sample space in this quotation correspond to $\Omega$ and $\mathscr{F}$ respectively.

In summary, I think the author of your quotation had confused "$P$" and "$\mu$" -- when he/she drafted this paragraph, he/she probably had $P$ in his/her mind and tried to elaborate it in words, for the random variable $X$, which is a prerequisite to define the term distribution, looks completely slipped away.

$\endgroup$
8
  • $\begingroup$ Re "very different:" not really, once you remember that part of a function is its domain; that its domain is a sigma algebra; and that algebra determines its underlying space (which is the union of the elements of the algebra). In the Wikipedia characterization the algebra is generated by all singletons ("outcomes"). In the special circumstances envisioned in Wikipedia, from a nonzero function $f:X\to\mathbb R$ we recover $\mathfrak F=\mathcal P(X)$ (the power set), $\Omega=\bigcup_{A\in\mathfrak F}A=X,$ and $\Pr:\mathfrak F\to\mathbb R$ via $\Pr(A)=\sum_{\omega\in A}f(\omega).$ $\endgroup$ Commented Nov 28, 2023 at 19:51
  • $\begingroup$ I am assuming you are trying to argue that $P$ and $(\Omega, \mathscr{F}, P)$ are not "very different" (which is slightly different from OP's original question unless he interprets "probability distribution" as "probability measure"). I think that depends on how "tolerant" you are in viewing two mathematical objects are the same or different. For me, $(\Omega, \mathscr{F}, P)$ and $P$ are still "very different" in spite of their connections as you pointed out, because the latter is only a part/a component of the former. $\endgroup$ Commented Nov 28, 2023 at 20:08
  • $\begingroup$ My point is that the definition of $P$ includes its domain $\mathfrak F,$ so the only distinction you are making is to insist that the ordered pair $(\Omega, \mathfrak F)$ is not the same as $\mathfrak F.$ Agreed, but I find the distinction to be of little interest, because it's a mere notational matter having no underlying substance. $\endgroup$ Commented Nov 28, 2023 at 20:51
  • $\begingroup$ @whuber To be honest, your perspective is really new to me and I have to say I couldn't accept that. And I don't see any benefits of taking this view to intentionally blur the boundary of $(\Omega, \mathscr{F}, P)$ and $P: \mathscr{F} \to [0, 1]$. Also, the main point of my answer is not trying to compare $(\Omega, \mathscr{F}, P)$ with $P$, but $(\Omega, \mathscr{F}, P)$ with $\mu$. $\endgroup$ Commented Nov 28, 2023 at 21:07
  • $\begingroup$ @Zhanxiong probably you are too smart to even be on the platform if you cannot understand how they can be confused. But not enough to 1)understand that it may happen that most people don't grasp concepts that are easy to others; 2) to carefully read a question before answering. The key part in the question was 'for what concerns the assignment of probability'. Also, as said, the quoted text is from Wikipedia for probability distribution, so you can access it just by reading the 'Probability distribution' on Wikipedia. $\endgroup$ Commented Nov 29, 2023 at 18:48

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.