Probability Mass Function

Definition

Definition 79 (State Space)

The set of all possible states of \(X\) is called the state space of \(X\) and is denoted as \(X(\S)\).

In particular, the state space of a discrete random variable \(X\) is a countable set as per Definition 78.

Example 25 (State Space of Coin Toss)

Let us revisit the example in Example 22 and examine the state space of \(X\).

The state space of \(X\) is the set of all possible values that \(X\) can take. As enumerated in the example, we see that the state space of \(X\) is \(\{0, 1, 2\}\) (i.e. \(X\) takes 3 states 0, 1 and 2).

Example 26 (State Space of Dice Roll)

Let us revisit the example in Example 23 and examine the state space of \(X\).

The state space of \(X\) is the set of all possible values that \(X\) can take. As enumerated in the example, we see that the state space of \(X\) is \(\{1, 2, 3, 4, 5, 6\}\) (i.e. \(X\) takes 6 states 1, 2, 3, 4, 5 and 6).

For two dice rolls, the state space is \(\{(1, 1), (1, 2), \ldots, (6, 6)\}\) where each state is a tuple of two dice rolls.

Definition 80 (Probability Mass Function)

The probability mass function (PMF) of a random variable \(X\) is a function that maps each state \(x\) in the state space \(X(\S)\) to its probability \(\pmf(x) = \P\lsq X = x \rsq\).

We denoted the PMF as

\[\begin{split} \begin{align} \pmf: X(\S) &\to [0, 1] \\ X(\S) \ni x &\mapsto \pmf(x) \end{align} \end{split}\]

Example 27 (PMF of Coin Toss)

Let us revisit Example 22 on coin toss and compute the PMF of \(X\).

Recall the sample space is given by \(\S = \{(HH), (HT), (TH), (TT)\}\) and the state space is given by \(X(\S) = \{0, 1, 2\}\) as enumerated in Example 25.

Thus, our domain of \(\pmf\) is \(X(\S) = \{0, 1, 2\}\) we have 3 mappings to compute:

\[\begin{split} \begin{align} \pmf(0) &= \P\lsq X = 0 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 0 \rset \rsq = \P\lsq \{(TT)\} \rsq = \dfrac{1}{4} \\ \pmf(1) &= \P\lsq X = 1 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 1 \rset \rsq = \P\lsq \{(HT), (TH)\} \rsq = \dfrac{2}{4} \\ \pmf(2) &= \P\lsq X = 2 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 2 \rset \rsq = \P\lsq \{(HH)\} \rsq = \dfrac{1}{4} \end{align} \end{split}\]

Here we have enumerated all the possible states of \(X\) and computed the probability of each state. Thus, the PMF of \(X\) is completely determined by the 3 mappings above.

Example 28 (PMF of Two Dice Rolls)

Let us revisit Example 23 on two dice rolls and compute the PMF of \(X\).

Recall the sample space is given by \(\S = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) and the state space is given by \(X(\S) = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) as enumerated in Example 26.

Thus, our domain of \(\pmf\) is \(X(\S) = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) we have 36 states to compute:

\[\begin{split} \begin{align} \pmf((1, 1)) &= \P\lsq X = (1, 1) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (1, 1) \rset \rsq = \P\lsq \{(1, 1)\} \rsq = \dfrac{1}{36} \\ \pmf((1, 2)) &= \P\lsq X = (1, 2) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (1, 2) \rset \rsq = \P\lsq \{(1, 2)\} \rsq = \dfrac{1}{36} \\ \vdots \\ \pmf((6, 6)) &= \P\lsq X = (6, 6) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (6, 6) \rset \rsq = \P\lsq \{(6, 6)\} \rsq = \dfrac{1}{36} \end{align} \end{split}\]

Normalization

Theorem 17 (Normalization Property of PMF)

A PMF should satisfy the following normalization property:

(175)\[ \sum_{x \in X(\S)} \pmf(x) = 1 \]

Proof. TODO

Sturges’ Rule and Cross Validation

See Introduction to Probability for Data Science section 3.2.5.