Probability Mass Function

Definition

Definition 76 (State Space)

The set of all possible states of \(X\) is called the state space of \(X\) and is denoted as \(X(\S)\).

In particular, the state space of a discrete random variable \(X\) is a countable set as per Definition 75.

Example 22 (State Space of Coin Toss)

Let us revisit the example in Example 19 and examine the state space of \(X\).

The state space of \(X\) is the set of all possible values that \(X\) can take. As enumerated in the example, we see that the state space of \(X\) is \(\{0, 1, 2\}\) (i.e. \(X\) takes 3 states 0, 1 and 2).

Example 23 (State Space of Dice Roll)

Let us revisit the example in Example 20 and examine the state space of \(X\).

The state space of \(X\) is the set of all possible values that \(X\) can take. As enumerated in the example, we see that the state space of \(X\) is \(\{1, 2, 3, 4, 5, 6\}\) (i.e. \(X\) takes 6 states 1, 2, 3, 4, 5 and 6).

For two dice rolls, the state space is \(\{(1, 1), (1, 2), \ldots, (6, 6)\}\) where each state is a tuple of two dice rolls.

Definition 77 (Probability Mass Function)

The probability mass function (PMF) of a random variable \(X\) is a function that maps each state \(x\) in the state space \(X(\S)\) to its probability \(\pmf(x) = \P\lsq X = x \rsq\).

We denoted the PMF as

\[\begin{split} \begin{align} \pmf: X(\S) &\to [0, 1] \\ X(\S) \ni x &\mapsto \pmf(x) \end{align} \end{split}\]

Example 24 (PMF of Coin Toss)

Let us revisit Example 19 on coin toss and compute the PMF of \(X\).

Recall the sample space is given by \(\S = \{(HH), (HT), (TH), (TT)\}\) and the state space is given by \(X(\S) = \{0, 1, 2\}\) as enumerated in Example 22.

Thus, our domain of \(\pmf\) is \(X(\S) = \{0, 1, 2\}\) we have 3 mappings to compute:

\[\begin{split} \begin{align} \pmf(0) &= \P\lsq X = 0 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 0 \rset \rsq = \P\lsq \{(TT)\} \rsq = \dfrac{1}{4} \\ \pmf(1) &= \P\lsq X = 1 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 1 \rset \rsq = \P\lsq \{(HT), (TH)\} \rsq = \dfrac{2}{4} \\ \pmf(2) &= \P\lsq X = 2 \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = 2 \rset \rsq = \P\lsq \{(HH)\} \rsq = \dfrac{1}{4} \end{align} \end{split}\]

Here we have enumerated all the possible states of \(X\) and computed the probability of each state. Thus, the PMF of \(X\) is completely determined by the 3 mappings above.

Example 25 (PMF of Two Dice Rolls)

Let us revisit Example 20 on two dice rolls and compute the PMF of \(X\).

Recall the sample space is given by \(\S = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) and the state space is given by \(X(\S) = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) as enumerated in Example 23.

Thus, our domain of \(\pmf\) is \(X(\S) = \lset (1, 1), (1, 2), \ldots, (6, 6) \rset\) we have 36 states to compute:

\[\begin{split} \begin{align} \pmf((1, 1)) &= \P\lsq X = (1, 1) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (1, 1) \rset \rsq = \P\lsq \{(1, 1)\} \rsq = \dfrac{1}{36} \\ \pmf((1, 2)) &= \P\lsq X = (1, 2) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (1, 2) \rset \rsq = \P\lsq \{(1, 2)\} \rsq = \dfrac{1}{36} \\ \vdots \\ \pmf((6, 6)) &= \P\lsq X = (6, 6) \rsq = \P\lsq \lset \xi \in \S \st X(\xi) = (6, 6) \rset \rsq = \P\lsq \{(6, 6)\} \rsq = \dfrac{1}{36} \end{align} \end{split}\]

Normalization

Theorem 17 (Normalization Property of PMF)

A PMF should satisfy the following normalization property:

(129)\[ \sum_{x \in X(\S)} \pmf(x) = 1 \]

Proof. TODO

Sturges’ Rule and Cross Validation

See Introduction to Probability for Data Science section 3.2.5.