Expectation

Expectation#

Definition 82 (Expectation)

Let $P$ be a probability function defined over the probability space $(Ω, F, P)$ .

Let $X$ be a discrete random variable with $Ω = {ξ_{1}, ξ_{2}, \dots}$ .

Then the expectation of $X$ is defined as:

E (X) = \sum_{x \in X (Ω)} x \cdot P [X = x]

Theorem 20 (Existence of Expectation)

Let $P$ be a probability function defined over the probability space $(Ω, F, P)$ .

A discrete random variable $X$ with $Ω = {ξ_{1}, ξ_{2}, \dots}$ has an expectation if and only if it is absolutely summable.

That is,

E [| X |] \overset{def}{=} \sum_{x \in X (Ω)} | x | \cdot P [X = x] < \infty

Let $P$ be a probability function defined over the probability space $(Ω, F, P)$ .

Let $X$ be a discrete random variable with $Ω = {ξ_{1}, ξ_{2}, \dots}$ .

Then the expectation of $X$ has the following properties:

Property 1 (The Law of The Unconscious Statistician)

For any function $g$ ,

E [g (X)] = \sum_{x \in X (Ω)} g (x) \cdot P [X = x]

This is not a trivial result, proof can be found here.

Property 2 (Linearity)

For any constants $a$ and $b$ ,

E [a X + b] = a \cdot E (X) + b

Property 3 (Scaling)

For any constant $c$ ,

E [c X] = c \cdot E (X)

Property 4 (DC Shift)

For any constant $c$ ,

E [X + c] = E (X)

Property 5 (Stronger Linearity)

It follows that for any random variables $X_{1}$ , $X_{2}$ , …, $X_{n}$ ,

E [\sum_{i = 1}^{n} a_{i} X_{i}] = \sum_{i = 1}^{n} a_{i} \cdot E [X_{i}]

Concept

Expectation is a measure of the mean value of a random variable and is deterministic. It is also synonymous with the population mean.
Average is a measure of the average value of a random sample from the true population and is random.
Average of a random sample is a random variable and as sample size increases, the average of a random sample converges to the population mean.

Pishro-Nik, Hossein. “Chapter 3.2.3. Functions of Random Variables.” In Introduction to Probability, Statistics, and Random Processes, 199–201. Kappa Research, 2014.
Chan, Stanley H. “Chapter 3.4. Expectation.” In Introduction to Probability for Data Science, 125-133. Ann Arbor, Michigan: Michigan Publishing Services, 2021.