Notes on Probability Essentials - 1 - Axioms of Probability

Probability

“他说 / 你任何为人称道的美丽 / 不及他第一次遇见你。”
——《南山南》

本文用尽可能少的内容提供了建立概率测度(probability measure)所需要的公理和定义。

Abstract：这里想谈一下直觉性的理解，为什么概率测度要与$\sigma$-algebra扯上关系呢？就概率事件而言，空集和全集肯定都得在$\mathcal{A}$里头，这刚好是下文的性质1；任何一个事件必然有它的反面存在，这就是性质2——任何一个事件的补集也必须在$\mathcal{A}$里头；性质4也是满足了概率的研究条件——因为概率测度(probability measure)的建立需要countable additivity, 那么显然这性质4（countable union and intersection）就是需要有的。所以$\sigma$-algebra就像是为概率测度量身打造的一种代数结构。
这一节介绍了$\sigma$-algebra,定义在$\mathbb{R}$上的Borel $\sigma$-algebra, probability measure，以证明完概率测度$P$具有连续性结束。

Let $\Omega$ be an abstract space. Let $2^\Omega$ denote all subsets of $\Omega$, including the empty set denoted by $\emptyset$. With $\mathcal{A}$ being a subset of $2^\Omega$, we consider the following properties:

1. $\emptyset\in\mathcal{A}$ and $\Omega\in\mathcal{A}$;
2. If $A\in\mathcal{A}$ then $A^c\in\mathcal{A}$, where $A^c$ denotes the complement of A;
3. $\mathcal{A}$ is closed under finite unions and finite intersections: that is, if $A_1,...,A_n$ are all in $\mathcal{A}$, then $\bigcup_{i=1}^n{A_i}$ and $\bigcap_{i=1}^n{A_i}$ are in $\mathcal{A}$ as well;
4. $\mathcal{A}$ is closed under countable unions and intersections: that is, if $A_1,A_2,A_3,...$ is a countable sequence of events in $\mathcal{A}$, then $\bigcup_{i=1}^n{A_i}$ and $\bigcap_{i=1}^n{A_i}$ are both also in $\mathcal{A}$.

Definition 1 $\mathcal{A}$ is an algebra if it satisfies (1), (2) and (3) above. It is a $\sigma$-algebra (or a $\sigma$-field) if it satisfies (1),(2), and (4) above.
Note: (1)+(4) implies (3), hence any $\sigma$-algebra is an algebra (Exercise: Suppose that $\Omega$ is an infinite set, countable or not, and let $\mathcal{A}$ be the family of all subsets which are either finite or have a finite complement. Show that $\mathcal{A}$ is an algebra, but not a $\sigma$-algebra, 点击这里看解答).

Definition 2 If $\mathcal{C}\subset 2^\Omega$, the $\sigma$-algebra generated by $\mathcal{C}$, and written $\sigma(\mathcal{C})$, is the smallest $\sigma$-algebra containing $\mathcal{C}$.

Example:
1. $\mathcal{A}=\{\emptyset,\Omega\}$ (the trivial $\sigma-algebra$)
2. $A\subset\Omega$, then $\sigma(A)=\{\emptyset,A,A^c,\Omega\}$
3. If $\Omega=\mathbb{R}$, the Borel $\sigma-algebra$ is the $\sigma-algebra$ generated by the open sets.

Theorem 1 The Borel $\sigma$-algebra of $\mathbb{R}$ is generated by intervals of the form $(-\infty,a]$, where $a\in\mathbb{Q}$.

Proof: Let $\mathcal{C}$ denote all open intervals. Since every open set in $\mathbb{R}$ is the countable union of open intervals, we have $\sigma(\mathcal{C})=$ the Borel $\sigma$-algebra of $\mathbb{R}$.
Let $\mathcal{D}$ denote all intervals of the form $(-\infty,a]$, where $a\in\mathbb{Q}$. Let $(a,b)\in\mathcal{C}$, and let $(a_n)_{n\ge 1}$ be a sequence of rationals decreasing to $a$ and $(b_n)_{n\ge 1}$ be a sequence of rationals increasing strictly to $b$. Then

$$(a,b) = \bigcup_{n=1}^\infty(a_n,b_n] \\ = \bigcup_{n=1}^\infty((-\infty,b_n]\cap(-\infty,a_n]^c)$$
Therefore $\mathcal{C}\subset\sigma(\mathcal{D})$, where $\sigma(\mathcal{C})\subset\sigma(\mathcal{D})$. However since each element of $\mathcal{D}$ is a closed set, it is also a Borel set, and therefore $\sigma(\mathcal{D})$ is contained in the Borel sets $\mathcal{B}$. Thus we have
$$\mathcal{B}=\sigma(\mathcal{C})\subset\sigma(\mathcal{D})\subset\mathcal{B}$$
and hence $\sigma(D)=\mathcal{B}$.

Definition 3 A probability measure defined on a $\sigma$-algebra $\mathcal{A}$ of $\Omega$ is a function $P:\mathcal{A}\rightarrow[0,1]$ that satisfies:
1. $P(\Omega)=1$
2. For every countable sequence $(A_n)_{n\ge 1}$ of elements of $\mathcal{A}$, pairwise disjoint(that is, $A_n\cap A_m=\emptyset$ whenever $n\ne m$), one has

Note: Axiom (2) is called countable additivity, the number $P(A)$ is called the probability of the event $A$.

Theorem 2 If $P$ is a probability measure on $(\Omega,\mathcal{A})$, then:
1. $P(\emptyset)=0$
2. P is finite additive.

Note: 第二点finite additivity可以直接Definition 3(2)的countable additivity获得。需要注意finite additivity并不是countable additivity的充分条件。尽管finite additivity从直觉上很容易理解，但是它能做的事情实在是太少了（比如它无法处理极限情况），下面这个定理将揭示countable additivity的作用。

Theorem 3 Let $\mathcal{A}$ be a $\sigma$-algebra. Suppose that $P:\mathcal{A}\rightarrow [0,1]$ satisfies (1) and is additive. Then the following are equivalent:
1. Axiom(2) of Definition 3 (Countable Additivity)
2. If $A_n\in\mathcal{A}$ and $A_n\downarrow\emptyset$, then $P(A_n)\downarrow 0$.
3. If $A_n\in\mathcal{A}$ and $A_n\downarrow A$, then $P(A_n)\downarrow P(A)$.
4. If $A_n\in\mathcal{A}$ and $A_n\uparrow\Omega$, then $P(A_n)\uparrow 1$.
5. If $A_n\in\mathcal{A}$ and $A_n\uparrow A$, then $P(A_n)\uparrow P(A)$.

Note：这里的Notation $A_n\uparrow A$是指$A_n\subset A_{n+1}$ and $\cup_{n=1}^\infty A_n=A$; $A_n\downarrow A$是指$A_{n+1}\subset A_{n}$ and $\cap_{n=1}^\infty A_n=A$

Theorem 4 Let $P$ be a probability measure, and let $A_n$ be a sequence of events in $\mathcal{A}$ which converges to $A$. Then $A\in\mathcal{A}$ and $\lim_{n\rightarrow\infty}P(A_n)=P(A)$.

Proof: Let us define

$$\limsup_{n\rightarrow\infty} A_n = \cap_{n=1}^\infty\cup_{m\ge n} A_m \\ \liminf_{n\rightarrow\infty} A_n = \cup_{n=1}^\infty\cap_{m\ge n} A_m$$
Since $\mathcal{A}$ is a $\sigma$-algebra, we have $\limsup_{n\rightarrow\infty}A_n\in\mathcal{A}$ and $\liminf_{n\rightarrow\infty}A_n\in\mathcal{A}$.
By hypothesis $A_n$ converges to $A$, which means $\lim_{n\rightarrow\infty}1_{A_n}=1_A$, all $\omega$. This is equivalent to saying that $A=\limsup_{n\rightarrow\infty}A_n=\liminf_{n\rightarrow\infty}A_n$. Therefore $A\in\mathcal{A}$.
Now let $B_n=\cap_{m\ge n}A_m$ and $C_n=\cup_{m\ge n}A_m$. Then $\lim_{n\rightarrow\infty}P(B_n)=\lim_{n\rightarrow\infty}P(C_n)=P(A)$, by Theorem 3. However $B_n\subset A_n\subset C_n$, therefore $P(B_n)\le P(A_n)\le P(C_n)$, so $\lim_{n\rightarrow\infty}P(A_n)=P(A)$ as well.

Note: 这里通过引入$\limsup$和$\liminf$，将$A_n$转化成了一个递增与一个递减的序列。而后就可以直接利用Theorem 3证明结论了。