概率统计笔记 CH2

概率统计

Probability

概率

Probability is the mathematical language of uncertainty. It refers to the study of randomness and uncertainty.

1. Sample Spaces and Events

   • Random experiment: An experiment is any action or process that generates observations.
   • Sample Space: The sample space of an experiment, denoted by S, is the set of all possible outcomes of that experiment. e.g: S = {ss, sf, fs, ff}
   • Sample point: The sample point (element) of the sample space, denoted by s, is an outcome of the experiment.

   • Random event: An event, is any collection (subset) of outcomes contained in the sample space S . An event is said to be simple(简单的) if it consists of exactly one (elementary) outcome and compound(复合的) if it consists of more than one outcome. e.g: A = {ss, sf} (A is an event of S - success at first)

     Impossible event (不可能事件)
     Certain event (必然事件)
     Complementary event(逆事件、对立事件) e.g: $\bar{A} or A^{'}$= {fs,\ ff}

   • Some relations from set theory:

     Union(并 $\bigcup$)
     Intersection(交 $\bigcap$)

   • When A and B have no outcomes in common, they are said to be mutually exclusive(互斥) or disjoint(不相交)events.

     $A \bigcup A^{'} = \Phi$

   • Venn Diagrams (维恩图)

     E1 and E2, are collectively exhaustive(完全穷尽) if all sample points are contained within their union. 所有样本点充满了他们的并集

2. Axioms(公理), Interpretations, and Properties of Probability

   • Axioms:

     $For\ any\ event A,\ P(A) \geq 0$
     $P(S) = 1$
     $P(A_1 \bigcup A_2 \bigcup A_3 \bigcup ...) = 1$

   • How do we assign probability:

     Classical probability(古典概型): Based on gambling ideas, the fundamental assumption is that the game is fair and all elementary outcomes have the same probability.
     Relative frequency(相对频数，即频率): When an experiment can be repeated, an event’s probability is the proportion of times the event occurs in the long run.
     Personal or subjective probability(主观概率): Degree of belief that it is rational to put on given evidence. Personal assessment.

   • Properties:

     $P(A^{'}) = 1 - P(A)$
     $P(\Phi) = 0$
     $if\ A \subset B, then P(A) \leq P(B)$
     $Addition\ law:\ P(A \bigcup B) = P(A) + P(B) - P(A \bigcap B)$ 也可类比地用到三个数的并

3. Counting Techniques

   • When the various outcomes of an experiment are equally likely, if N is the number of outcomes in a sample space and N(A) is the number of outcomes contained in an event A, then $P(A) = \frac{N(A)}{N}$.

   • The product rule for ordered pairs

     If one experiment has m outcomes and another experiment has n outcomes ,then there are m×n possible outcomes for the two experiments.

   • Tree Diagram (树图)

     
     

   • A more general product rule

     If there are p experiments and the first has $n_1$ possible outcomes, the second $n_2$..., and the pth $n_p$ possible outcomes ,then there are a total of $n_1\times n_2\times ...n_p$ possible outcomes for the p experiments.

   • Permutation(排列): Any ordered sequence of k objects taken from a set of n distinct objects is called a permutation of size k of the objects. The number of permutations of size k that can be constructed from the n objects is denoted by $P_n^k$.

     $$P_n^k = n(n-1)(n-2)...(n-k+1) = \frac{n!}{(n-k)!}$$

   • Combination(组合): Given a set of n distinct objects, any unordered subset of size k of the objects is called a combination. The number of combinations of size k that can be formed from the n distinct objects will be denoted by $C_n^k$ or $\left(^n_k\right)$.

     $$C_n^k = \frac{P_n^k}{k!} = \frac{n!}{k!(n-k)!}$$

4. Conditional Probability(条件概率)

   • Conditional Probability: For any two events A and B with P(B)>0, the conditional probability of A given that B has occurred is defined by

     $$P(A|B) = \frac{P(A \bigcap B)}{P(B)}$$
     

   • Multiplication Law

     $P(A \bigcap B) = P(A|B)P(B) = P(B|A)P(A)$
     $P(A_1 \bigcap A_2 \bigcap A_3) = P(A_3|A_1 \bigcap A_2)P(A_1 \bigcap A_2) = P(A_3|A_1 \bigcap A_2)P(A_2|A_1)P(A_1)$

   • De Morgan's Law(狄摩根律)

     $(P(\overline{A_1 \bigcup A_2 \bigcup A_3}) = P(\bar{A_1} \bigcap \bar{A_2} \bigcap \bar{A_3})$

   • Law of Total Probability(全概率公式)

     If $A_1,A_2,...,A_k$ be mutually exclusive and exhaustive events, and $P(A_i) > 0\ with\ i = 1,...,k$, then we name $A_1,A_2,...,A_k$ a dividing of the sample space(样本空间的划分).
     Suppose $A_1,A_2,...,A_k$ is a dividing of the sample space, then

     $$P(B) = \sum_{i=1}^{n}P(A_i)P(B|A_i)$$

   • Baye's Theorem(贝叶斯定理)

     Suppose $A_1,A_2,...,A_k$ is a dividing of the sample space, then

     $$P(A_i|B) = \frac{P(A_i \bigcap B)}{P(B)} = \frac{P(A_i)P(B|A_i)}{P(B)} = \frac{P(A_i)P(B|A_i)}{\sum_{k=1}^{n}P(A_k)P(B|A_k)}$$
     

5. Independence(独立性)

   • Definition: Two events A and B are independent if P(A|B)=P(A) and are dependent otherwise.

     The definition of independence might seem “unsymmetric” because we do not demand that P(B|A)= P(B) also. However, using the definition of conditional probability and the multiplication rule, we have P(A|B)=P(A) iff P(B|A)=P(B).

   • Proposition: A and B are independent iff P(A$\bigcap$B)=P(A)P(B).

     Events $A_1,A_2,...,A_n$ are mutually independent(相互独立) if for every k (k=2,3,…,n) and every subset of indices $i_1,i_2,...,i_k$,

     $$P(A_{i_1} \bigcap A_{i_2} \bigcap ... \bigcap A_{i_k}).$$

Copyright © 2014 by Xuan Dai. All rights reserved.