概率统计笔记 CH3

Discrete Random Variables and Probability Distributions

离散随机变量与概率分布

概率统计
https://www.zybuluo.com/daixuan1996/note/57380

• The concept of a random variable allows us to pass from the experimental outcomes themselves to a numerical function of the outcomes.
• There are two fundamentally different types of random variables--discrete random variables and continuous random variables.

1. Random Variables

   • random variable (rv):

     • For a given sample space S of some experiment, a random variable is any rule that associates a number with each outcome in S.
     • A random variable is a function whose domain(值域) is the sample space of the experiment, and range(定义域) is the real numbers. $X: S → R$
     • The notation $X(s)=x$ means that x is the value associated with the outcome s by the rv X.
     • Any random variable whose only possible values are 0 and 1 is called a Bernoulli(伯努利) random variable.

   • Two types of random variables

     1. A discrete random variable is an rv whose possible values either constitute a finite set or else can be listed in an infinite sequence in which there is a first element, a second element, and so on.
     2. A random variable is continuous if its set of possible values consists of an entire interval on the number line.

2. Probability Distributions for Discrete Random Variables

   • The rule for describing the probability measure associated with all values of a random variable is called a probability distribution.

   • The probability distribution(概率分布) or probability mass function - pmf (概率质量函数) of a discrete rv is defined for every number x by $p(x)=P(X=x)$

     • Any pmf must be required to satisfy the following conditions: $$p(x)\ >=\ 0\ and\ \sum_{all\ possible\ x}{p(x)}\ =\ 1$$
     • A probability distribution can also be represented graphically as a histogram. The histogram represents the relative frequency we would expect to see if we repeated an experiment infinitely many times.

   • A parameter of a probability distribution

     • Suppose p(x) depends on a quantity that can be assigned any one of a number of possible values, with each different value determining a different probability distribution. Such a quantity is called a parameter(参数) of the distribution.
     • The collection of all probability distributions for different values of the parameter is called a family of probability distributions(概率分布族).
       e.g: The family of Bernoulli distributions:
       $$\begin{equation} p(x; \alpha) \begin{cases} 1-\alpha & if\ x = 0 \\ \alpha & if\ x = 1 \\ 0 & otherwise \end{cases} \end{equation}$$

   • The cumulative distribution function - cdf (累积分布函数) F(x) of a discrete rv X with pmf p(x) is defined for every number x by

     $$F(X)\ =\ P(X\ \le\ x) = \sum_{y:y \le x}{p(y)}$$

     • The cdf satisfies the following properties:
       
       1. The cdf is non-decreasing.
       2. The cdf satisfies : $lim_{x\to-\infty}{F(x)} = 0$ $lim_{x\to+\infty}{F(x)} = 1$
       3. It is comtinuous in x.
          For any two numbers a and b with a $\le$ b, $P(a \le n) = F(b) - F(a-)$ where “a-” represents the largest possible X value that is strictly less than a.
          a, b都是整数时，$P(a \le n) = F(b) - F(a-1)$
     • $P(X = a) = F(a) - F(a-1)$
       

3. Expected Values(期望) of Discrete Random Variables

   • Let X be a discrete rv with set of possible values D and pmf p(x).The expected values(期望值) or mean value(均值) of X, denoted by E(X) or $\mu_x$ , is

     $$E(X) = \mu_x = \sum_{x \in D}{x*p(x)}$$

     • When the sum does not exist, we say the expectation of X does not exist.
     • The population mean(总体均值) is the mean value of the population.
     • The probability distribution of X has "a heavy tail(重尾)" if its E(X) is not finite.

   • The Expected Value of a Function

     • Let X be a discrete rv with set of possible values D and pmf p(x). Then the expected values or mean value of any function h(X), denoted by $E[h(X)]$ or $\mu_{h(X)}$, is computed by $$E[h(X)] = \sum_{x\in D}{h(x)p(x)}$$

   • Rules of Expected Value

     1. $E(Y_1 + Y_2) = E(Y_1) + E(Y_2)$
     2. $E(bY) = bE(Y)$
     3. $E(C) = C(for constant\ C)$
     4. $E(Y_1Y_2) = E(Y_1)E(Y_2)\ when\ Y_1 and\ Y_2\ are\ independent$

   • The Variance of X

     • Let X have pmf p(x) and the expected value $\mu$. Then the variance(方差) of X, denoted by V(X) or just $\sigma^2$, is $$V(X) = \sum_{x \in D}{(x-\mu)^{2}·p(x)} = E[(X - \mu)^2]$$
     • The standard deviation(标准差) of X is $\sigma_x = \sqrt{\sigma_x^2}$.
     • popuplation variance/standard deviation(总体方差/标准差)

   • Properties of Variance
     
     1. $V(X) = E(X^2)-E(X)^2$
     2. $V(C) = 0, for all constant C$
     3. $V(aX+b) = a^2 V(X)$
     4. $V(X+Y) = V(X) + V(Y)\ if\ X\ and\ Y\ are\ indenpendent$

4. The Binomial Probability Distribution(二项分布)

   • Properties of a Binomial experiment

     1. The experiment consists of a sequence of n smaller trials, where n is fixed in advance of the experiment.
     2. Two outcomes are possible on each trial.
     3. The probability of a success or a failure, denoted by p and 1-p, does not change from trial to trial.
     4. The trials are independent.

   • Suppose each trial of an experiment can result in S or F, but the sampling is without replacement from a population of size N. If the sample size (number of trials) n is at most 5% of the population size, the experiment can be analyzed as though it were exactly a binomial experiment.

   • The Binomial Random Variable and Distribution

     • Possible values for X in an n-trial experiment are $x = 0,1,2,…,n$. We will often write X~Bin(n,p) to indicate that X is a binomial rv based on n trials with success probability p.
     • We denote the pmf by $b(x;n,p)$
       $$\begin{equation} b(x;n,p) = \begin{cases} C_n^x p^x (1-p)^{n-x} & x = 0,1,2,...n \\ 0 & otherwise \end{cases} \end{equation}$$
     • For X~Bin(n,p), the cdf will be denote by
       $$P(X\le x)=B(x;n,p) = \sum_{y=0}^x{b(y;n,p)}\ \ x = 0,1,2,...,n$$

   • The Mean and Variance of X

     • $E(X)=np$
     • $V(X)=np(1-p)=npq$
     • $\sigma_x=\sqrt{npq}$

5. Hypergeometric and Negative Binomial Distributions(超几何分布与负二项分布)

   • Properties of Hypergeometric Distribution(超几何分布)

     1. The population or set to be sampled consists of N individuals, objects, or elements.(a finite population)
     2. Each individual can be characterized as a success (S) or a failure (F), and there are M successes in the population.
     3. A sample of n individuals is selected without replacement in such a way that each subset of size n is equally likely to be chosen.

   • The Hypergeometric Random Variable and Distribution

     • The random variable of interest is X = the number of S’s in the sample.
     • If X is the number of S’s in a completely random sample of size n drawn from a population consisting of M S’s and (N-M) F’s, then the probability distribution of X, called the hypergeometric distribution(超几何分布), is given by
       $$P(X=x) = h(x;n,M,N)=\frac{C_M^x C_{N-M}^{n-x}}{C_N^n}$$

   • The Mean and Variance of X

     • $E(X) = n·\frac{M}{N}$
     • $V(X) = \frac{N-n}{N-1}·n·\frac{M}{N}·(1-\frac{M}{N})$
     • $\frac{N-n}{N-1}$ is often called the finite population correction factor(有限总体校正因子).

   • Properties of Negative Binomial Distribution(负二项分布)

     1. The experiment consists of a sequence of independent trials.
     2. Each trial can result in either a success(S) or a failure(F).
     3. The probability of success is constant from trial to trial, so $P(S on trial i) = p\ for\ i=1,2,3,…$
     4. The experiment continues (trials are performed) until a total of r successes have been observed, where r is a specified positive integer.

   • The Negative Binomial Random Variable and Distribution

     • The random variable of interest is X = the number of failures that precede the rth success.
       X is called a negative binomial random variable because, in contrast to the binomial rv, the number of successes is fixed and the number of trials is random.
     • The pmf of the negative binomial rv X with parameters r = number of S’s and p = P(S) is
       $$P(X=x) = nb(x;r,p) = C_{x+r-1}^{r-1}p^r (1-p)^x\ \ x=0,1,2,...$$
     • In the special case r = 1, the pmf is
       $$nb(x;1,p) = (1-p)^x p\ \ x=0,1,2,...$$
       Then the distribution is called the geometric distribution(几何分布).

   • The Mean and Variance of X

     • $E(X) = \frac{r(1-p)}{p}$
     • $V(X) = \frac{r(1-p)}{p^2}$

6. The Poisson Probability Distribution(泊松分布)

   • Definition:

     • A random variable X is said to have a Poisson distribution if the pmf of X is $$p(x;\lambda) = \frac{e^{-\lambda} \lambda^x}{x!} \ \ x=0,1,2,3...\ \ \ with\ \lambda > 0$$
       The value of λ is frequently a rate per unit time or per unit area.
     • $$e^{\lambda} = 1 + \lambda + \frac{\lambda^2}{2!} + ... = \sum_{x=0}^{\infty}\frac{\lambda^x}{x!}$$ This show that $$\sum_{x=0}^{\infty}p(x; \lambda) = 1$$

   • Properties of a Possion variable

     1. There may have infinite number of trials
     2. Each trial results in either S or F
     3. Trials are independent
     4. The probability that an event occurs in a short interval is proportional(成比例的) to the length of the interval
     5. The probability of two or more events occurring in a very short interval is negligible(可以忽略的)

   • The Possition Distribution as a Limit

     • Suppose that in the binomial pmf b(x;n,p), we let n → $\infty$ and p → 0 in such a way that np approaches a value λ > 0. Then b(x;n,p) → p(x;λ).
     • As a rule of thumb, this approximation can safely be applied if n ≥ 100, p ≤ 0.01, and np ≤ 20.

   • The Mean and Variance of X

     • $E(X)\ =\ V(X) = \lambda$

   • The Poisson Process

     • $$\lambda = \alpha t \Rightarrow P_k(t) = \frac{e^{-\alpha t}·(\alpha t)^k}{k!}$$
     • The number of events occurring during a fixed time interval of length t has a Possion distribution with parameter $\alpha$t. Any process that has this distribution is called a Poisson process.

Copyright © 2014 by Xuan Dai. All rights reserved.