User:Hylon.chen/Probability concepts

Probability Concepts and Notations

djvu notes: Probability, distribution, density

Events(Samples, Outcomes)

Event (or sample or outcome) is the results of random experiments, which designated by $\mathbf\omega$.

For example, the result of tossing a coin once should be either head or tail, i.e., $\mathbf\omega =$ head or $\mathbf\omega =$ tail.
If we toss the coin twice, then $\mathbf\omega =$ {head, tail} or $\mathbf\omega =$ {head head} or $\mathbf\omega =$ {tail head} or $\mathbf\omega =$ {tail tail}.

Algebra of Events

In some ways the algebra of events is similar to the algebra of real numbers, with intersection ( $\cap$ ) corresponding to multiplication ( $\times$ ), complement ( $^{c}$ ) to subtraction ( $\mathbf -$ or $\setminus$ ) and union ( $\cup$ ) to addition ( $\mathbf +$ ).

The union of $n$ events $A_{1}, A_{2}, ..., A_{n}$ is the set consisting of those points that at least in one of those events $A_{1}, A_{2}, ..., A_{n}$.

Notation: $A_{1} \cup A_{2} \cup ... \cup A_{n}$, or $\cup_{i=1}^{\infty}A_{i}$

The intersection of $n$ events $A_{1}, A_{2}, ..., A_{n}$ is the set of points belonging to all those events $A_{1}, A_{2}, ..., A_{n}$.

Notation: $A_{1} \cap A_{2} \cap ... \cap A_{n}$, or $\cap_{i=1}^{\infty}A_{i}$

The complement of a event $A$ in $\Omega$ is the set of points in $\Omega$ but not the event $A$.

Notation: $A^{c}$.

Sample Space (Outcome Space)

Sample space or outcome space is a collection of possible outcomes (or events or samples) of random experiments, which denoted by $\mathbf \Omega$. For example, in the coin - tossing experiment, a coin is tossed once, the outcome space $\mathbf \Omega$ ={heads, tails}. Thus, $\omega \in \mathbf\Omega$ means the event $\mathbf \omega$ is either "heads" or "tails".

( Xiu 2010, p.9;, Shao 2007, p.1.).

Sigma-Field

Sigma-Field is a collection of subsets of sample space $\mathbf\Omega$ (not necessary all), which denoted by $\mathcal F$. For instance, $\mathcal F = \{ \emptyset, {\rm heads}, {\rm tails}, \mathbf\Omega\}$ in the coin - tossing experiment.

Three conditions that the sigma-field must satisfy:

$\bullet$ Non-empty: $\Omega \in \mathcal F$ and $\emptyset \in \mathcal F$;

$\bullet$ Given $A \in \mathcal F$, then $A^c \in \mathcal F$;

$\bullet$ Given $A_1$, $A_2$,...$\in \mathcal F$, then

$\bigcap_{i=1}^{\infty} A_{i} \in \mathcal F$ and $\bigcup_{i=1}^{\infty} A_{i} \in \mathcal F$.

i.e., sum or union of any subsets of $\mathcal F$ is a subset of $\mathcal F$.

( Xiu 2010, p.10;, Shao 2007, p.2.)

Note:
$\mathcal F$ is called a " sigma-field " or " sigma-algebra ", written as $\sigma$-field or $\sigma$-algebra.
$\sigma$ is mnemonic for " S ", and " Sum ", due to property.

Probability

Probability is used to measure the likelihood of occurrence of certain event (or outcome). Probability of an event $\mathbf \omega$ belonging to an element $A \in \mathcal F$ is a non-negative number (or measure), which is mathematically denoted by $P(\omega \in A)=P(A)$.

For example, in the coin - tossing experiment,
$P(heads)=P(tails)=\frac{1}{2}$, $P(\emptyset)=0$, $P(heads, tails)=P(heads)+P(tails)=\frac{1}{2}+\frac{1}{2}=1$.

Algebra of Probability

The complement of an event $A$ is the event not $A$ (that is, the event of $A$ not occurring); its probability is given by $P(not A) = 1 - P(A)$. As an example, the chance of not rolling a six on a six-sided die is 1 – (chance of rolling a six)= $1 - \frac{1}{6} = \frac{5}{6}$.

If both events $A$ and $B$ occur on a single performance of an experiment, this is called the intersection or joint of $A$ and $B$, denoted as $P(A \cap B)$. If two events, $A$ and $B$ are independent, then the joint probability is

$P(A \mbox{ and }B) = P(A \cap B) = P(A) P(B),\,$

for example, if two coins are flipped the chance of both being heads is $\frac{1}{2}\times\frac{1}{2} = \frac{1}{4}$.

If either event $A$ or event $B$ or both events occur on a single performance of an experiment this is called the union of the events $A$ and $B$ denoted as $P(A \cup B)$. If two events are mutually exclusive, then the probability of either occurring is :$P(A\mbox{ or }B) = P(A \cup B)= P(A) + P(B)$. For example, the chance of rolling a 1 or 2 on a six-sided die is $P(1\mbox{ or }2) = P(1) + P(2) = \frac{1}{6} + \frac{1}{6} = \frac{1}{3}$.

If the events are not mutually exclusive then :$\mathrm{P}\left(A \hbox{ or } B\right)=\mathrm{P}\left(A\right)+\mathrm{P}\left(B\right)-\mathrm{P}\left(A \mbox{ and } B\right)$. For example, when drawing a single card at random from a regular deck of cards, the chance of getting a heart or a face card (J,Q,K) (or one that is both) is $\frac{13}{52} + \frac{12}{52} - \frac{3}{52} = \frac{11}{26}$, because of the 52 cards of a deck 13 are hearts, 12 are face cards, and 3 are both: here the possibilities included in the "3 that are both" are included in each of the "13 hearts" and the "12 face cards" but should only be counted once.

Random Variable

Intuitively, Random Variable is used to designate a random outcome (event or sample) in a random experiment, usually denoted in capital letters, e.g., $\mathbf X$ is a random variable. It's a numerical description of the outcome of an experiment.

Formally, it is a mapping from a probability space to the real numbers, which is measurable.

$\displaystyle \mathbf X:(\Omega ,\mathcal F ) \to (\mathbb R, \mathcal B)$

$\omega \mapsto\mathbf X(\omega)$

Where $(\Omega ,\mathcal F )$ is event space $\Omega$ endowed with $\sigma$ - algebra $\mathcal F$, $(\mathbb R, \mathcal B)$ is set of real numbers $\mathbb R$ endowed with " Borel $\sigma$ - algebra " $\mathcal B$ (sigma-algebra of finite open subsets of $\mathbb R$). (Shao 2007, p.7.)

$\mathbf X(\omega)$ = ( arbitrary ) number selected to represent each event $\mathbf\omega$ in $\mathbf\Omega$. For Example, typically, in the coin - tossing experiment, we can use number 1 to designate the heads and 0 for the tails, i.e., $\mathbf X(\rm heads)= 1$, $\mathbf X(\rm tails)= 0$. But it is also possible to select, event though not a good choice, since not as mnemonic as $\{0,1\}$, $\mathbf X(\rm heads)= 5$, $\mathbf X(\rm tails)= -3$.

Example:

$\mathcal B = \sigma \Big( \{(a,b]: a,b \in \mathbb R \} \Big)$

$\{(a,b]: a,b \in \mathbb R \}$ Set of finite open intervals in $\mathcal B$

This choice of $\mathcal B$ allows for the probability of

$\mathbf X \in (a,b]$, i.e., $\mathbf P \big(\mathbf X \in (a,b]\big)$.

(Xiu 2010, p.11)

$\mathbf P_{\mathbf X}$ Probability Distribution

Probability Distribution is a function that describes the probability of a random variable taking certain values.

$\displaystyle P_X((a,b]) \equiv P_X(X( \omega) \in (a,b])$.

$\mathbf F$_{$\mathbf X$} Cumulative distribution function ( CDF )

Cumulative Distribution Function (CDF), or only Distribution Function, describes the probability a real-valued random variable $\mathbf X$ with a given probability distribution will be found at a value less than or equal to x. Intuitively, it is the "area so far" function of the probability distribution.

$\displaystyle F_X(x) := P_X((-\infty, x]) = P_X(X \le x)$

$f_{\mathbf X}$ Probability density function ( PDF )

Probability Density Function (PDF), or density of a continuous random variable is a function that describes the relative likelihood for this random variable to occur at a given point. The probability for the random variable to fall within a particular region is given by the integral of this variable’s density over the region. The probability density function is nonnegative everywhere, and its integral over the entire space is equal to one.

$\displaystyle f_{\mathbf X}(x):=\frac{d}{dx}\mathbf F_{\mathbf X}(x)$
$\displaystyle \mathbf F_{\mathbf X}(x)=\int_{-\infty}^{x} f_{\mathbf X}(t)dt$

Note:

Notation $\mathbf X$ and $x$

In recent literature, an uppercase letter, e.g., $\mathbf X$, is used to designate a random variable , whereas the corresponding lowercase letter, e.g., $x$, is used to designate the real variable that is the upper bound of $\mathbf X$.

Kolmogorov, 1933; Famous work influencing subsequent mathematical probability and statistics works.