Chien-Lin Su
$\newcommand{\cupinfty}[0]{\cup_{i=1}^{\infty}A_i}$ $\newcommand{\cupinftyup}[1]{\cup_{i=1}^{#1}A_i}$ $\newcommand{\capinfty}[0]{\cap_{i=1}^{\infty}A_i}$ $\newcommand{\capinftyup}[1]{\cap_{i=1}^{#1}A_i}$ $\newcommand{\limitinfty}[0]{\lim_{n \rightarrow \infty}A_n}$ $\newcommand\braces[1]{\left\{#1\right\}}$ $\newcommand{\mathcomment}[1]{\qquad\color{blue}{(#1)}}$ $\newcommand{\bigsum}[2]{\sum\limits_{#1}^{#2}}$
Experiment - process by which an observation is made
Ex dice tossing is an experiment since we observe the number appearing on the other side
Random Experiment
Random experiment - TODO
Sample space - S or Ω - collection of every possible outcome
Event - A, B, C - partial collection of sample space
Let P(A) = probability of A ∀A, A ∈ Ω P(A) has the following axioms:
Limit of a sequence of sets
Non-decreasing sequence - sequence of $A_i$ such that $A_j \subseteq A_k$ if $i < k$
Non-increasing sequence - sequence of $A_i$ such that $A_j \supseteq A_k$ if $i > k$
Example 1
\[\begin{align} \text{Let } A_k & = \braces{x \mid 1 < x \le 2 - \dfrac{1}{k}} \\ A_1 & = \braces{x \mid 1 < x \le 2 - \dfrac{1}{1}} \\ A_2 & = \braces{x \mid 1 < x \le 2 - \dfrac{1}{2}} \\ & ... \end{align}\]Note that the sequence is non-decreasing.
$\limitinfty = \cupinfty = \braces{x \mid 1 < x < 2}$
Note that $x < 2$ is open
Example 2
$A_k = \braces{x \mid 1 < x \le x + \dfrac{1}{k}}$
Note that the sequence is non-increasing.
$\limitinfty = \capinfty = \braces{x \mid 1 < x \le 1} = \emptyset$
Note that $x \le 1$ is still closed.
Partition - sequence of mutually exclusive sets which together form the whole. More formally:
Let $\braces{A_i}_{i = 1}^{\infty (k)}$ be a sequence of sets
$A_i \le \Omega, \forall i$
If $A_i \cap A_j = \emptyset, \forall i \ne j$ and $\cupinftyup{\infty (k)} = \Omega$
then $\braces{A_i}_{i=1}^{\infty (k)}$ is a partition of sample space $\Omega$
Note - if $B$ is any subset of $\Omega$ & $\braces{A_1, …, A_k}$ is a partition of $\Omega \Rightarrow B = \cupinftyup{k} (A_i \cap B)$
Let $F$ be a set that collects possible sets from $\Omega$
P: $F \rightarrow R^+$ is called a probability measure if
a. $P(A) \ge 0 \quad \forall A \in F \mathcomment{\text{non-negative}}$
b. $P(\Omega) = 1$
c. $P(\cupinfty) = \Sigma_{i=1}^{\infty} P(A_i), A_i \cap A_j = \emptyset \quad \forall i \ne j$
Some extensions:
Proof
Claim: $P(\phi) = 0$
$\because \Omega = \Omega \cup \phi$
$\therefore P(\Omega) = P(\Omega \cup \phi)$
$= P(\Omega) + P(\phi) (\therefore \Omega \cap \phi = \phi)$
$= 1+ P(\phi)$
$\therefore P(\phi) = 0$
Claim: $P(A^C) = 1 - P(A)$
$\because \Omega = A \cup A^C$
$\therefore P(\Omega) = P(A \cup A^C)$
$= P(A) + P(A^C) — (A \cap A^C = \phi)$
$\therefore 1 = P(A) + P(A^C)$
$\therefore P(A^C) = 1 - P(A)$
Claim: $A \subseteq B \Rightarrow P(A) \le P(B)$
$P(B) = P(A \cup (A^C \cap B))$
$= P(A) + P(A^C \cap B) — (A \cap (A^C \cap B) = \emptyset)$
$\ge P(A)$
Claim: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
$\because A \cup B = A \cup (A^C \cap B)$
$\therefore P(A \cup B) = P(A \cup (A^C \cap B)) — (A \cap (A^C \cap B) = \emptyset)$
$= P(A) + P(A^C \cap B)$
Note: $P(B) = P((A \cap B) \cup (A^C \cap B))$
$= P(A \cap B) + P(A^C \cap B)$
$\Rightarrow P(A^C \cap B) = P(B) - P(A \cap B)$
//todo include other stuff
1.5 Conditional Probability and Baye’s Rule
Def 1 The conditional probability of an event A, given that an event B has occurred, is equal to
$P(A \mid B) = \dfrac{P(A \cap B)}{P(B)} if P(B) > 0$
Remark 1:
Here A is the event whose uncertainty we want to update, and B is the evidence we observe (or want to treat as given
We call P(A) the prior probability of A, and P(A | B) the posterior probability of A.)
Prior - before updating based on evidence
Posterior - after updating based on evidence
Remark 2:
$P(A \mid B)$ is a probability measure
Check 1:
$P(A \mid B) = \dfrac{P(A \cap B)}{P(B)} > 0$
Since $P(B) > 0$ and $P(A \cap B) > 0$
Check 2:
$P(\Omega \mid B) = \dfrac{P(\Omega \cap B)}{P(B)} = \dfrac{P(B)}{P(B)} = 1$
Check 3:
see check 3