Probability Theory
Setup
Suppose that we are about to perform an experiment whose outcome is
not predictable in advance. The set of all possible outcomes of an
experiment is known as the sample space
For example, if the experiment consists of flipping a coin, then:
Any subset
We can define unions and intersections between 2 or more events. The
union of two events
Probabilities Defined on Events
Consider an experiment whose sample space is
- For any sequence of events
that are mutually exclusive:
Conditional Probabilities
Conditional probabilities are a powerful and useful concept. First, we are often interested in calculating probabilities and expectations when some partial information is available. Second, in calculating a desired probability or expectation, it is often extremely useful to first “condition” on some appropriate random variable.
We denote
Recall that for any 2 events
If
for all values of
Finally, the conditional expectation of
Computing Expectations by Conditioning
Let us denote by
Independent Events
Two events
This also implies that
Bayes’ Formula
Let