Introduction
It frequently occurs that in performing an experiment, we are mainly interested in some function of the outcome rather than the outcome itself. These quantities of interest, are real-valued functions defined on the sample space \(\mathcal{S}\), known as random variables.
Discrete Random Variables
For a discrete random variable, we define the probability mass function \(p(a)\) of \(X\) by:
\begin{equation} \label{dfn:pmf} p(a) = P(X = a) \end{equation}
The probability mass function \(p(a)\) is positive for at most a countable number of values of \(a\).
Since \(X\) must take on one of the values \(x_i\), we have \(\sum_{i=1}^{\infty} p(x_i) = 1\).
The cumulative distribution function \(F\) can be expressed in terms of \(p(a)\) by:
\begin{equation} \label{dfn:cdf} F(a) = \sum_{\text{all } x_i \le a} p(x_i) \end{equation}
Bernoulli Random Variable
A random variable \(X\) is said to be a Bernoulli random variable if its probability mass function is given by:
\begin{equation} p(0) = P(X = 0) = 1 - p, p(1) = P(X = 1) = p \end{equation}
This corresponds to the outcome of a trial with binary outcomes.
Binomial Random Variable
Suppose \(n\) independent trials, each of which have binary outcomes and result in a success with probability \(p\). If \(X\) represents the number of successes that occur in the \(n\) trials, then \(X\) is a binomial random variable denoted by \(Bin(n, p)\).
The pmf of a binomial random variable is given by:
\begin{equation} p(i) = P(X = i) = {n \choose i}p^i (1-p)^i, i = 0,1,\dots,n \end{equation}
Geometric Random Variable
Suppose that independent trials, each having probability \(p\) of being a success, are performed until a success occurs. If we let \(X\) be the number of trials required until the first success, then \(X\) is said to be a geometric variable with parameter \(p\).
\begin{equation} p(n) = P(X = n) = (1-p)^{n-1}p \end{equation}
Poisson Random Variable
A random variable \(X\), taking on one of the values \(0, 1, 2, \dots\) is said to be a poisson random variable with parameter \(\lambda\) if for some \(\lambda > 0\),
\begin{equation} p(i) = p(X = i) = e^{-\lambda}\frac{\lambda^i}{i!}, i = 0, 1, \dots \end{equation}
An important property of the Poisson random variable is that it may be used to approximate a binomial random variable when the binomial parameter \(n\) is large and \(p\) is small.
Continuous Random Variables
Continuous random variables have an uncountable set of possible values.
\begin{equation} P[X \in B] = \int_{B}f(x)dx \end{equation}
The cumulative distribution \(F(\cdot)\) and the probability density function is expressed by:
\begin{equation} F(a) = P(X \in (-\infty, a]) = \int_{-\infty}^{a} f(x) dx \end{equation}
Uniform Random Variable
\(X\) is a uniform random variable on the interval \((a, b)\) if its probability density function is given by:
\begin{equation}
f(x) = \begin{cases}
\frac{1}{b - a} & a < x < b \\
0 & \text{otherwise}
\end{cases}
\end{equation}
Exponential Random Variables
\begin{equation}
f(x) = \begin{cases}
\lambda e^{-\lambda x} & x \ge 0 \\
0 & x < 0
\end{cases}
\end{equation}
\begin{equation} F(a) = 1 - e^{-\lambda }, a \ge 0 \end{equation}
Gamma Random Variables
\begin{equation}
f(x) =
\begin{cases}
\frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda
x} & x \ge 0 \\
0 & x < 0
\end{cases}
\end{equation}
The quantity \(\Gamma(\alpha)\) is called the gamma function and is defined by:
\begin{equation} \Gamma(\alpha) = \int_{0}^{\infty}e^{-x}x^{\alpha-1}dx \end{equation}
and \(\Gamma(n) = (n - 1)!\) for some integral \(\alpha = n\).
Normal Random Variables
\begin{equation} f(x) = \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}}, -\infty < x < \infty \end{equation}
if \(X ~ N(\mu, \sigma^2)\) and \(Y = aX + b\), then \(Y ~ N(a\mu + b, a^2\sigma^2)\).
Expectation of Random Variables
The expected value of a discrete random variable \(X\) is defined by:
\begin{equation} E(X) = \sum_{x:p(x)>0} x p(x) \end{equation}
and for a continuous random variable is defined similarly:
\begin{equation} E(X) = \int_{-\infty}^{\infty} x f(x) dx \end{equation}
Suppose we are interested in getting the expected value of a function of a random variable \(E(g(X))\). One way is to obtain the distribution of \(g(X)\), and compute \(E(g(X))\) by definition of expectation.
\begin{equation} E(g(X)) = \sum_{x:p(x)>0} g(x) p(x) \end{equation}
We also have linearity of expectations:
\begin{equation} E(aX + b) = aE(X) + b \end{equation}
\(E(X)\) is the first moment of \(X\), and \(E(X^n)\) is the nth moment of \(X\).
The variance of \(X\), \(Var(X)\) measures the expected square of the deviation of \(X\) from its expected value:
\begin{equation} Var(X) = E((X-E(X))^2) = E(X^2) - (E(X))^2 \end{equation}
Jointly Distributed Random Variables
The joint cumulative probability distribution function of \(X\) and \(Y\) is given by:
\begin{equation} F(a, b) = P(X \le a, Y \le b), -\infty < a , b < \infty \end{equation}
The distribution of \(X\) can then be obtained from the joint distribution of \(X\) and \(Y\) as follows:
\begin{align}
F_X(a) &= P(X \le a, Y \le \infty) \\
&= F(a, \infty)
\end{align}
Covariance and Variance of Sums of Random Variables
The covariance of any 2 random variables \(X\) and \(Y\), denoted by \(Cov(X,Y)\), is defined by:
\begin{align}
Cov(X,Y) &= E\left[ (X - E[X])(Y - E[Y]) \right] \\
&= E[XY] - E[X]E[Y]
\end{align}
If \(X\) and \(Y\) are independent, then \(Cov(X,Y) = 0\).
Some properties of covariance:
- \(Cov(X,X) = Var(X)\)
- \(Cov(X,Y) = Cov(Y,X)\)
- \(Cov(cX, Y) = c Cov(X,Y)\)
- \(Cov(X, Y+Z) = Cov(X,Y) + Cov(X,Z)\)