Introduction

It frequently occurs that in performing an experiment, we are mainly interested in some function of the outcome rather than the outcome itself. These quantities of interest, are real-valued functions defined on the sample space \(\mathcal{S}\), known as random variables.

Discrete Random Variables

For a discrete random variable, we define the probability mass function \(p(a)\) of \(X\) by:

\begin{equation} \label{dfn:pmf} p(a) = P(X = a) \end{equation}

The probability mass function \(p(a)\) is positive for at most a countable number of values of \(a\).

Since \(X\) must take on one of the values \(x_i\), we have \(\sum_{i=1}^{\infty} p(x_i) = 1\).

The cumulative distribution function \(F\) can be expressed in terms of \(p(a)\) by:

\begin{equation} \label{dfn:cdf} F(a) = \sum_{\text{all } x_i \le a} p(x_i) \end{equation}

Bernoulli Random Variable

A random variable \(X\) is said to be a Bernoulli random variable if its probability mass function is given by:

\begin{equation} p(0) = P(X = 0) = 1 - p, p(1) = P(X = 1) = p \end{equation}

This corresponds to the outcome of a trial with binary outcomes.

Binomial Random Variable

Suppose \(n\) independent trials, each of which have binary outcomes and result in a success with probability \(p\). If \(X\) represents the number of successes that occur in the \(n\) trials, then \(X\) is a binomial random variable denoted by \(Bin(n, p)\).

The pmf of a binomial random variable is given by:

\begin{equation} p(i) = P(X = i) = {n \choose i}p^i (1-p)^i, i = 0,1,\dots,n \end{equation}

Geometric Random Variable

Suppose that independent trials, each having probability \(p\) of being a success, are performed until a success occurs. If we let \(X\) be the number of trials required until the first success, then \(X\) is said to be a geometric variable with parameter \(p\).

\begin{equation} p(n) = P(X = n) = (1-p)^{n-1}p \end{equation}

Poisson Random Variable

A random variable \(X\), taking on one of the values \(0, 1, 2, \dots\) is said to be a poisson random variable with parameter \(\lambda\) if for some \(\lambda > 0\),

\begin{equation} p(i) = p(X = i) = e^{-\lambda}\frac{\lambda^i}{i!}, i = 0, 1, \dots \end{equation}

An important property of the Poisson random variable is that it may be used to approximate a binomial random variable when the binomial parameter \(n\) is large and \(p\) is small.

Continuous Random Variables

Continuous random variables have an uncountable set of possible values.

\begin{equation} P[X \in B] = \int_{B}f(x)dx \end{equation}

The cumulative distribution \(F(\cdot)\) and the probability density function is expressed by:

\begin{equation} F(a) = P(X \in (-\infty, a]) = \int_{-\infty}^{a} f(x) dx \end{equation}

Uniform Random Variable

\(X\) is a uniform random variable on the interval \((a, b)\) if its probability density function is given by:

\begin{equation} f(x) = \begin{cases} \frac{1}{b - a} & a < x < b \\
0 & \text{otherwise} \end{cases} \end{equation}

Exponential Random Variables

\begin{equation} f(x) = \begin{cases} \lambda e^{-\lambda x} & x \ge 0 \\
0 & x < 0 \end{cases} \end{equation}

\begin{equation} F(a) = 1 - e^{-\lambda }, a \ge 0 \end{equation}

Gamma Random Variables

\begin{equation} f(x) = \begin{cases} \frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x} & x \ge 0 \\
0 & x < 0 \end{cases} \end{equation}

The quantity \(\Gamma(\alpha)\) is called the gamma function and is defined by:

\begin{equation} \Gamma(\alpha) = \int_{0}^{\infty}e^{-x}x^{\alpha-1}dx \end{equation}

and \(\Gamma(n) = (n - 1)!\) for some integral \(\alpha = n\).

Normal Random Variables

\begin{equation} f(x) = \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}}, -\infty < x < \infty \end{equation}

if \(X ~ N(\mu, \sigma^2)\) and \(Y = aX + b\), then \(Y ~ N(a\mu + b, a^2\sigma^2)\).

Expectation of Random Variables

The expected value of a discrete random variable \(X\) is defined by:

\begin{equation} E(X) = \sum_{x:p(x)>0} x p(x) \end{equation}

and for a continuous random variable is defined similarly:

\begin{equation} E(X) = \int_{-\infty}^{\infty} x f(x) dx \end{equation}

Suppose we are interested in getting the expected value of a function of a random variable \(E(g(X))\). One way is to obtain the distribution of \(g(X)\), and compute \(E(g(X))\) by definition of expectation.

\begin{equation} E(g(X)) = \sum_{x:p(x)>0} g(x) p(x) \end{equation}

We also have linearity of expectations:

\begin{equation} E(aX + b) = aE(X) + b \end{equation}

\(E(X)\) is the first moment of \(X\), and \(E(X^n)\) is the nth moment of \(X\).

The variance of \(X\), \(Var(X)\) measures the expected square of the deviation of \(X\) from its expected value:

\begin{equation} Var(X) = E((X-E(X))^2) = E(X^2) - (E(X))^2 \end{equation}

Jointly Distributed Random Variables

The joint cumulative probability distribution function of \(X\) and \(Y\) is given by:

\begin{equation} F(a, b) = P(X \le a, Y \le b), -\infty < a , b < \infty \end{equation}

The distribution of \(X\) can then be obtained from the joint distribution of \(X\) and \(Y\) as follows:

\begin{align} F_X(a) &= P(X \le a, Y \le \infty) \\
&= F(a, \infty) \end{align}

Covariance and Variance of Sums of Random Variables

The covariance of any 2 random variables \(X\) and \(Y\), denoted by \(Cov(X,Y)\), is defined by:

\begin{align} Cov(X,Y) &= E\left[ (X - E[X])(Y - E[Y]) \right] \\
&= E[XY] - E[X]E[Y] \end{align}

If \(X\) and \(Y\) are independent, then \(Cov(X,Y) = 0\).

Some properties of covariance:

  1. \(Cov(X,X) = Var(X)\)
  2. \(Cov(X,Y) = Cov(Y,X)\)
  3. \(Cov(cX, Y) = c Cov(X,Y)\)
  4. \(Cov(X, Y+Z) = Cov(X,Y) + Cov(X,Z)\)