# Random Variables

tags
Statistics

## Introduction

It frequently occurs that in performing an experiment, we are mainly interested in some function of the outcome rather than the outcome itself. These quantities of interest, are real-valued functions defined on the sample space $$\mathcal{S}$$, known as random variables.

## Discrete Random Variables

For a discrete random variable, we define the probability mass function $$p(a)$$ of $$X$$ by:

$$\label{dfn:pmf} p(a) = P(X = a)$$

The probability mass function $$p(a)$$ is positive for at most a countable number of values of $$a$$.

Since $$X$$ must take on one of the values $$x_i$$, we have $$\sum_{i=1}^{\infty} p(x_i) = 1$$.

The cumulative distribution function $$F$$ can be expressed in terms of $$p(a)$$ by:

$$\label{dfn:cdf} F(a) = \sum_{\text{all } x_i \le a} p(x_i)$$

### Bernoulli Random Variable

A random variable $$X$$ is said to be a Bernoulli random variable if its probability mass function is given by:

$$p(0) = P(X = 0) = 1 - p, p(1) = P(X = 1) = p$$

This corresponds to the outcome of a trial with binary outcomes.

### Binomial Random Variable

Suppose $$n$$ independent trials, each of which have binary outcomes and result in a success with probability $$p$$. If $$X$$ represents the number of successes that occur in the $$n$$ trials, then $$X$$ is a binomial random variable denoted by $$Bin(n, p)$$.

The pmf of a binomial random variable is given by:

$$p(i) = P(X = i) = {n \choose i}p^i (1-p)^i, i = 0,1,\dots,n$$

### Geometric Random Variable

Suppose that independent trials, each having probability $$p$$ of being a success, are performed until a success occurs. If we let $$X$$ be the number of trials required until the first success, then $$X$$ is said to be a geometric variable with parameter $$p$$.

$$p(n) = P(X = n) = (1-p)^{n-1}p$$

### Poisson Random Variable

A random variable $$X$$, taking on one of the values $$0, 1, 2, \dots$$ is said to be a poisson random variable with parameter $$\lambda$$ if for some $$\lambda > 0$$,

$$p(i) = p(X = i) = e^{-\lambda}\frac{\lambda^i}{i!}, i = 0, 1, \dots$$

An important property of the Poisson random variable is that it may be used to approximate a binomial random variable when the binomial parameter $$n$$ is large and $$p$$ is small.

## Continuous Random Variables

Continuous random variables have an uncountable set of possible values.

$$P[X \in B] = \int_{B}f(x)dx$$

The cumulative distribution $$F(\cdot)$$ and the probability density function is expressed by:

$$F(a) = P(X \in (-\infty, a]) = \int_{-\infty}^{a} f(x) dx$$

### Uniform Random Variable

$$X$$ is a uniform random variable on the interval $$(a, b)$$ if its probability density function is given by:

f(x) = \begin{cases} \frac{1}{b - a} & a < x < b \\\
0 & \text{otherwise} \end{cases}

### Exponential Random Variables

f(x) = \begin{cases} \lambda e^{-\lambda x} & x \ge 0 \\\
0 & x < 0 \end{cases}

$$F(a) = 1 - e^{-\lambda }, a \ge 0$$

### Gamma Random Variables

f(x) = \begin{cases} \frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x} & x \ge 0 \\\
0 & x < 0 \end{cases}

The quantity $$\Gamma(\alpha)$$ is called the gamma function and is defined by:

$$\Gamma(\alpha) = \int_{0}^{\infty}e^{-x}x^{\alpha-1}dx$$

and $$\Gamma(n) = (n - 1)!$$ for some integral $$\alpha = n$$.

### Normal Random Variables

$$f(x) = \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}}, -\infty < x < \infty$$

if $$X ~ N(\mu, \sigma^2)$$ and $$Y = aX + b$$, then $$Y ~ N(a\mu + b, a^2\sigma^2)$$.

## Expectation of Random Variables

The expected value of a discrete random variable $$X$$ is defined by:

$$E(X) = \sum_{x:p(x)>0} x p(x)$$

and for a continuous random variable is defined similarly:

$$E(X) = \int_{-\infty}^{\infty} x f(x) dx$$

Suppose we are interested in getting the expected value of a function of a random variable $$E(g(X))$$. One way is to obtain the distribution of $$g(X)$$, and compute $$E(g(X))$$ by definition of expectation.

$$E(g(X)) = \sum_{x:p(x)>0} g(x) p(x)$$

We also have linearity of expectations:

$$E(aX + b) = aE(X) + b$$

$$E(X)$$ is the first moment of $$X$$, and $$E(X^n)$$ is the nth moment of $$X$$.

The variance of $$X$$, $$Var(X)$$ measures the expected square of the deviation of $$X$$ from its expected value:

$$Var(X) = E((X-E(X))^2) = E(X^2) - (E(X))^2$$

## Jointly Distributed Random Variables

The joint cumulative probability distribution function of $$X$$ and $$Y$$ is given by:

$$F(a, b) = P(X \le a, Y \le b), -\infty < a , b < \infty$$

The distribution of $$X$$ can then be obtained from the joint distribution of $$X$$ and $$Y$$ as follows:

\begin{align} F_X(a) &= P(X \le a, Y \le \infty) \\\
&= F(a, \infty) \end{align}

## Covariance and Variance of Sums of Random Variables

The covariance of any 2 random variables $$X$$ and $$Y$$, denoted by $$Cov(X,Y)$$, is defined by:

\begin{align} Cov(X,Y) &= E\left[ (X - E[X])(Y - E[Y]) \right] \\\
&= E[XY] - E[X]E[Y] \end{align}

If $$X$$ and $$Y$$ are independent, then $$Cov(X,Y) = 0$$.

Some properties of covariance:

1. $$Cov(X,X) = Var(X)$$
2. $$Cov(X,Y) = Cov(Y,X)$$
3. $$Cov(cX, Y) = c Cov(X,Y)$$
4. $$Cov(X, Y+Z) = Cov(X,Y) + Cov(X,Z)$$