Stochastic Processes

A stochastic process $$X(t), t \in T$$ is a collection of random variables. For each $$t \in T$$, $$X(t)$$ is a random variable. The index $$t$$ is often interpreted as time, and as a result, we refer to $$X(t)$$ as the state of the process at time $$t$$.

The set $$T$$ is called the index set of the process. When $$T$$ is a countable set, the stochastic is a discrete-time process. If $$T$$ is an interval of the real line, the process is said to be a continuous-time process.

Focus: Discrete time, discrete state space Markov Chain

• Stochastic = random

• A stochastic process describes random phenomena that change over time

• values that $$X_t$$‘s take

• set of all possible states, denoted by $$\mathcal{S}$$.

• can be thought of as time. If $$T = \{0, 1, 2, \dots \}$$ then it is a discrete-time process. If $$T$$ is an interval, it is a continuous time process.

Each $$X_t$$ is a random variable.

Example of stochastic process: Gambler’s ruin

1. A gambler starts with an initial fortune of $$k$$ dollars.
2. The gambler plays against $$B$$ with an initial fortune of $$N-k$$ dollars.
3. Each game he bets \$1, wins with probability $$p$$
4. Let $$\{X_t = t = 0,1,2 \dots\}$$ represent his fortune as the betting goes on.
5. Game only stops when either gambler or $$B$$ is ruined.
6. Here $$\mathcal{S} = \{0,1,\dots,N\}$$
7. For a realization of the results of the first 10 games, (here $$p=1/2$$):
  sample(c(-1, 1), 10, replace=T)


Bibliography

Pinsky, Mark, and Samuel Karlin. n.d. An Introduction to Stochastic Modeling. Academic press.

Ross, Sheldon M. n.d. Introduction to Probability Models. Academic press.