Stochastic Processes

A stochastic process $X (t), t \in T$ is a collection of random variables. For each $t \in T$ , $X (t)$ is a random variable. The index $t$ is often interpreted as time, and as a result, we refer to $X (t)$ as the state of the process at time $t$ .

The set $T$ is called the index set of the process. When $T$ is a countable set, the stochastic is a discrete-time process. If $T$ is an interval of the real line, the process is said to be a continuous-time process.

Focus: Discrete time, discrete state space Markov Chain

Stochastic = random
A stochastic process describes random phenomena that change over time
values that $X_{t}$ ‘s take
set of all possible states, denoted by $S$ .
can be thought of as time. If $T = {0, 1, 2, \dots}$ then it is a discrete-time process. If $T$ is an interval, it is a continuous time process.

Each $X_{t}$ is a random variable.

Example of stochastic process: Gambler’s ruin

A gambler starts with an initial fortune of $k$ dollars.
The gambler plays against $B$ with an initial fortune of $N - k$ dollars.
Each game he bets $1, wins with probability $p$
Let ${X_{t} = t = 0, 1, 2 \dots}$ represent his fortune as the betting goes on.
Game only stops when either gambler or $B$ is ruined.
Here $S = {0, 1, \dots, N}$
For a realization of the results of the first 10 games, (here $p = 1 / 2$ ):

  sample(c(-1, 1), 10, replace=T)

Reference Textbooks

, Ross, n.d., @pinsky2010introduction

Jethro's Braindump

Stochastic Processes

Reference Textbooks

Links to this note