Stochastic Processes
A stochastic process \(X(t), t \in T\) is a collection of random variables. For each \(t \in T\), \(X(t)\) is a random variable. The index \(t\) is often interpreted as time, and as a result, we refer to \(X(t)\) as the state of the process at time \(t\).
The set \(T\) is called the index set of the process. When \(T\) is a countable set, the stochastic is a discrete-time process. If \(T\) is an interval of the real line, the process is said to be a continuous-time process.
Focus: Discrete time, discrete state space Markov Chain
-
Stochastic = random
-
A stochastic process describes random phenomena that change over time
-
values that \(X_t\)‘s take
-
set of all possible states, denoted by \(\mathcal{S}\).
-
can be thought of as time. If \(T = \{0, 1, 2, \dots \}\) then it is a discrete-time process. If \(T\) is an interval, it is a continuous time process.
Each \(X_t\) is a random variable.
Example of stochastic process: Gambler’s ruin
- A gambler starts with an initial fortune of \(k\) dollars.
- The gambler plays against \(B\) with an initial fortune of \(N-k\) dollars.
- Each game he bets $1, wins with probability \(p\)
- Let \(\{X_t = t = 0,1,2 \dots\}\) represent his fortune as the betting goes on.
- Game only stops when either gambler or \(B\) is ruined.
- Here \(\mathcal{S} = \{0,1,\dots,N\}\)
- For a realization of the results of the first 10 games, (here \(p=1/2\)):
sample(c(-1, 1), 10, replace=T)
Reference Textbooks
, Ross, n.d., @pinsky2010introduction