Jethro's Braindump

Gibbs Sampling

Gibbs sampling is a special case of the Metropolis-Hastings method, where a sequence of proposal distributions \(q\) is defined in terms of the conditional distributions of the joint distribution \(p(\mathbf{x})\), and proposals are always accepted.

In the general case of a system with \(K\) variables, a single iteration involves sampling one parameter at a time:

\begin{equation} \begin{array}{l}{x_{1}^{(t+1)} \sim P\left(x_{1} | x_{2}^{(t)}, x_{3}^{(t)}, \ldots, x_{K}^{(t)}\right)} \ {x_{2}^{(t+1)} \sim P\left(x_{2} | x_{1}^{(t+1)}, x_{3}^{(t)}, \ldots, x_{K}^{(t)}\right)} \ {x_{3}^{(t+1)} \sim P\left(x_{3} | x_{1}^{(t+1)}, x_{2}^{(t+1)}, \ldots, x_{K}^{(t)}\right), \text { etc. }}\end{array} \end{equation}

Pros and Cons

  1. Suffers the same defects as Metropolis-Hastings methods
  2. No adjustable parameters, so it’s easy to start with