Gibbs Sampling

Gibbs sampling is a special case of the Metropolis-Hastings method, where a sequence of proposal distributions $q$ is defined in terms of the conditional distributions of the joint distribution $p (x)$ , and proposals are always accepted.

In the general case of a system with $K$ variables, a single iteration involves sampling one parameter at a time:

$\begin{array}{l} x_{1}^{(t + 1)} \sim P (x_{1} | x_{2}^{(t)}, x_{3}^{(t)}, \dots, x_{K}^{(t)}) x_{2}^{(t + 1)} \sim P (x_{2} | x_{1}^{(t + 1)}, x_{3}^{(t)}, \dots, x_{K}^{(t)}) x_{3}^{(t + 1)} \sim P (x_{3} | x_{1}^{(t + 1)}, x_{2}^{(t + 1)}, \dots, x_{K}^{(t)}), etc. \end{array}$

Pros and Cons

Suffers the same defects as Metropolis-Hastings methods
No adjustable parameters, so it’s easy to start with

Jethro's Braindump

Gibbs Sampling

Pros and Cons

Links to this note