Laplace's Method

Suppose we have an unnormalized probability density \(P^\star(x)\), whose normalizing constant:

\begin{equation} Z_P \equiv \int P^\star(x) dx \end{equation}

is of interest, and has a peak at point \(x_0\).

We perform a Taylor expansion of \(\ln P^\star(x)\) at this peak:

\begin{equation} \ln P^\star(x) \approxeq \ln P^\star(x_0) - \frac{c}{2}(x - x_0)^2 + \dots \end{equation}

where \(c = - \frac{\partial^2}{\partial x^2} \ln P^\star(x) \text{ where } {x = x_0}\).

\(P^\star(x)\) can be approximated by an unnormalized Gaussian:

\begin{equation} Q^\star(x) \equiv P^\star(x_0) \textrm{exp}\left[- \frac{c}{2} (x-x_0)^2\right] \end{equation}

and the normalizing constant is approximated with:

\begin{equation} Z_Q \equiv P^\star(x_0) \sqrt{\frac{2\pi}{c}} \end{equation}

This is easily generalizable to a K-dimensional space \(\mathbf{x}\).