Jethro's Braindump

Reference Prior

The notion of reference prior is similar to that of a non-informative prior, but there are subtle differences. The uniform prior is arguably non-informative, but is it not a good reference, because it is not always invariant under reparamaterization.

Consider a reparameterization \(\gamma = \log \theta\), converting the support of the parameter to the real line. The prior on \(\gamma\) is given by:

\begin{equation} p_\gamma (\gamma) = p(\theta)|J| = |J| \end{equation}

where \(|J| = \frac{d \theta}{d \gamma}\) from the Change of Variables Theorem.

\begin{equation} p_\gamma (\gamma) = e^\gamma, - \infty < \gamma < + \infty \end{equation}

which is clearly not uniform. A prior that is invariant to transformation, and easy to compute, is the Jeffreys Prior.