Reference Prior

The notion of reference prior is similar to that of a non-informative prior, but there are subtle differences. The uniform prior is arguably non-informative, but is it not a good reference, because it is not always invariant under reparamaterization.

Consider a reparameterization $γ = \log θ$ , converting the support of the parameter to the real line. The prior on $γ$ is given by:

$p_{γ} (γ) = p (θ) | J | = | J |$

where $| J | = \frac{d θ}{d γ}$ from the Change of Variables Theorem.

$p_{γ} (γ) = e^{γ}, - \infty < γ < + \infty$

which is clearly not uniform. A prior that is invariant to transformation, and easy to compute, is the Jeffreys Prior.