Exponential Family
- tags
- Statistics
A one-parameter exponential family model is any model whose density can be expressed as:
\begin{equation} p(y | \theta)=h(y) g(\theta) \exp \{\eta(\theta) t(y)\} \end{equation}
where \(\theta\) is the parameter of the family, and \(t(y)\) is the sufficient statistic for \(\theta\).
When a model belongs to the one-parameter exponential family, a family of conjugate prior distributions is given by:
\begin{equation} p(\theta) \propto g(\theta)^{\nu} \exp \{\eta(\theta) \tau\} \end{equation}
where \(\nu\) and \(\tau\) are parameters of the prior, such that \(p(\theta)\) is a well-defined pdf.
Combining this prior with a sampling model \(Y \sim p(y|\theta)\) yields the posterior:
\begin{align} p(\theta | y) & \propto p(y | \theta) p(\theta) \ & \propto g(\theta) \exp \{\eta(\theta) t(y)\} \cdot g(\theta)^{\nu} \exp \{\eta(\theta) \tau\} \ & \propto g(\theta)^{\nu+1} \exp \{\eta(\theta)[\tau+t(y)]\} \end{align}
which belongs to the same family as the prior distribution, with parameters \(\nu + 1\) and \(\tau + t(y)\).