Jethro's Braindump

Exponential Family

tags
Statistics

A one-parameter exponential family model is any model whose density can be expressed as:

p(y|θ)=h(y)g(θ)exp{η(θ)t(y)}

where θ is the parameter of the family, and t(y) is the sufficient statistic for θ.

When a model belongs to the one-parameter exponential family, a family of conjugate prior distributions is given by:

p(θ)g(θ)νexp{η(θ)τ}

where ν and τ are parameters of the prior, such that p(θ) is a well-defined pdf.

Combining this prior with a sampling model Yp(y|θ) yields the posterior:

p(θ|y)p(y|θ)p(θ) g(θ)exp{η(θ)t(y)}g(θ)νexp{η(θ)τ} g(θ)ν+1exp{η(θ)[τ+t(y)]}

which belongs to the same family as the prior distribution, with parameters ν+1 and τ+t(y).

Links to this note