From TheBestLinks.com
In statistics, the exponential family of probability density functions or probability mass functions comprises those that have the following form:
- <math>f(x|\eta) = h(x) e^{\eta^{\top} T(x) - A(\eta)}<math>
where:
- h(x) is the reference density,
- η is the natural parameter, a column vector, so that ηT, its transpose, is a row vector,
- T(x) is called the sufficient statistic, a column vector whose number of scalar components is the same as that of η. (However, the concept of sufficient statistic is broader than what may appear from this article.)
The parameter space -- i.e., the set of values of η for which this function is integrable -- is necessarily convex.
The term exponential family is also frequently used to refer to any particular concrete case, i.e., any parametrized family of probability distributions of this form.
The Bernoulli, normal, gamma, Poisson and binomial distributions are all exponential families.
According to the Pitman-Koopman-Darmois theorem, only in exponential families is there a sufficient statistic whose dimension remains bounded as sample size increases. More long-windedly, suppose Xn, n = 1, 2, 3, ... are independent identically distributed random variables whose distribution is known to be in some family of probability distributions. Only if that family is an exponential family is there a (possibly vector-valued) sufficient statistic T(X1, ..., Xn) whose number of scalar components does not increase as the sample size n increases.
Exponential families are also important in Bayesian statistics. In Bayesian statistics a prior distribution is multiplied by a likelihood function and then normalised to produce a posterior distribution. In the case of a likelihood which is an exponential family there exists a conjugate prior. A conjugate prior is one which, when combined with the likelihood and normalised, produces a posterior distribution which is of the same type as the prior. For example, if one is estimating the parameter theta, the success probability, of a binomial distribution then if one chooses to use a beta distribution as one's prior, then the posterior is always another beta distribution. This makes the computation of the posterior particularly simple. Similarly, if one is estimating the lambda parameter of a Poisson distribution the use of a gamma prior will lead to another gamma posterior. Conjugate priors are often very flexible and can be very convenient. However, if one's belief about the likely value of the theta parameter of a binomial is represented by (say) a bimodal (two-humped) distribution then this cannot be represented by a beta distribution. In general, a likelihood will not belong to an exponential family and thus no conjugate prior exists. The posterior will then have to be computed by numerical methods. Thus, classical frequentist hypothesis testing is seriously impeded in the case of likelihoods which are not exponential families, because of the lack of sufficient statistics. By contrast, Bayesian inference is based on the posterior distribution and can still be carried out if the requisite numerical integrals can be performed either directly or (more usually) by simulation.
Related links
Top visited
0 of
0 links
[no links posted yet]
>> place link >>
Discussion
Last posted
0 of
0 messages
[no messages posted yet]
>> post message >>
Watch
You can
add this article to your own "watchlist" and receive e-mail notification about all changes in this page.
