Statistics - Akaike information criterion (AIC)
About
AIC stands for Akaike Information Criterion.
Akaike is the name of the guy who came up with this idea.
AIC is a quantity that we can calculate for many different model types, not just linear models, but also classification model such logistic regression and so on.
Articles Related
Definition
The AIC criterion is defi ned for a large class of models fi t by maximum likelihood:
<MATH> AIC = -2 log L + 2 . d </MATH>
where:
- L is the maximized value of the likelihood function for the estimated model.
- d is the total # of parameters used in the model (regression coefficients + intercept)
Linear model
It turns out that in the case of a linear model with Gaussian errors, negative 2 log L is just equal to RSS over <math>\hat{\sigma}</math> squared
<MATH> 2 log L = \frac{\href{RSS}{RSS}}{\href{variance}{\hat{\sigma}}^2} </MATH>
where:
- <math>\hat{\sigma}^2</math> is the variance estimate of the error associated with each response measurement (ie each error epsilon in the linear model)
Then by plugging it in the aboe formula, we can see that AIC and Mallow's Cp are actually proportional to each other. They are the same thing for linear models.