Statistics - Generalized Linear Models (GLM) - Extensions of the Linear Model
1 - About
The Generalized Linear Model allows for lots of different, non-linear models to be tested in the context of regression.
GLM is the mathematical framework used in many statistical analyses such as:
- and/or Regression
- and linear regression for continuous targets.
Confidence bounds are supported with a
- GLM classification for prediction probabilities.
- GLM regression for predictions.
2 - Articles Related
3 - Assumptions
The General Linear model has two main characteristics:
- Linear: linear relationships between the predictors and the outcome measure.
- Additive: the effects of each predictor are additive with one another
That doesn't mean that the GLM can't handle non-additive or non-linear effects.
Removing the additive assumption:
- interactions and
GLM can accommodate such non-additive or non-linear effects with:
- Transformation of variables: in order to make them linear
- Adding interaction terms or moderation terms: in order to do a moderation analysis and test for non-additive facts.
4 - Methods
Methods that expand the scope of linear models and how they are fit:
- Non-linearity: kernel smoothing, splines and generalized additive models; nearest neighbour methods.
- Interactions: Tree-based methods, bagging, random forests and boosting (these also capture non-linearities)
- Regularized fitting: Ridge regression and lasso. These have become very popular lately, especially when we have data sets where we have very large numbers of variables–so-called wide data sets, and even linear models are too rich for them, and so we need to use methods to control the variability.