About
Stacking is a ensemble of models combined sequentially.
Stacking uses a “meta learner” (not voting) to combine the predictions of “base learners.”
The base learners (the expert) are not combined by voting but by using a meta-learner, another learner scheme that combines the output of the base learners.
For example create a linear model and then as a second step use random forests (residual errors as a response).
- Base learners: level-0 model
- Meta Learner: level-1 model
The predictions of the base learners are input for the meta-learner.
Typically, different machines learning schemes are used as based-learners to get different perspective.
It can’t use predictions on training data to generate data for level‐1 model (Instead use cross‐validation‐like scheme)
Articles Related
Implementation
Weka
Stacking C is the more efficient version (allow multiple level‐0 models by specifying a metaclassifier)