Model Building - ReSampling Validation

About

Resampling method are a class of methods that estimate the test error by holding out a subset of the training set from the fitting process, and then applying the statistical learning method to those held out observations.

For supervised Learning problems, separate data sets are required for building (training) and testing the predictive models.

The training error is too optimistic. The more we fit to the data, the lower the training error but the test error can get higher if we over fit and it often will. For this reaons, models are fitted on part of the data, and then evaluated against on a holdout set (Test set).

Generally, the Build Activity splits the data into two mutually exclusive subsets:

• the training set for building the model: Evaluate the model on the training set. Performance estimates obtained on the training set are overly optimistic because of overfitting.
• the test set for testing the model. Evaluate the model on a separate test set

However, if the data is already split into Build and Test subsets, you can run a Build activity and specify the Split step.

Of course, the build data (training data) and test data must have the same column structure.

Articles Related