Data Mining - Decision boundary Visualization
1 - About
In Weka, the visualization is restricted to numeric attributes, and 2D plots
2 - Articles Related
3 - Example
- Logistic Regression method produces linear boundary with a gradual transition from one color to another. Logistic regression is a sophisticated way of choosing a linear decision boundary for classification.
- Support Vector Machine method: The resulting plot have no areas of pure color
- Random Forest method, The boundary shapes has a checkered pattern with slightly fuzzy boundaries
|Logistic Regression||Strictly Linear|
|support vector machine||piecewise linear|
|Decision tree||definitely non-linear|
knn decision boundary in any localized region of instance space is linear, determined by the nearest neighbors of the various classes in that region. But the neighbors change when you move around instance space, so the boundary is a set of linear segments that join together.
Support vector machines also produce piecewise linear boundaries.
Logistic regression is a sophisticated way of producing a good linear decision boundary, which is necessarily simple and therefore less likely to overfit.
The Logistic classifier (and also meta.ClassificationViaRegression) calculates a linear decision boundary.
The boosting algorithm AdaBoostM1 has a checkered pattern with crisp boundaries