Statistics - (Multiclass Logistic|multinomial) Regression

1 - About

Multiclass logistic regression is also referred to as multinomial regression.

Multinomial Naive Bayes is designed for text classification. It's a lot faster than plain Naive Bayes.

also known as maximum entropy classifiers ?

3 - Model

The symmetric form:

<MATH> \begin{array}{rrrl} Pr(Y = k|X) & = & \frac{\displaystyle e^{\displaystyle B_{0k} + B_{1k} . X_1 + \dots + B_{ik} . X_i }}{\displaystyle \sum^K_{l=1} e^{\displaystyle B_{0l} + B_{1l} . X_1 + \dots + B_{il} . X_i }} \\ \end{array} </MATH>

  • k is the index of a outcome class
  • K is the number of outcome classes (ie bigger than 2)
  • in the numerator we've got an exponential to the linear model. This is for the probability that Y is k given X, a small k.
  • In the denominator, we've just got the sum of those exponentials for all the classes. In this case, each class gets its own linear model.
  • And then we just weigh them against each other with this exponential function, sometimes called the softmax function.
  • some cancellation is possible,
  • only K - 1 linear functions are needed as in 2-class logistic regression.

4 - R

package glmnet

data_mining/multiclass_logistic_regression.txt · Last modified: 2014/09/10 22:22 by gerardnico