Data Mining - Maximum Entropy Algorithm

Thomas Bayes

About

Maximum Entropy (MaxEnt) models are feature-based classifier models.

In a two-class scenario, it is the same as using logistic regression to find a distribution over the classes.

Feature overlap

MaxEnt makes no independence assumptions for its features, unlike Naive Bayes. This means that features like bigrams and phrases can be added to MaxEnt without worrying about features overlapping.

??? Theoretically, MaxEnt performs better than Naive Bayes because it handles feature overlap better. However, in practice, Naive Bayes can still perform well on a variety of problems.

Documentation / Reference

  • Stanford Classifier is a MaxEnt classifier. The Stanford Classifier shines is in working with mainly textual data. For small data sets and numeric predictors, you'd generally be better off using another tool such as R or weka.





Discover More
Classification
Data Mining - (Classifier|Classification Function)

A classifier is a Supervised function (machine learning tool) where the learned (target) attribute is categorical (“nominal”) in order to classify. It is used after the learning process to classify...
Data Mining Algorithm
Data Mining - Algorithms

An is a mathematical procedure for solving a specific kind of problem. For some data mining functions, you can choose among several algorithms. Algorithm Function Type Description Decision...



Share this page:
Follow us:
Task Runner