What is a Pattern ?

Thomas Bayes

About

A pattern means that the data (visual or not) are correlated that they have a relationship and that they are predictable.

When you have a lack of pattern, you have true randomness

When you find a pattern, you can have a good idea when or where something will happen before it actually happens.

See Data Mining - Signal (Wanted Variation)

Pattern detection is a goal of unsupervised learning

Beware of the human tendency bias to see patterns in random data.

See:

  • wiki/Apophenia: tendency to perceive meaningful connections between unrelated things
  • wiki/Pareidolia: tendency to interpret a vague stimulus as something known to the observer: seeing shapes in clouds, faces in inanimate objects, …

Example

Validation of the pattern

Further, the discovery of a particular pattern in a particular set of data does not necessarily mean that pattern is representative of the whole population from which that data was drawn. Hence, an important part of the process is the verification and validation of patterns on other samples of data.

…the curse of big data is the fact that when you search for patterns in very, very large data sets with billions or trillions of data points and thousands of metrics, you are bound to identify coincidences that have no predictive power.”

Documentation / Reference





Discover More
Weapons Of Mass Creation
(Innovation|Creativity|Genius)

(Innovation|Creativity|Genius) 0393240835The Language of Food by Dan JurafskycreativefocusB00E257T6CThe Design of Everyday Things:...
Rating Collaborative Filtering
(Prediction|Recommender System) - Collaborative filtering

Collaborative filtering is a method of making automatic predictions (filtering) the interests of a user by collecting preferences or taste information from many users (collaborating). But in general,...
Division
Data Mining - (Descriptive|Discovery) (Analysis|Statistics)

Descriptive analysis is also known as Descriptive statistics They are procedures used to summarize, organize, and simplify data. Descriptive function are always unsupervised See also . Visual...
Feature Extraction
Data Mining - (Feature|Attribute) Extraction Function

Feature extraction is the second class of methods for dimension reduction. dimension reduction It creates new attributes (features) using linear combinations of the (original|existing) attributes. ...
P Value Pipeline
Data Mining - (Life cycle|Project|Data Pipeline)

Data mining is an experimental science. Data mining reveals correlation, not causation. With good data, you will make good algorithm. The most preferable solution is then to work on good features....
Thomas Bayes
Data Mining - (Prediction|Guess)

Something predictable is showing a pattern and is therefore not truly random. entropytrue randomness Many forms of data mining model are predictive. For example, a model might predict income based on...
Thomas Bayes
Data Mining - Data Mining - (Data|Knowledge) Discovery - Statistical Learning

Data Mining can be defined as the automatic or semiautomatic task of extracting previously unknown information from a large quantity of data. Data mining try to discover in data unknown: unexpected...
Thomas Bayes
Data Mining - Entropy (Information Gain)

The degree to which a system has no pattern is known as entropy. A high-entropy source is completely chaotic, is unpredictable, and is called true randomness. Entropy is a function “Information”...
Thomas Bayes
Data Mining - Intrusion detection systems (IDS) / Intrusion Prevention / Misuse

Classical security mechanisms, i.e. authentication and encryption, and infrastructure components like firewalls cannot provide perfect security. Therefore, intrusion detection systems (IDS) have been...
Thomas Bayes
Data Mining - Result Considerations

Before tackling a data mining problem, some considerations must be take into account in order to get good interpretations of the results. Strong correlations of data do not necessarily prove a cause-and-effect...



Share this page:
Follow us:
Task Runner