Natural Language - Token (Word|Term)

> (Natural|Human) Language - Text (Mining|Analytics)

1 - About

In Natural Language processing, Tokens can be things like:

  • words,
  • numbers,
  • acronyms,
  • word-roots
  • or fixed-length character strings.

A token is the result of parsing (tokenization) the document down to the atomic elements generally of a language.

The token are then searchable.

See:

Advertising

3 - Management

3.1 - Create

3.2 - Tokenization

3.3 - Stemming

3.4 - Visualize

natural_language/token.txt · Last modified: 2019/06/12 19:51 by gerardnico