Java - Token / Parser / Lexer

> Procedural Languages > Java

1 - About

3 - API

3.1 - Scanner

Simple scanner that can parse primitive types and strings using regular expressions.

Scanner:

  • parsing primitive data type
  • very flexible
  • but don't return an array of strings.
String input = "1 fish 2 fish red fish blue fish";
Scanner s = new Scanner(input).useDelimiter("\\s*fish\\s*");
System.out.println(s.nextInt());
System.out.println(s.nextInt());
System.out.println(s.next());
System.out.println(s.next());
s.close();
Advertising

3.2 - String and Pattern

String.split() and Pattern.split().

You can't change the delimiter halfway through depending on a particular token.

String[] result = "this is a test".split("\\s");
for (int x=0; x<result.length; x++)
   System.out.println(result[x]);

3.3 - StringTokenizer

StringTokenizer is essentially designed for pulling out tokens delimited by fixed substrings.

StringTokenizer st = new StringTokenizer("this is a test");
	while (st.hasMoreTokens()) {
	    System.out.println(st.nextToken());
	}

4 - Utility

lang/java/token.txt · Last modified: 2017/09/06 09:15 by gerardnico