Class StringTokenizer

java.lang.Object
java.util.StringTokenizer
All Implemented Interfaces:
Enumeration<Object>
public class StringTokenizer extends Object implements Enumeration<Object>
The StringTokenizer class allows an application to break a string into tokens by performing code point comparison. The StringTokenizer methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.

The set of delimiters (the codepoints that separate tokens) may be specified either at creation time or on a per-token basis.

An instance of StringTokenizer behaves in one of three ways, depending on whether it was created with the returnDelimiters flag having the value true or false:

  • If returnDelims is false, delimiter code points serve to separate tokens. A token is a maximal sequence of consecutive code points that are not delimiters.
  • If returnDelims is true, delimiter code points are themselves considered to be tokens. In this case a token will be received for each delimiter code point.

A token is thus either one delimiter code point, or a maximal sequence of consecutive code points that are not delimiters.

A StringTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the code point processed.

A token is returned by taking a substring of the string that was used to create the StringTokenizer object.

Here's an example of the use of the default delimiter StringTokenizer : 

StringTokenizer st = new StringTokenizer("this is a test");
while (st.hasMoreTokens()) {
    println(st.nextToken());
}

This prints the following output: 

    this
    is
    a
    test

Here's an example of how to use a StringTokenizer with a user specified delimiter: 

StringTokenizer st = new StringTokenizer(
        "this is a test with supplementary characters ?𐀀?",
        " 𐀀");
while (st.hasMoreTokens()) {
    println(st.nextToken());
}

This prints the following output: 

    this
    is
    a
    test
    with
    supplementary
    characters
    ?
    ?

  • Constructor Summary

    Constructors
    Constructor
    Description
    StringTokenizer(String string)
    Constructs a new StringTokenizer for the parameter string using whitespace as the delimiter.
    StringTokenizer(String string, String delimiters)
    Constructs a new StringTokenizer for the parameter string using the specified delimiters.
    StringTokenizer(String string, String delimiters, boolean returnDelimiters)
    Constructs a new StringTokenizer for the parameter string using the specified delimiters, returning the delimiters as tokens if the parameter returnDelimiters is true.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    countTokens()
    Returns the number of unprocessed tokens remaining in the string.
    boolean
    hasMoreElements()
    Returns true if unprocessed tokens remain.
    boolean
    hasMoreTokens()
    Returns true if unprocessed tokens remain.
    Object
    nextElement()
    Returns the next token in the string as an Object.
    String
    nextToken()
    Returns the next token in the string as a String.
    String
    nextToken(String delims)
    Returns the next token in the string as a String.

    Methods inherited from class Object

    clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • StringTokenizer

      public StringTokenizer(String string)
      Constructs a new StringTokenizer for the parameter string using whitespace as the delimiter. The returnDelimiters flag is set to false.
      Parameters:
      string - the string to be tokenized.

    • StringTokenizer

      public StringTokenizer(String string, String delimiters)
      Constructs a new StringTokenizer for the parameter string using the specified delimiters. The returnDelimiters flag is set to false. If delimiters is null, this constructor doesn't throw an Exception, but later calls to some methods might throw a NullPointerException.
      Parameters:
      string - the string to be tokenized.
      delimiters - the delimiters to use.

    • StringTokenizer

      public StringTokenizer(String string, String delimiters, boolean returnDelimiters)
      Constructs a new StringTokenizer for the parameter string using the specified delimiters, returning the delimiters as tokens if the parameter returnDelimiters is true. If delimiters is null this constructor doesn't throw an Exception, but later calls to some methods might throw a NullPointerException.
      Parameters:
      string - the string to be tokenized.
      delimiters - the delimiters to use.
      returnDelimiters - true to return each delimiter as a token.

  • Method Details

    • countTokens

      public int countTokens()
      Returns the number of unprocessed tokens remaining in the string.
      Returns:
      number of tokens that can be retreived before an Exception will result from a call to nextToken().

    • hasMoreElements

      public boolean hasMoreElements()
      Returns true if unprocessed tokens remain. This method is implemented in order to satisfy the Enumeration interface.
      Specified by:
      hasMoreElements in interface Enumeration<Object>
      Returns:
      true if unprocessed tokens remain.

    • hasMoreTokens

      public boolean hasMoreTokens()
      Returns true if unprocessed tokens remain.
      Returns:
      true if unprocessed tokens remain.

    • nextElement

      public Object nextElement()
      Returns the next token in the string as an Object. This method is implemented in order to satisfy the Enumeration interface.
      Specified by:
      nextElement in interface Enumeration<Object>
      Returns:
      next token in the string as an Object
      Throws:
      NoSuchElementException - if no tokens remain.

    • nextToken

      public String nextToken()
      Returns the next token in the string as a String.
      Returns:
      next token in the string as a String.
      Throws:
      NoSuchElementException - if no tokens remain.

    • nextToken

      public String nextToken(String delims)
      Returns the next token in the string as a String. The delimiters used are changed to the specified delimiters.
      Parameters:
      delims - the new delimiters to use.
      Returns:
      next token in the string as a String.
      Throws:
      NoSuchElementException - if no tokens remain.