Re: StreamTokenizer

Discussion in 'Java' started by Ian deSouza, Aug 1, 2003.

  1. Ian deSouza

    Ian deSouza Guest

    Thanks Dale..

    Sorry about the delay in getting back to the group..

    I did come up with..

    void resetTokenizer()
    {
    StreamTokenizer streamTokenizer = tokenizer_.getStreamTokenizer();
    streamTokenizer.resetSyntax();
    streamTokenizer.eolIsSignificant(true);
    streamTokenizer.quoteChar('"');
    streamTokenizer.whitespaceChars(' ',' ');
    streamTokenizer.whitespaceChars('\t','\t');
    streamTokenizer.whitespaceChars(13,13); // '\r'
    streamTokenizer.ordinaryChars(35,126);
    streamTokenizer.wordChars(33,33); //skip '"','#' (34,35)
    streamTokenizer.wordChars(36,60); //skip '=' (35)
    streamTokenizer.wordChars(62,122); //skip '{'
    streamTokenizer.wordChars(124,124); //skip '}'
    streamTokenizer.wordChars(126,126);
    }


    Dale King <> wrote in message news:<>...
    > In article <>,
    > says...
    > > How do I avoid having the stream tokenizer parse digits?
    > >
    > > Example:
    > > import java.io.*;
    > > public class it
    > > {
    > > static public void main(String args[])
    > > {
    > > try {
    > > StreamTokenizer tokenizer = new StreamTokenizer(new
    > > BufferedReader(new FileReader(args[0])));
    > > String wordChars = "-0123456789&";
    > > for (int i = 0; i < wordChars.length(); i++)
    > > {
    > > int c = wordChars.charAt(i);
    > > tokenizer.wordChars(c,c);
    > > }

    >
    > That makes them both digits and word characters. You need to mark them as
    > not being numeric. You can do that by doing a resetSyntax first, but that
    > requires you set up everything yourself. Or you could set them to
    > ordinary characters before setting them to word characters.
    >
    > While '.' is not in your list of word characters it is marked as numeric
    > by default so you probably want to make it an ordinary character as well.
     
    Ian deSouza, Aug 1, 2003
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. D. Lane
    Replies:
    1
    Views:
    2,499
    Dave Rathnow
    Jun 30, 2003
  2. John C. Bollinger

    Re: How does StreamTokenizer work

    John C. Bollinger, Jun 30, 2003, in forum: Java
    Replies:
    0
    Views:
    875
    John C. Bollinger
    Jun 30, 2003
  3. Sudsy

    Re: StreamTokenizer

    Sudsy, Jul 24, 2003, in forum: Java
    Replies:
    2
    Views:
    521
    Sudsy
    Aug 2, 2003
  4. JVarsoke
    Replies:
    1
    Views:
    1,543
    Dale King
    Oct 1, 2003
  5. Eric
    Replies:
    3
    Views:
    1,151
Loading...

Share This Page