Any replacement for StringTokenizer/StreamTokenizer?

[Chris Berg]

Is there any replacement for StringTokenizer or StreamTokenizer in the
newer Java libraries? After all, they date back to Java 1.0.

I find it is difficult to fully understand and utilize
StreamTokenizer, and StringTokenizer is usually too basic to be really
competitive to parsing 'by hand' when it's a bit more complicated than
just words and whitespace.

One would expect Sun to come up with something more sophisticated and
user-friendly, as has been done in many other fields.

Chris

 

[Thomas Hawtin]

Is there any replacement for StringTokenizer or StreamTokenizer in the
newer Java libraries? After all, they date back to Java 1.0.

So does String...

java.util.regex (and Scanner) is supposed to replace them.

I find it is difficult to fully understand and utilize
StreamTokenizer, and StringTokenizer is usually too basic to be really
competitive to parsing 'by hand' when it's a bit more complicated than
just words and whitespace.

Yeah, I was looking at its use in the GTK PL&F's Metacity parser. It's
much easier to do anything non-trivial by hand.

Perhaps a full lexer should be standardised.

Tom Hawtin

 

[Roedy Green]

One would expect Sun to come up with something more sophisticated and
user-friendly, as has been done in many other fields.

There are two tools for more complex problems:

http://mindprod.com/jgloss/regex.html
http://mindprod.com/jgloss/parser.html

 

[yakovfain]

String.split() is a recommended replacement for StringTokenizer. Read
the javadoc.

Yakov Fain

 

[Chris Berg]

Hmmm... I was hoping I could somehow avoid those regular expressions.
But in vane! Regex always turns me off. I mean - whole books are
written about them (http://www.oreilly.com/catalog/regex/) - and it
was supposed to be simpler than the tokenizers ?!!? I mean, I already
have about a meter of books that I am supposed to have read.... and
then yet another one just to solve a minor parsing task?

Oh well, it was a nice try, anyway. Thanks a lot, guys... (sob) ... no
fishing trip this week-end, it seems. (now let's see... www.amazon.com
.... wheres did I put that that VISA card?)

Chris

 

[Roedy Green]

I mean, I already
have about a meter of books that I am supposed to have read.... and
then yet another one just to solve a minor parsing task?

Just learn split. See http://mindprod.com/jgloss/regex.html

It is not hard at all.