StringTokenizer Question?

J

Jim Crowell

I have a method to unpack a String via a given String delimiter. It works
fine but it used the 'String' Class 'indexOf' and 'substring' Methods within
a 'for' loop.

To make this method more efficient I tried to substitute the
'StringTokenizer' Class. Easy enough but it looks like the following input
String returns 3 token where I would expect 4 tokens:

"AA,BB,,1," with delimiter = ","

Also, the following String returns 3 Tokens where I would expect 4:

",BB,CC,1," with delimiter = ","

With my code I use an 'if' condition before I unpack to ensure that the
string ends with the delimiter but for 'StringTokenizer' I see that the
following input String returns 4 tokens:

"AA,BB,CC,1" with delimiter = ","

I assume that a token must have 1 or more characters and that I must return
to my original code.

Is there a way to make 'StringTokenizer' recognize consecutive delimiters
and return a null String [e.g. ""] as a token?

Thanks

Jim.
 
A

Andy Fish

I can't see any way to do it with StringTokenizer. However, you could use
String.split()
 
T

Tor Iver Wilhelmsen

Jim Crowell said:
To make this method more efficient I tried to substitute the
'StringTokenizer' Class. Easy enough but it looks like the following input
String returns 3 token where I would expect 4 tokens:

"AA,BB,,1," with delimiter = ","

Thet's because you haven't told it to return "empty" tokens - check
the constructors.
Also, the following String returns 3 Tokens where I would expect 4:

",BB,CC,1," with delimiter = ","

Same thing, except you should expect five. :)
 
R

Roedy Green

Also, the following String returns 3 Tokens where I would expect 4:

StringTokenizer(String str, String delim, boolean returnDelims)

Try setting the last parameter of the constructor to true.
 
S

samspade

I have a method to unpack a String via a given String delimiter. It
works fine but it used the 'String' Class 'indexOf' and 'substring'
Methods within a 'for' loop.
To make this method more efficient I tried to substitute the
'StringTokenizer' Class. Easy enough but it looks like the following
input
String returns 3 token where I would expect 4 tokens:
"AA,BB,,1," with delimiter = ","
Also, the following String returns 3 Tokens where I would expect 4:
",BB,CC,1," with delimiter = ","
With my code I use an 'if' condition before I unpack to ensure that
the string ends with the delimiter but for 'StringTokenizer' I see
that the following input String returns 4 tokens:
"AA,BB,CC,1" with delimiter = ","
I assume that a token must have 1 or more characters and that I must
return to my original code.
Is there a way to make 'StringTokenizer' recognize consecutive
delimiters and return a null String [e.g. ""] as a token?

If you use string.split(regex) then it will return the empty matches for
you. I.e, you'd get 4 from your first example.

StringTokenizer is (almost) deprecated now for most uses.
 
J

John C. Bollinger

samspade said:
If you use string.split(regex) then it will return the empty matches for
you. I.e, you'd get 4 from your first example.

StringTokenizer is (almost) deprecated now for most uses.

Really? That's news to me. I am perfectly comfortable with Java 1.4
regexes, but I prefer StringTokenizer for those tasks to which it is
suited. In practice, StringTokenizer is suited to a substantial number
of the most frequently occurring String parsing tasks. Regexes are more
powerful, but also heavier, and their API is more complex.


John Bollinger
(e-mail address removed)
 
J

Jim Crowell

Roedy,
It might make it terser, but it won't make it more efficient.
StringTokenizer is generic code while, your IndexOf tokenizer is tuned
to your particular problem. You should be able to whip the pants off
StringTokenizer unless you did something stupid.

Thanks. I'll stick with my original code at least until I upgrade to
1.4 and learn more about Regex.

Jim
 
J

Jim Crowell

Jim Crowell said:
Is there a way to make 'StringTokenizer' recognize consecutive delimiters
and return a null String [e.g. ""] as a token?

Thanks for the many responses. In my original post I neglected to
state that I am still using Java 1.3.1.

Therefore, I believe I can not use String.split(Regex) yet.

To be honest, I did not know about 'Regex' until I received your
responses. I have since read a tutorial and look forward to using it
somewhere in my App.

I am avoiding using Java 1.4.1 because whenever I go to a new upgrade
I have had to make many changes to my many existing code and I can not
afford the time right now.

This is most likely do to poor programming practices on my part since
I think that a new version should be upword compatable.

Thanks again,
Jim
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top