StringTokenizer Question?

Discussion in 'Java' started by Jim Crowell, Aug 20, 2003.

  1. Jim Crowell

    Jim Crowell Guest

    I have a method to unpack a String via a given String delimiter. It works
    fine but it used the 'String' Class 'indexOf' and 'substring' Methods within
    a 'for' loop.

    To make this method more efficient I tried to substitute the
    'StringTokenizer' Class. Easy enough but it looks like the following input
    String returns 3 token where I would expect 4 tokens:

    "AA,BB,,1," with delimiter = ","

    Also, the following String returns 3 Tokens where I would expect 4:

    ",BB,CC,1," with delimiter = ","

    With my code I use an 'if' condition before I unpack to ensure that the
    string ends with the delimiter but for 'StringTokenizer' I see that the
    following input String returns 4 tokens:

    "AA,BB,CC,1" with delimiter = ","

    I assume that a token must have 1 or more characters and that I must return
    to my original code.

    Is there a way to make 'StringTokenizer' recognize consecutive delimiters
    and return a null String [e.g. ""] as a token?

    Thanks

    Jim.
     
    Jim Crowell, Aug 20, 2003
    #1
    1. Advertising

  2. Jim Crowell

    Andy Fish Guest

    I can't see any way to do it with StringTokenizer. However, you could use
    String.split()

    "Jim Crowell" <> wrote in message
    news:4wM0b.106464$...
    > I have a method to unpack a String via a given String delimiter. It works
    > fine but it used the 'String' Class 'indexOf' and 'substring' Methods

    within
    > a 'for' loop.
    >
    > To make this method more efficient I tried to substitute the
    > 'StringTokenizer' Class. Easy enough but it looks like the following input
    > String returns 3 token where I would expect 4 tokens:
    >
    > "AA,BB,,1," with delimiter = ","
    >
    > Also, the following String returns 3 Tokens where I would expect 4:
    >
    > ",BB,CC,1," with delimiter = ","
    >
    > With my code I use an 'if' condition before I unpack to ensure that the
    > string ends with the delimiter but for 'StringTokenizer' I see that the
    > following input String returns 4 tokens:
    >
    > "AA,BB,CC,1" with delimiter = ","
    >
    > I assume that a token must have 1 or more characters and that I must

    return
    > to my original code.
    >
    > Is there a way to make 'StringTokenizer' recognize consecutive delimiters
    > and return a null String [e.g. ""] as a token?
    >
    > Thanks
    >
    > Jim.
    >
    >
     
    Andy Fish, Aug 20, 2003
    #2
    1. Advertising

  3. "Jim Crowell" <> writes:

    > To make this method more efficient I tried to substitute the
    > 'StringTokenizer' Class. Easy enough but it looks like the following input
    > String returns 3 token where I would expect 4 tokens:
    >
    > "AA,BB,,1," with delimiter = ","


    Thet's because you haven't told it to return "empty" tokens - check
    the constructors.

    > Also, the following String returns 3 Tokens where I would expect 4:
    >
    > ",BB,CC,1," with delimiter = ","


    Same thing, except you should expect five. :)
     
    Tor Iver Wilhelmsen, Aug 20, 2003
    #3
  4. Jim Crowell

    Roedy Green Guest

    On Wed, 20 Aug 2003 15:32:16 GMT, "Jim Crowell"
    <> wrote or quoted :

    >Also, the following String returns 3 Tokens where I would expect 4:


    StringTokenizer(String str, String delim, boolean returnDelims)

    Try setting the last parameter of the constructor to true.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Aug 20, 2003
    #4
  5. Jim Crowell

    samspade Guest


    > I have a method to unpack a String via a given String delimiter. It
    > works fine but it used the 'String' Class 'indexOf' and 'substring'
    > Methods within a 'for' loop.


    > To make this method more efficient I tried to substitute the
    > 'StringTokenizer' Class. Easy enough but it looks like the following
    > input
    > String returns 3 token where I would expect 4 tokens:


    > "AA,BB,,1," with delimiter = ","


    > Also, the following String returns 3 Tokens where I would expect 4:


    > ",BB,CC,1," with delimiter = ","


    > With my code I use an 'if' condition before I unpack to ensure that
    > the string ends with the delimiter but for 'StringTokenizer' I see
    > that the following input String returns 4 tokens:


    > "AA,BB,CC,1" with delimiter = ","


    > I assume that a token must have 1 or more characters and that I must
    > return to my original code.


    > Is there a way to make 'StringTokenizer' recognize consecutive
    > delimiters and return a null String [e.g. ""] as a token?


    If you use string.split(regex) then it will return the empty matches for
    you. I.e, you'd get 4 from your first example.

    StringTokenizer is (almost) deprecated now for most uses.
     
    samspade, Aug 20, 2003
    #5
  6. samspade wrote:
    > If you use string.split(regex) then it will return the empty matches for
    > you. I.e, you'd get 4 from your first example.
    >
    > StringTokenizer is (almost) deprecated now for most uses.


    Really? That's news to me. I am perfectly comfortable with Java 1.4
    regexes, but I prefer StringTokenizer for those tasks to which it is
    suited. In practice, StringTokenizer is suited to a substantial number
    of the most frequently occurring String parsing tasks. Regexes are more
    powerful, but also heavier, and their API is more complex.


    John Bollinger
     
    John C. Bollinger, Aug 21, 2003
    #6
  7. Jim Crowell

    Jim Crowell Guest

    Roedy,
    >>To make this method more efficient I tried to substitute the
    >>'StringTokenizer' Class

    >
    > It might make it terser, but it won't make it more efficient.
    > StringTokenizer is generic code while, your IndexOf tokenizer is tuned
    > to your particular problem. You should be able to whip the pants off
    > StringTokenizer unless you did something stupid.


    Thanks. I'll stick with my original code at least until I upgrade to
    1.4 and learn more about Regex.

    Jim
     
    Jim Crowell, Aug 23, 2003
    #7
  8. Jim Crowell

    Jim Crowell Guest

    "Jim Crowell" <> wrote in message news:<4wM0b.106464$>...
    > Is there a way to make 'StringTokenizer' recognize consecutive delimiters
    > and return a null String [e.g. ""] as a token?


    Thanks for the many responses. In my original post I neglected to
    state that I am still using Java 1.3.1.

    Therefore, I believe I can not use String.split(Regex) yet.

    To be honest, I did not know about 'Regex' until I received your
    responses. I have since read a tutorial and look forward to using it
    somewhere in my App.

    I am avoiding using Java 1.4.1 because whenever I go to a new upgrade
    I have had to make many changes to my many existing code and I can not
    afford the time right now.

    This is most likely do to poor programming practices on my part since
    I think that a new version should be upword compatable.

    Thanks again,
    Jim
     
    Jim Crowell, Aug 23, 2003
    #8
  9. Stephen Ostermiller, Aug 23, 2003
    #9
  10. Jim Crowell

    Roedy Green Guest

    On 23 Aug 2003 07:28:26 -0700, (Jim Crowell)
    wrote or quoted :

    >
    >Therefore, I believe I can not use String.split(Regex) yet.


    For a quick introduction to what Regexes can do for you, see
    http://mindprod.com/jgloss/regex.html

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Aug 23, 2003
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Phil Hanna

    Re: StringTokenizer Help

    Phil Hanna, Jul 1, 2003, in forum: Java
    Replies:
    0
    Views:
    879
    Phil Hanna
    Jul 1, 2003
  2. ARMANDO  PADILLA

    StringTokenizer help

    ARMANDO PADILLA, Jul 4, 2003, in forum: Java
    Replies:
    3
    Views:
    621
    Chris Smith
    Jul 4, 2003
  3. Liang Yew
    Replies:
    3
    Views:
    655
    Jacob
    Jul 21, 2003
  4. Newbie
    Replies:
    4
    Views:
    1,474
    Stefan Siegl
    Dec 7, 2003
  5. Replies:
    2
    Views:
    607
Loading...

Share This Page