StringTokenizer Lab

Discussion in 'Java' started by Andrew Thompson, May 23, 2004.

  1. Andrew Thompson, May 23, 2004
    #1
    1. Advertising

  2. Andrew Thompson

    Liz Guest

    "Andrew Thompson" <> wrote in message
    news:nr4vgymkx8a6.hfk5bg12urtd$...
    > I made a configurable 'StringTokenizer Lab'.
    > Comments welcome, especially on the (..Java) code.
    > <http://www.physci.org/token.jsp>
    >
    > --
    > Andrew Thompson
    > http://www.PhySci.org/ Open-source software suite
    > http://www.PhySci.org/codes/ Web & IT Help
    > http://www.1point1C.org/ Science & Technology


    Lab is a kind of dog isn't it? Golden or Black?
     
    Liz, May 23, 2004
    #2
    1. Advertising

  3. On Sun, 23 May 2004 18:22:10 GMT, Liz wrote:

    > ..Lab is a kind of dog isn't it? Golden or Black?


    Specially bred, of course. Still took a
    great deal of effort to teach it how to
    tokenize strings, I can tell you.

    Maybe I'll try a RegEX Alsation next.. ;-)

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
     
    Andrew Thompson, May 23, 2004
    #3
  4. Andrew Thompson

    Ryan Stewart Guest

    "Liz" <> wrote in message
    news:mZ5sc.27325$zw.25819@attbi_s01...
    > Lab is a kind of dog isn't it? Golden or Black?
    >

    I don't know that I've heard it referred to as a golden lab. A golden
    retriever, sure, but I'm telling you up front it's a fraud. If they really
    retrieved gold, I'd sure buy one.
     
    Ryan Stewart, May 23, 2004
    #4
  5. Andrew Thompson

    Chris Smith Guest

    Andrew Thompson wrote:
    > I made a configurable 'StringTokenizer Lab'.
    > Comments welcome, especially on the (..Java) code.
    > <http://www.physci.org/token.jsp>


    My one comment is this. Simple uses of StringTokenizer aren't that
    difficult; the interesting bits come from use of the nextToken(String)
    method. Not that I'd ever encourage anyone to use that method, but it
    might be interesting to add a step-by-step version that allows changing
    the delimiter in the middle of tokenizing.

    If you did this and could find a way, it would be *very* interesting to
    be allowed to try that last with several different API versions, so you
    could observe the incompatible behavior.

    --
    www.designacourse.com
    The Easiest Way to Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
     
    Chris Smith, May 24, 2004
    #5
  6. On Sun, 23 May 2004 17:48:45 -0600, Chris Smith wrote:

    > Andrew Thompson wrote:
    >> I made a configurable 'StringTokenizer Lab'.
    >> Comments welcome, especially on the (..Java) code.
    >> <http://www.physci.org/token.jsp>

    >
    > My one comment is this. Simple uses of StringTokenizer aren't that
    > difficult; the interesting bits come from use of the nextToken(String)
    > method. Not that I'd ever encourage anyone to use that method,


    Why not, and is ther then, any more than
    theoretical reasons to test/demonstrate it?*

    >..but it
    > might be interesting to add a step-by-step version that allows changing
    > the delimiter in the middle of tokenizing.


    * ..I have used lots of StringTokenizers,
    but never had the need to change one
    on the fly..

    > If you did this and could find a way, it would be *very* interesting to
    > be allowed to try that last with several different API versions, so you
    > could observe the incompatible behavior.


    ...I'm not familiar with the API differences
    to which you refer. Could this test be done
    by simply alternating between the characters
    of the supplied delimiter?

    E.G. ..

    <hackish vague waving of arms>

    String delim = ";|,"
    StringTokenizer st = new
    StringTokenizer(delim.substring(0,1))
    for (ii=1...) {
    ..
    st.nextToken(delim.substring(ii,ii+1))..
    }

    </hackish vague waving of arms>

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
     
    Andrew Thompson, May 24, 2004
    #6
  7. Andrew Thompson

    Chris Smith Guest

    Andrew Thompson wrote:
    > > My one comment is this. Simple uses of StringTokenizer aren't that
    > > difficult; the interesting bits come from use of the nextToken(String)
    > > method. Not that I'd ever encourage anyone to use that method,

    >
    > Why not, and is ther then, any more than
    > theoretical reasons to test/demonstrate it?*


    Guess not. Just that I am failing to see the point of code that only
    demonstrates the obvious behavior, so I thought I'd throw out an idea.

    > ..I'm not familiar with the API differences
    > to which you refer.


    http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4338282

    --
    www.designacourse.com
    The Easiest Way to Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
     
    Chris Smith, May 24, 2004
    #7
  8. On Sun, 23 May 2004 18:16:06 -0600, Chris Smith wrote:
    > Andrew Thompson wrote:

    (C.S.)
    >>> ..the interesting bits come from use of the nextToken(String)
    >>> method. Not that I'd ever encourage anyone to use that method,

    ...
    >> Why not, and is ther then, any more than
    >> theoretical reasons to test/demonstrate it?*

    >
    > Guess not. Just that I am failing to see the point of code that only
    > demonstrates the obvious behavior,


    To be able to formulate a _link_ that
    expresses (and demonstrates) that
    obvious answer, that is the basic idea.

    That which is obvious to you and me
    and 97 others may not be obvious to
    1-in-100 people. It is for them. [ ;-) ]

    Also, I wanted to come up with a few
    bits of easy 'this is what it works like'
    type code, one for StringTokenizer..
    a configurable Regex Lab is next.

    >...so I thought I'd throw out an idea.


    Thanks! Especially since you were actually
    discussing Java, and there was no mention
    of dogs ... ;-)

    >> ..I'm not familiar with the API differences ..

    ...
    > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4338282


    Uuuggh! That Sun log-in always gets me,
    I must have three 'accounts' set-up with
    them now, but cannot get _any_ of them
    to work..

    10 word summary for those who
    (need to see a StringTokenizer
    work to understand i.. ..errr,)
    cannot get access to Sun?

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
     
    Andrew Thompson, May 24, 2004
    #8
  9. Andrew Thompson

    Chris Smith Guest

    Andrew Thompson wrote:
    > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4338282

    >
    > Uuuggh! That Sun log-in always gets me,
    > I must have three 'accounts' set-up with
    > them now, but cannot get _any_ of them
    > to work..
    >
    > 10 word summary for those who
    > (need to see a StringTokenizer
    > work to understand i.. ..errr,)
    > cannot get access to Sun?


    Basically, it deals with code like:

    StringTokenizer st = new StringTokenizer("test:1,2", ":");
    System.out.println(st.nextToken());
    System.out.println(st.nextToken(","));

    IIRC, in versions prior to 1.3, the colon delimiter is consumed by the
    first nextToken operation. In versions 1.3 and above, it is not, and
    the second token returned is ":1".

    Okay, that was more than 10 words.

    --
    www.designacourse.com
    The Easiest Way to Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
     
    Chris Smith, May 24, 2004
    #9
  10. On Wed, 26 May 2004 08:18:46 GMT-5, Dale King wrote:

    > So I would question whether there is any benefit to a tutorial
    > for newbies when we should be telling them to not use the class.


    What would you recommend for pre-1.4
    code where the delimiter is not changed
    during tokenization?

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
     
    Andrew Thompson, May 27, 2004
    #10
  11. On Mon, 31 May 2004 16:06:25 GMT-5, Dale King wrote:
    > Hello, Andrew Thompson !> You wrote:
    >> On Wed, 26 May 2004 08:18:46 GMT-5, Dale King wrote:
    >>
    >>> So I would question whether there is any
    >>> benefit to a tutorial
    >>> for newbies when we should be telling them to not use the
    >>> class.

    >>
    >> What would you recommend for pre-1.4
    >> code where the delimiter is not changed
    >> during tokenization?

    >
    > As I said there are plentyof third-party replacements for
    > StringTokenizer


    There may well be.. but I cannot see
    it mentioned in this thread.. Are
    you thinking of some other thread perhaps?

    >..that are more capable and can handle the often
    > requested case of CSV files. A google search should turn up a
    > plethora.


    Given both your and Chris' examples involved
    changing the delimiter mid-tokenization, I fail
    to se how they relate in the slightest to the
    case I outline.

    I think it would be more profitable
    to tell the noobs.
    a) Use String.split() from 1.4 if possible, otherwise
    b) StringTokenize *without* stuffing with the
    delimiter in the middle or the process.*

    * with a link to the error at Sun
    ...and a Google search for
    'StringTokenizer replacement'
    at the end.. ;-)

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
     
    Andrew Thompson, Jun 1, 2004
    #11
  12. Andrew Thompson, Jun 1, 2004
    #12
  13. On Mon, 24 May 2004 00:38:50 GMT, Andrew Thompson wrote:

    > ...I wanted to come up with a few
    > bits of easy 'this is what it works like'
    > type code, one for StringTokenizer..
    > a configurable Regex Lab is next.


    <http://www.physci.org/split.jsp>

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
     
    Andrew Thompson, Jun 2, 2004
    #13
  14. Andrew Thompson

    Dale King Guest

    Hello, Chris Smith !
    You wrote:

    > Andrew Thompson wrote:
    > > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4338282

    > >
    > > Uuuggh! That Sun log-in always gets me,
    > > I must have three 'accounts' set-up with
    > > them now, but cannot get _any_ of them
    > > to work..
    > >
    > > 10 word summary for those who
    > > (need to see a StringTokenizer
    > > work to understand i.. ..errr,)
    > > cannot get access to Sun?

    >
    > Basically, it deals with code like:
    >
    > StringTokenizer st = new StringTokenizer("test:1,2", ":");
    > System.out.println(st.nextToken());
    > System.out.println(st.nextToken(","));
    >
    > IIRC, in versions prior to 1.3, the colon delimiter is consumed

    by the
    > first nextToken operation. In versions 1.3 and above, it is

    not, and
    > the second token returned is ":1".
    >
    > Okay, that was more than 10 words.


    I got bit by that one by myself, which led me to file bug
    4645089, which basically said that since they changed the
    behavior they have essentially made the method worthless because
    you can't know what behavior the VM will have. I said they might
    as well deprecate the method.

    They didn't deprecate it officially, but beginning in 1.4 the
    javadocs say that StringTokenizer is only there for supporting
    legacy code and is not recommended for new code.

    And considering the new behavior, I don't think there is anyway
    to actually use it in a beneficial way. I can't think of any real
    world application where you would actually want the delimiter(s)
    returned and there really isn't a clean, simple workaround to
    remove the delimiters at the beginning in general.

    To show what I mean about the difficulty in stripping it consider
    this case:

    StringTokenizer st = new StringTokenizer(s, ": ");
    t1 = st.nextToken();
    t2 = st.nextToken(" ,"));
    t3 = st.nextToken();

    I changed Chris' examle by having sace also a delimiter. If I run
    this on "test : 1, :w" it works as expected before 1.3, but in
    1.3 you get "the colon as the second token.

    And if the string didn't have the spaces you have the additional
    complication that you should only strip the colon from the first
    entry in the comma separated list which complicates any loop to
    read them.

    While you can come up with solutions they are generally more
    complex than just doing the whole scanning job yourself without
    the "aid" of StringTokenizer.

    All this is on top ofall of its failings with respect to handling
    CSV files.

    So my general advice is the same as Sun's, avoid the class
    altogether . Use the String.split method which is much more
    powerful or there are better replacements which work better out
    there.

    So I would question whether there is any benefit to a tutorial
    for newbies when we should be telling them to not use the class.

    --
    Dale King
    My Blog: http://daleking.homedns.org/Blog
     
    Dale King, Apr 15, 2006
    #14
  15. Andrew Thompson

    Dale King Guest

    Hello, Andrew Thompson !
    You wrote:

    > On Wed, 26 May 2004 08:18:46 GMT-5, Dale King wrote:
    >
    > > So I would question whether there is any benefit to a

    tutorial
    > > for newbies when we should be telling them to not use the

    class.
    >
    > What would you recommend for pre-1.4
    > code where the delimiter is not changed
    > during tokenization?


    As I said there are plentyof third-party replacements for
    StringTokenizer that are more capable and can handle the often
    requested case of CSV files. A google search should turn up a
    plethora.
    --
    Dale King
    My Blog: http://daleking.homedns.org/Blog
     
    Dale King, Apr 15, 2006
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Phil Hanna

    Re: StringTokenizer Help

    Phil Hanna, Jul 1, 2003, in forum: Java
    Replies:
    0
    Views:
    889
    Phil Hanna
    Jul 1, 2003
  2. Cawas

    Cipher Lab / Syntech

    Cawas, May 15, 2007, in forum: C Programming
    Replies:
    14
    Views:
    2,515
    David Thompson
    Jul 1, 2007
  3. Replies:
    0
    Views:
    359
  4. dbee
    Replies:
    0
    Views:
    580
  5. marksauder

    Need help with LAB assignment

    marksauder, Nov 16, 2008, in forum: VHDL
    Replies:
    0
    Views:
    620
    marksauder
    Nov 16, 2008
Loading...

Share This Page