StringTokenizer insert into Array??

Discussion in 'Java' started by Newbie, Dec 6, 2003.

  1. Newbie

    Newbie Guest

    I'm reading the text from a plain text file with the StringTokenizer with a
    double loop
    (outer to read each line, inner to read each word on each line).

    Confused a little how the StringTokenizer really works.

    Could some kind soul assist me here as what I really want to do is to
    "insert"
    each word into a new array.

    Each word being an element of the new array that I can manipulate later.

    A small example of code on how to achieve this would be extremely grateful!

    Thankyou for your continued support to a newbie.
    Newbie, Dec 6, 2003
    #1
    1. Advertising

  2. Newbie

    Chris Smith Guest

    Newbie wrote:
    > I'm reading the text from a plain text file with the StringTokenizer with a
    > double loop
    > (outer to read each line, inner to read each word on each line).
    >
    > Confused a little how the StringTokenizer really works.
    >
    > Could some kind soul assist me here as what I really want to do is to
    > "insert"
    > each word into a new array.
    >
    > Each word being an element of the new array that I can manipulate later.
    >
    > A small example of code on how to achieve this would be extremely grateful!
    >
    > Thankyou for your continued support to a newbie.


    Can you privide a little more context regarding what you're actually
    confused about? StringTokenizer is actually rather simple, and looks
    like this:

    StringTokenizer st = new StringTokenizer(line);
    while (st.hasMoreTokens())
    {
    String word = st.nextToken();
    ... // do what you like with the word.
    }

    As you said, you'll need to wrap that in another loop to do it for every
    line of the file. Since I'm unsure exactly what you're doing with the
    words when you get them, I leave that part to you as well.

    --
    www.designacourse.com
    The Easiest Way to Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
    Chris Smith, Dec 6, 2003
    #2
    1. Advertising

  3. Newbie

    Chris Smith Guest

    Chris Smith wrote:
    > StringTokenizer st = new StringTokenizer(line);
    > while (st.hasMoreTokens())
    > {
    > String word = st.nextToken();
    > ... // do what you like with the word.
    > }


    (One thing that does sometimes confuse people about StringTokenizer:
    when you put this code in a loop, you'll still need to create a *new*
    StringTokenizer each time through the loop. StringTokenizer objects are
    of the disposable "use once and then throw away" variety, and can't be
    reused for a different String in the future.)

    --
    www.designacourse.com
    The Easiest Way to Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
    Chris Smith, Dec 6, 2003
    #3
  4. Newbie

    Tony Dahlman Guest

    Newbie wrote:
    >
    > I'm reading the text from a plain text file with the StringTokenizer with a
    > double loop
    > (outer to read each line, inner to read each word on each line).
    >
    > Confused a little how the StringTokenizer really works.
    >
    > Could some kind soul assist me here as what I really want to do is to
    > "insert"
    > each word into a new array.
    >
    > Each word being an element of the new array that I can manipulate later.
    >
    > A small example of code on how to achieve this would be extremely grateful!
    >
    > Thankyou for your continued support to a newbie.


    java.text.BreakIterator is actually the best way to get individual words/
    sentences/lines etc., out of a text stream or file. There is example code
    in the API description. For almost all purposes, I have found it to be
    every bit as good as StreamTokenizer and StringTokenizer.

    Regards, Tony Dahlman
    ---------------------------------------
    a (no spam)d ahlman( a t )att global( d o t )ne t
    Tony Dahlman, Dec 7, 2003
    #4
  5. Newbie

    Stefan Siegl Guest

    Hi I have an solution to offer, but it is surely not the simplest one.
    Just have a look, I hope this will help. It will parse a given String
    for all words. Notice that i introduced two inner classes to make the
    code more readable.

    hth, Stefan

    public class Test {
    class Line{
    private List words = new ArrayList();
    public void addWord (String word){
    words.add(word);
    }
    public List getWords(){
    return words;
    }
    public String toString(){
    StringBuffer ret = new StringBuffer();
    ret.append("the words are:\n");
    Iterator wIt = words.iterator();
    while (wIt.hasNext()){
    ret.append(" "+wIt.next()+" \n");
    }
    return ret.toString();
    }
    }

    class Text{
    List lines = new ArrayList();
    public void addLine (Line l){
    lines.add(l);
    }
    public List getLines(){
    return lines;
    }
    public String toString(){
    StringBuffer ret = new StringBuffer();
    ret.append("the lines are:");
    Iterator lIt = lines.iterator();
    while (lIt.hasNext()){
    ret.append(lIt.next());
    }
    return ret.toString();
    }
    }

    public Text parseText (String text){
    Text toReturn = new Text();

    StringTokenizer outer = new StringTokenizer(text, "\n");
    while (outer.hasMoreElements()){
    String line = (String) outer.nextElement();
    //create the inner StringTokenizer and search for the words
    StringTokenizer inner = new StringTokenizer(line, " ");
    Line currentLine = new Line();
    while (inner.hasMoreElements()){
    currentLine.addWord((String) inner.nextElement());
    }
    // add the line object to the text
    toReturn.addLine(currentLine);
    }
    return toReturn;
    }

    public static void main (String args[]){
    String theText = "I am the text from the file \n Please use
    java.io to parse me \n thanks";
    System.out.println(new Test().parseText(theText));
    }
    }

    Newbie wrote:

    > I'm reading the text from a plain text file with the StringTokenizer with a
    > double loop
    > (outer to read each line, inner to read each word on each line).
    >
    > Confused a little how the StringTokenizer really works.
    >
    > Could some kind soul assist me here as what I really want to do is to
    > "insert"
    > each word into a new array.
    >
    > Each word being an element of the new array that I can manipulate later.
    >
    > A small example of code on how to achieve this would be extremely grateful!
    >
    > Thankyou for your continued support to a newbie.
    >
    >
    Stefan Siegl, Dec 7, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Phil Hanna

    Re: StringTokenizer Help

    Phil Hanna, Jul 1, 2003, in forum: Java
    Replies:
    0
    Views:
    858
    Phil Hanna
    Jul 1, 2003
  2. ARMANDO  PADILLA

    StringTokenizer help

    ARMANDO PADILLA, Jul 4, 2003, in forum: Java
    Replies:
    3
    Views:
    603
    Chris Smith
    Jul 4, 2003
  3. Liang Yew
    Replies:
    3
    Views:
    634
    Jacob
    Jul 21, 2003
  4. Jim Crowell

    StringTokenizer Question?

    Jim Crowell, Aug 20, 2003, in forum: Java
    Replies:
    9
    Views:
    3,745
    Roedy Green
    Aug 23, 2003
  5. Replies:
    2
    Views:
    591
Loading...

Share This Page