strtok() problem

Discussion in 'C Programming' started by ern, Sep 20, 2005.

  1. ern

    ern Guest

    I'm using it like this:

    char * _command = "one two three four";
    char * g_UserCommands[4];
    const char * delimeters = " ";
    g_UserCommands[0] = strtok(_command, delimeters);
    g_UserCommands[1] = strtok(g_UserCommands[0], delimeters);
    g_UserCommands[2] = strtok(g_UserCommands[1], delimeters);
    g_UserCommands[3] = strtok(g_UserCommands[2], delimeters);
    //Then I print each entry of g_UserCommands.

    When I run this thing, it will just print out one one one one...

    I can't figure out what I'm doing wrong...
    ern, Sep 20, 2005
    #1
    1. Advertising

  2. "ern" <> wrote in message
    news:...
    > char * _command = "one two three four";
    > char * g_UserCommands[4];
    > const char * delimeters = " ";
    > g_UserCommands[0] = strtok(_command, delimeters);
    > g_UserCommands[1] = strtok(g_UserCommands[0], delimeters);
    > g_UserCommands[2] = strtok(g_UserCommands[1], delimeters);
    > g_UserCommands[3] = strtok(g_UserCommands[2], delimeters);
    > //Then I print each entry of g_UserCommands.
    >
    > When I run this thing, it will just print out one one one one...
    >
    > I can't figure out what I'm doing wrong...


    1st call to strtok() is with ptr to the string to be parsed.
    All subsequent calls to strtok() must be with NULL.
    To begin parsing next string, use a new non-NULL ptr.

    Read The Manual.

    Alex
    Alexei A. Frounze, Sep 20, 2005
    #2
    1. Advertising

  3. ern

    ern Guest

    Thanks. That worked. Actually, I did read the manual and couldn't
    figure it out... thus the question.
    ern, Sep 20, 2005
    #3
  4. ern

    Flash Gordon Guest

    Alexei A. Frounze wrote:
    > "ern" <> wrote in message
    > news:...
    >
    >> char * _command = "one two three four";


    It is best to avoid identifiers starting with an _. Some of them can be
    used in some scopes, but it is far easier to completely avoid them that
    to remember which ones can be used where.

    >> char * g_UserCommands[4];
    >> const char * delimeters = " ";
    >>g_UserCommands[0] = strtok(_command, delimeters);
    >>g_UserCommands[1] = strtok(g_UserCommands[0], delimeters);
    >>g_UserCommands[2] = strtok(g_UserCommands[1], delimeters);
    >>g_UserCommands[3] = strtok(g_UserCommands[2], delimeters);
    >> //Then I print each entry of g_UserCommands.
    >>
    >>When I run this thing, it will just print out one one one one...
    >>
    >>I can't figure out what I'm doing wrong...

    >
    > 1st call to strtok() is with ptr to the string to be parsed.
    > All subsequent calls to strtok() must be with NULL.
    > To begin parsing next string, use a new non-NULL ptr.
    >
    > Read The Manual.


    Another problem is that strtok modifies the string it is tokenising and
    the C standard forbids you from modifying string literals. Common
    effects of this include the program aborting, finding that string
    literals have been merged so when you change one instance all the others
    change, and demons flying out of your nose and then slapping you with a
    wet kipper.

    *Never* pass a string literal to strtok and never try to modify a string
    literal in any other way.

    char command[] = "one two three four";

    would have been far better.
    --
    Flash Gordon
    Living in interesting times.
    Although my email address says spam, it is real and I read it.
    Flash Gordon, Sep 20, 2005
    #4
  5. ern

    Default User Guest

    ern wrote:

    > Thanks. That worked. Actually, I did read the manual and couldn't
    > figure it out... thus the question.


    Please quote enough of the previous message to provide context. To do
    so from Google, click "show options" and use the Reply shown in the
    expanded header.


    You have another big problem in your code:

    > char * _command = "one two three four";


    > g_UserCommands[0] = strtok(_command, delimeters);



    You are trying to use strtok() on a string literal, which is undefined
    behavior. That's because the function modifies the string. You can't do
    that.

    Replace the first line with:

    char command[] = "one two three four"; /* leading underscores suck */




    Brian
    Default User, Sep 21, 2005
    #5
  6. "ern" <> writes:
    > I'm using it like this:
    >
    > char * _command = "one two three four";
    > char * g_UserCommands[4];
    > const char * delimeters = " ";
    > g_UserCommands[0] = strtok(_command, delimeters);
    > g_UserCommands[1] = strtok(g_UserCommands[0], delimeters);
    > g_UserCommands[2] = strtok(g_UserCommands[1], delimeters);
    > g_UserCommands[3] = strtok(g_UserCommands[2], delimeters);
    > //Then I print each entry of g_UserCommands.
    >
    > When I run this thing, it will just print out one one one one...
    >
    > I can't figure out what I'm doing wrong...


    In addition to the problem Alexei pointed out, you're modifying the
    contents of a string literal, which invokes undefined behavior.
    Change
    char * _command = "one two three four";
    to
    char _command[] = "one two three four";

    (But in a real program, you'll presumbly be getting the data from some
    external source rather than a string literal.)

    And using identifiers that start with an underscore is a bad idea;
    many of them are reserved to the implementation. The rules are
    slightly more complicated than that, but it's best just to avoid such
    identifiers altogethers.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
    Keith Thompson, Sep 21, 2005
    #6
  7. "Flash Gordon" <> wrote in message
    news:-gordon.me.uk...
    ....
    > Another problem is that strtok modifies the string it is tokenising and
    > the C standard forbids you from modifying string literals. Common
    > effects of this include the program aborting, finding that string
    > literals have been merged so when you change one instance all the others
    > change, and demons flying out of your nose and then slapping you with a
    > wet kipper.
    >
    > *Never* pass a string literal to strtok and never try to modify a string
    > literal in any other way.
    >
    > char command[] = "one two three four";
    >
    > would have been far better.


    You're right on that. That's safe. An array is guaranteed not to intersect
    with some other array, while the string literals may be combined (if
    identical) and reused and be indeed put into read-only part of address
    space. I'm not doing much of the string/text processing, so to me this is
    something I don't catch, unldss I really do need to something like this.

    How much does it take one to find all the rope to shoot in the foot if
    speaking C? A few intensive years of using and rediscovering C again and
    again? :)

    Alex
    Alexei A. Frounze, Sep 21, 2005
    #7
  8. ern

    Randy Howard Guest

    Alexei A. Frounze wrote
    (in article <>):

    > How much does it take one to find all the rope to shoot in the foot if
    > speaking C?


    I guess that depends upon how long it takes you to read K&R2,
    the clc FAQ, and whether or not you learn from your mistakes on
    the first, second, or any subsequent incidents.


    --
    Randy Howard (2reply remove FOOBAR)
    Randy Howard, Sep 21, 2005
    #8
  9. ern wrote:
    > I'm using it like this:
    >
    > char * _command = "one two three four";
    > char * g_UserCommands[4];
    > const char * delimeters = " ";
    > g_UserCommands[0] = strtok(_command, delimeters);
    > g_UserCommands[1] = strtok(g_UserCommands[0], delimeters);
    > g_UserCommands[2] = strtok(g_UserCommands[1], delimeters);
    > g_UserCommands[3] = strtok(g_UserCommands[2], delimeters);
    > //Then I print each entry of g_UserCommands.
    >
    > When I run this thing, it will just print out one one one one...
    >
    > I can't figure out what I'm doing wrong...


    Try:

    char *command = "one two three four";
    char *g_UserCommands[4];
    const char *delimiters = " ";
    g_UserCommands[0] = strtok(command, delimiters);
    g_UserCommands[1] = strtok(0, delimiters);
    g_UserCommands[2] = strtok(0, delimiters);
    g_UserCommands[3] = strtok(0, delimiters);

    Gregory Pietsch
    Gregory Pietsch, Sep 21, 2005
    #9
  10. ern

    Default User Guest

    Gregory Pietsch wrote:

    > Try:
    >
    > char *command = "one two three four";
    > char *g_UserCommands[4];
    > const char *delimiters = " ";
    > g_UserCommands[0] = strtok(command, delimiters);


    Undefined Behavior, attempt to modify a string literal. This has been
    posted several times over the past day or so.



    Brian
    Default User, Sep 21, 2005
    #10
  11. Default User wrote:
    > Gregory Pietsch wrote:
    >
    > > Try:
    > >
    > > char *command = "one two three four";
    > > char *g_UserCommands[4];
    > > const char *delimiters = " ";
    > > g_UserCommands[0] = strtok(command, delimiters);

    >
    > Undefined Behavior, attempt to modify a string literal. This has been
    > posted several times over the past day or so.
    >
    >
    >
    > Brian


    Oops ... forgot about that.

    #include <string.h>
    char command[128];
    strcpy(command, "one two three four");

    Gregory Pietsch
    Gregory Pietsch, Sep 22, 2005
    #11
  12. "Gregory Pietsch" <> writes:
    > Default User wrote:
    >> Gregory Pietsch wrote:
    >> > Try:
    >> >
    >> > char *command = "one two three four";
    >> > char *g_UserCommands[4];
    >> > const char *delimiters = " ";
    >> > g_UserCommands[0] = strtok(command, delimiters);

    >>
    >> Undefined Behavior, attempt to modify a string literal. This has been
    >> posted several times over the past day or so.

    >
    > Oops ... forgot about that.
    >
    > #include <string.h>
    > char command[128];
    > strcpy(command, "one two three four");


    Or just:

    char command[] = "one two three four";

    But as I noted earlier, in a real-world program you're unlikely to
    want to apply strtok() to constant data anyway.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
    Keith Thompson, Sep 22, 2005
    #12
  13. ern

    Default User Guest

    Keith Thompson wrote:

    > "Gregory Pietsch" <> writes:
    > > Default User wrote:
    > >> Gregory Pietsch wrote:
    > >> > Try:
    > >> >
    > >> > char *command = "one two three four";
    > >> > char *g_UserCommands[4];
    > >> > const char *delimiters = " ";
    > >> > g_UserCommands[0] = strtok(command, delimiters);
    > > >
    > >> Undefined Behavior, attempt to modify a string literal. This has

    > been >> posted several times over the past day or so.
    > >
    > > Oops ... forgot about that.
    > >
    > > #include <string.h>
    > > char command[128];
    > > strcpy(command, "one two three four");

    >
    > Or just:
    >
    > char command[] = "one two three four";
    >
    > But as I noted earlier, in a real-world program you're unlikely to
    > want to apply strtok() to constant data anyway.



    Probably so. It's still best to apply some safety measures if using
    strtok() or use some other method that doesn't need to modify the input.


    Brian
    Default User, Sep 22, 2005
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. strtok problem

    , Aug 28, 2003, in forum: C Programming
    Replies:
    4
    Views:
    499
  2. Ram Laxman

    strtok problem

    Ram Laxman, Apr 11, 2004, in forum: C Programming
    Replies:
    3
    Views:
    5,092
    Ram Laxman
    May 3, 2004
  3. collinm

    strtok problem - strcmp

    collinm, Mar 24, 2005, in forum: C Programming
    Replies:
    4
    Views:
    784
    Mark McIntyre
    Mar 24, 2005
  4. Replies:
    3
    Views:
    485
    Steven Kobes
    Jul 27, 2005
  5. strtok problem

    , Mar 29, 2006, in forum: C Programming
    Replies:
    6
    Views:
    431
    CBFalconer
    Mar 31, 2006
Loading...

Share This Page