use of strtok( )

Discussion in 'C Programming' started by Kelly B, Apr 22, 2007.

  1. Kelly B

    Kelly B Guest

    I need a function which returns me a "word" from a given string and
    then sets the pointer to the next one which is then retrieved during
    further calls to the function.

    I think strtok( ) is the solution but i could not understand the use of
    the function as given in the C99 standard

    EXAMPLE
    #include <string.h>
    static char str[] = "?a???b,,,#c";
    char *t;
    t = strtok(str, "?"); // t points to the token "a"
    t = strtok(NULL, ","); // t points to the token "??b"
    t = strtok(NULL, "#,"); // t points to the token "c"
    t = strtok(NULL, "?"); // t is a null pointer

    suppose i have a string " The C Programming Language"
    how do i use strtok( ) to retrieve one word at a time from the string i.e
    1st call :The
    2nd call :C
    3rd call :programming
    4th call :Language

    Can anyone please help me out?
     
    Kelly B, Apr 22, 2007
    #1
    1. Advertising

  2. Kelly B said:

    <snip>
    >
    > suppose i have a string " The C Programming Language"
    > how do i use strtok( ) to retrieve one word at a time from the string
    > i.e
    > 1st call :The
    > 2nd call :C
    > 3rd call :programming
    > 4th call :Language
    >
    > Can anyone please help me out?


    #include <stdio.h>
    #include <string.h>

    int main(void)
    {
    char writablestring[] = "The C Programming Language";
    char *token = strtok(writablestring, " ");
    while(token != NULL)
    {
    printf("[%s]\n", token);
    token = strtok(NULL, " ");
    }
    return 0;
    }

    --
    Richard Heathfield
    "Usenet is a strange place" - dmr 29/7/1999
    http://www.cpax.org.uk
    email: rjh at the above domain, - www.
     
    Richard Heathfield, Apr 22, 2007
    #2
    1. Advertising

  3. Kelly B

    Kelly B Guest

    Kelly B wrote:
    ....snip...

    > EXAMPLE
    > #include <string.h>
    > static char str[] = "?a???b,,,#c";
    > char *t;


    I have read the standard a number of times now but i still don't
    understand the following two lines/examples
    > t = strtok(NULL, ","); // t points to the token "??b"

    t points to "??b" ,why not "???b"
    > t = strtok(NULL, "#,"); // t points to the token "c"


    again #, are not in the same order in the string str
    then why does t point to "c"
     
    Kelly B, Apr 23, 2007
    #3
  4. Kelly B said:

    > Kelly B wrote:
    > ...snip...
    >
    >> EXAMPLE
    >> #include <string.h>
    >> static char str[] = "?a???b,,,#c";
    >> char *t;

    >
    > I have read the standard a number of times now but i still don't
    > understand the following two lines/examples
    >> t = strtok(NULL, ","); // t points to the token "??b"

    > t points to "??b" ,why not "???b"
    >> t = strtok(NULL, "#,"); // t points to the token "c"

    >
    > again #, are not in the same order in the string str
    > then why does t point to "c"


    The order of the delimiters is irrelevant. strtok continues to grab
    token bytes from the string until it hits any one of the specified
    delimiters (or, of course, the end of the string).

    --
    Richard Heathfield
    "Usenet is a strange place" - dmr 29/7/1999
    http://www.cpax.org.uk
    email: rjh at the above domain, - www.
     
    Richard Heathfield, Apr 23, 2007
    #4
  5. Kelly B

    Eric Sosman Guest

    Kelly B wrote:
    > Kelly B wrote:
    > ...snip...
    >
    >> EXAMPLE
    >> #include <string.h>
    >> static char str[] = "?a???b,,,#c";
    >> char *t;

    >
    > I have read the standard a number of times now but i still don't
    > understand the following two lines/examples
    >> t = strtok(NULL, ","); // t points to the token "??b"

    > t points to "??b" ,why not "???b"
    >> t = strtok(NULL, "#,"); // t points to the token "c"

    >
    > again #, are not in the same order in the string str
    > then why does t point to "c"


    You've snipped away too much of the example code, so your
    question makes no sense in isolation. Looking back at the
    start of the thread, it seems to me you have missed two things:

    1) In order to split the original string into individual
    tokens that are themselves strings, strtok() replaces the
    delimiter that ends a token with a '\0' to mark the end of
    a string. The original delimiter is gone, and subsequent
    calls to strtok() will not find it. (As it happens, they
    won't even look at the spot where it used to be.)

    2) The second argument is a *set* of delimiter characters,
    not a delimiting *sequence* of characters. Any character that
    appears in the second argument will be skipped if found before
    a token in the input, or will serve to mark the end of a token.

    Lots of people disparage strtok(), but I find it a useful
    function. Like everything else, you need to understand what
    it does and doesn't do and then use it properly, but if you
    do so there's no reason to fear it.

    --
    Eric Sosman
    lid
     
    Eric Sosman, Apr 23, 2007
    #5
  6. Kelly B

    Guest

    On 22 Apr, 19:08, Kelly B <> wrote:
    > I need a function which returns me a "word" from a given string and
    > then sets the pointer to the next one which is then retrieved during
    > further calls to the function.
    >
    > I think strtok( ) is the solution but i could not understand the use of
    > the function as given in the C99 standard


    You need to read the description more carefully, I suspect.

    > EXAMPLE
    > #include <string.h>
    > static char str[] = "?a???b,,,#c";
    > char *t;
    > t = strtok(str, "?"); // t points to the token "a"


    This call (with a non-NULL) first argument starts a sequence of calls
    to strtok.
    The second argument is a list of delimiter characters - any one of
    them (or any
    combination from the list) is taken as delimiting a token.

    As the standard says, we start by skipping over any leading set of
    delimiter
    characters and stop at the first non-delimiter - the letter 'a' in
    this case - this
    will be our return value. We then look for the next delimiter
    character, replace
    the first with a null and remember the location following.

    So after your first call, the second '?' in the string has been
    replaced with a
    null character, strtok has a note of the location of the third "?" and
    we return
    the address of the 'a'.

    > t = strtok(NULL, ","); // t points to the token "??b"


    On this call, as the first argument is a NULL pointer, strtok()
    continues from
    its saved location - the position of the 3rd '?'. As the character
    here isn't in the
    current delimiter list, this will be our return value. strtok then
    finds the next character
    in the delimiter list - the ',' following 'b' - replaces it with a
    null, and remembers the
    next location (that of the following ',').

    > t = strtok(NULL, "#,"); // t points to the token "c"


    strtok picks up from the second ','. As ',' is in the delimiter list,
    it skips over this and
    the subsequent ','. It finds '#' which is also in the delimiter list,
    so it skips that as well.

    > t = strtok(NULL, "?"); // t is a null pointer
     
    , Apr 23, 2007
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Adam Balgach
    Replies:
    2
    Views:
    599
    news-east
    Nov 28, 2004
  2. Alex Vinokur

    strtok() and std::string

    Alex Vinokur, Apr 14, 2005, in forum: C++
    Replies:
    6
    Views:
    4,970
    Pete Becker
    Apr 14, 2005
  3. strtok problem

    , Aug 28, 2003, in forum: C Programming
    Replies:
    4
    Views:
    528
  4. Robert

    strtok trouble

    Robert, Sep 5, 2003, in forum: C Programming
    Replies:
    17
    Views:
    1,242
    Jalapeno
    Sep 6, 2003
  5. Ram Laxman

    How can I use strtok for tokenize integers

    Ram Laxman, Apr 11, 2004, in forum: C Programming
    Replies:
    6
    Views:
    1,376
    Christopher Benson-Manica
    Apr 12, 2004
Loading...

Share This Page