use of strtok( )

Discussion in 'C Programming' started by Kelly B, Apr 22, 2007.

  1. Kelly B

    Kelly B Guest

    I need a function which returns me a "word" from a given string and
    then sets the pointer to the next one which is then retrieved during
    further calls to the function.

    I think strtok( ) is the solution but i could not understand the use of
    the function as given in the C99 standard

    EXAMPLE
    #include <string.h>
    static char str[] = "?a???b,,,#c";
    char *t;
    t = strtok(str, "?"); // t points to the token "a"
    t = strtok(NULL, ","); // t points to the token "??b"
    t = strtok(NULL, "#,"); // t points to the token "c"
    t = strtok(NULL, "?"); // t is a null pointer

    suppose i have a string " The C Programming Language"
    how do i use strtok( ) to retrieve one word at a time from the string i.e
    1st call :The
    2nd call :C
    3rd call :programming
    4th call :Language

    Can anyone please help me out?
     
    Kelly B, Apr 22, 2007
    #1
    1. Advertisements

  2. Kelly B said:

    #include <stdio.h>
    #include <string.h>

    int main(void)
    {
    char writablestring[] = "The C Programming Language";
    char *token = strtok(writablestring, " ");
    while(token != NULL)
    {
    printf("[%s]\n", token);
    token = strtok(NULL, " ");
    }
    return 0;
    }
     
    Richard Heathfield, Apr 22, 2007
    #2
    1. Advertisements

  3. Kelly B

    Kelly B Guest

    Kelly B wrote:
    ....snip...
    I have read the standard a number of times now but i still don't
    understand the following two lines/examples
    t points to "??b" ,why not "???b"
    again #, are not in the same order in the string str
    then why does t point to "c"
     
    Kelly B, Apr 23, 2007
    #3
  4. Kelly B said:
    The order of the delimiters is irrelevant. strtok continues to grab
    token bytes from the string until it hits any one of the specified
    delimiters (or, of course, the end of the string).
     
    Richard Heathfield, Apr 23, 2007
    #4
  5. Kelly B

    Eric Sosman Guest

    You've snipped away too much of the example code, so your
    question makes no sense in isolation. Looking back at the
    start of the thread, it seems to me you have missed two things:

    1) In order to split the original string into individual
    tokens that are themselves strings, strtok() replaces the
    delimiter that ends a token with a '\0' to mark the end of
    a string. The original delimiter is gone, and subsequent
    calls to strtok() will not find it. (As it happens, they
    won't even look at the spot where it used to be.)

    2) The second argument is a *set* of delimiter characters,
    not a delimiting *sequence* of characters. Any character that
    appears in the second argument will be skipped if found before
    a token in the input, or will serve to mark the end of a token.

    Lots of people disparage strtok(), but I find it a useful
    function. Like everything else, you need to understand what
    it does and doesn't do and then use it properly, but if you
    do so there's no reason to fear it.
     
    Eric Sosman, Apr 23, 2007
    #5
  6. Kelly B

    mark_bluemel Guest

    You need to read the description more carefully, I suspect.
    This call (with a non-NULL) first argument starts a sequence of calls
    to strtok.
    The second argument is a list of delimiter characters - any one of
    them (or any
    combination from the list) is taken as delimiting a token.

    As the standard says, we start by skipping over any leading set of
    delimiter
    characters and stop at the first non-delimiter - the letter 'a' in
    this case - this
    will be our return value. We then look for the next delimiter
    character, replace
    the first with a null and remember the location following.

    So after your first call, the second '?' in the string has been
    replaced with a
    null character, strtok has a note of the location of the third "?" and
    we return
    the address of the 'a'.
    On this call, as the first argument is a NULL pointer, strtok()
    continues from
    its saved location - the position of the 3rd '?'. As the character
    here isn't in the
    current delimiter list, this will be our return value. strtok then
    finds the next character
    in the delimiter list - the ',' following 'b' - replaces it with a
    null, and remembers the
    next location (that of the following ',').
    strtok picks up from the second ','. As ',' is in the delimiter list,
    it skips over this and
    the subsequent ','. It finds '#' which is also in the delimiter list,
    so it skips that as well.
     
    mark_bluemel, Apr 23, 2007
    #6
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.