use of strtok( )

[Kelly B]

I need a function which returns me a "word" from a given string and
then sets the pointer to the next one which is then retrieved during
further calls to the function.

I think strtok( ) is the solution but i could not understand the use of
the function as given in the C99 standard

EXAMPLE
#include <string.h>
static char str[] = "?a???b,,,#c";
char *t;
t = strtok(str, "?"); // t points to the token "a"
t = strtok(NULL, ","); // t points to the token "??b"
t = strtok(NULL, "#,"); // t points to the token "c"
t = strtok(NULL, "?"); // t is a null pointer

suppose i have a string " The C Programming Language"
how do i use strtok( ) to retrieve one word at a time from the string i.e
1st call :The
2nd call :C
3rd call rogramming
4th call :Language

Can anyone please help me out?

 

[Richard Heathfield]

Kelly B said:

suppose i have a string " The C Programming Language"
how do i use strtok( ) to retrieve one word at a time from the string
i.e
1st call :The
2nd call :C
3rd call rogramming
4th call :Language

Can anyone please help me out?

#include <stdio.h>
#include <string.h>

int main(void)
{
char writablestring[] = "The C Programming Language";
char *token = strtok(writablestring, " ");
while(token != NULL)
{
printf("[%s]\n", token);
token = strtok(NULL, " ");
}
return 0;
}

 

[Kelly B]

Kelly B wrote:
....snip...

EXAMPLE
#include <string.h>
static char str[] = "?a???b,,,#c";
char *t;

I have read the standard a number of times now but i still don't
understand the following two lines/examples

t = strtok(NULL, ","); // t points to the token "??b"

t points to "??b" ,why not "???b"

t = strtok(NULL, "#,"); // t points to the token "c"

again #, are not in the same order in the string str
then why does t point to "c"

 

[Richard Heathfield]

Kelly B said:

Kelly B wrote:
...snip...

EXAMPLE
#include <string.h>
static char str[] = "?a???b,,,#c";
char *t;

I have read the standard a number of times now but i still don't
understand the following two lines/examples

t = strtok(NULL, ","); // t points to the token "??b"

t points to "??b" ,why not "???b"

t = strtok(NULL, "#,"); // t points to the token "c"

again #, are not in the same order in the string str
then why does t point to "c"

The order of the delimiters is irrelevant. strtok continues to grab
token bytes from the string until it hits any one of the specified
delimiters (or, of course, the end of the string).

 

[Eric Sosman]

Kelly B wrote:
...snip...

EXAMPLE
#include <string.h>
static char str[] = "?a???b,,,#c";
char *t;

I have read the standard a number of times now but i still don't
understand the following two lines/examples

t = strtok(NULL, ","); // t points to the token "??b"

t points to "??b" ,why not "???b"

t = strtok(NULL, "#,"); // t points to the token "c"

again #, are not in the same order in the string str
then why does t point to "c"

You've snipped away too much of the example code, so your
question makes no sense in isolation. Looking back at the
start of the thread, it seems to me you have missed two things:

1) In order to split the original string into individual
tokens that are themselves strings, strtok() replaces the
delimiter that ends a token with a '\0' to mark the end of
a string. The original delimiter is gone, and subsequent
calls to strtok() will not find it. (As it happens, they
won't even look at the spot where it used to be.)

2) The second argument is a *set* of delimiter characters,
not a delimiting *sequence* of characters. Any character that
appears in the second argument will be skipped if found before
a token in the input, or will serve to mark the end of a token.

Lots of people disparage strtok(), but I find it a useful
function. Like everything else, you need to understand what
it does and doesn't do and then use it properly, but if you
do so there's no reason to fear it.

 

[mark_bluemel]

I need a function which returns me a "word" from a given string and
then sets the pointer to the next one which is then retrieved during
further calls to the function.

I think strtok( ) is the solution but i could not understand the use of
the function as given in the C99 standard

You need to read the description more carefully, I suspect.

EXAMPLE
#include <string.h>
static char str[] = "?a???b,,,#c";
char *t;
t = strtok(str, "?"); // t points to the token "a"

This call (with a non-NULL) first argument starts a sequence of calls
to strtok.
The second argument is a list of delimiter characters - any one of
them (or any
combination from the list) is taken as delimiting a token.

As the standard says, we start by skipping over any leading set of
delimiter
characters and stop at the first non-delimiter - the letter 'a' in
this case - this
will be our return value. We then look for the next delimiter
character, replace
the first with a null and remember the location following.

So after your first call, the second '?' in the string has been
replaced with a
null character, strtok has a note of the location of the third "?" and
we return
the address of the 'a'.

t = strtok(NULL, ","); // t points to the token "??b"

On this call, as the first argument is a NULL pointer, strtok()
continues from
its saved location - the position of the 3rd '?'. As the character
here isn't in the
current delimiter list, this will be our return value. strtok then
finds the next character
in the delimiter list - the ',' following 'b' - replaces it with a
null, and remembers the
next location (that of the following ',').

t = strtok(NULL, "#,"); // t points to the token "c"

strtok picks up from the second ','. As ',' is in the delimiter list,
it skips over this and
the subsequent ','. It finds '#' which is also in the delimiter list,
so it skips that as well.