strtok question

Discussion in 'C Programming' started by Stu Cazzo, May 31, 2008.

  1. Stu Cazzo

    Stu Cazzo Guest

    Hi all,
    I have a question on why strtok is doing what it's doing for my
    splitString( string2 ); call.

    Below is the output for the entire program:

    token was: word1
    token was: word2
    token was: word3
    token was: word1
    token was: word3
    empty field found - token <(null)>


    The splitString( string1 ); works as expected, 3 tokens are found.
    The splitString( string2 ); does not work as I expected.
    I was expecting this:

    token was: word1
    empty field found - token <(null)>
    token was: word3

    Why does it not see the empty field for lineToken2?
    Is there a better way to strip out the tokens for the case I have
    where
    there is no space between my delimiter?

    -------------------------------------------------------------------
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>

    void splitString( char *string )
    {
    const char lineDelimiter[] = ",";
    char *lineToken1;
    char *lineToken2;
    char *lineToken3;

    lineToken1 = strtok( string, lineDelimiter );
    if ( lineToken1 == '\0' )
    {
    printf("empty field found - token <%s>\n", lineToken1);
    }
    else
    {
    printf("token was: %s\n", lineToken1);
    }

    lineToken2 = strtok( NULL, lineDelimiter );
    if ( lineToken2 == '\0' )
    {
    printf("empty field found - token <%s>\n", lineToken2);
    }
    else
    {
    printf("token was: %s\n", lineToken2);
    }

    lineToken3 = strtok( NULL, lineDelimiter );
    if ( lineToken3 == '\0' )
    {
    printf("empty field found - token <%s>\n", lineToken3);
    }
    else
    {
    printf("token was: %s\n", lineToken3);
    }

    }



    int main (int argc, const char **argv)
    {
    char string1[] = "word1,word2,word3";
    char string2[] = "word1,,word3";

    splitString( string1 );
    splitString( string2 );

    return( 1 );

    }
    Stu Cazzo, May 31, 2008
    #1
    1. Advertising

  2. Stu Cazzo <> wrote:
    > Hi all,
    > I have a question on why strtok is doing what it's doing for my
    > splitString( string2 ); call.


    > Below is the output for the entire program:


    > token was: word1
    > token was: word2
    > token was: word3
    > token was: word1
    > token was: word3
    > empty field found - token <(null)>



    > The splitString( string1 ); works as expected, 3 tokens are found.
    > The splitString( string2 ); does not work as I expected.
    > I was expecting this:


    > token was: word1
    > empty field found - token <(null)>
    > token was: word3


    > Why does it not see the empty field for lineToken2?


    Because that's not how strtok() works. The man page on my machine
    for strtok() actually makes it rather clear:

    A sequence of two or more contiguous delimiter characters in the
    parsed string is considered to be a single delimiter. Delimiter
    characters at the start or end of the string are ignored. Put
    another way: the tokens returned by strtok() are always non-empty
    strings.

    So if you have more than one ',' in a row all of them are treated
    the same as a single ','.

    > Is there a better way to strip out the tokens for the case I have
    > where there is no space between my delimiter?


    I guess you will have to write your own function for that, probably
    repeatedly using strchr() or strstr().

    Regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
    Jens Thoms Toerring, May 31, 2008
    #2
    1. Advertising

  3. CBFalconer <> writes:
    <snip>
    > while (*src && (tknchar != *src)) {

    <snip body>
    > }
    > if (*src && (tknchar == *src)) src++;


    Some people might find that test confusing. It is certainly
    belt-and-braces code.

    --
    Ben.
    Ben Bacarisse, Jun 1, 2008
    #3
  4. On 31 May 2008 at 23:37, Ben Bacarisse wrote:
    > CBFalconer <> writes:
    ><snip>
    >> while (*src && (tknchar != *src)) {

    ><snip body>
    >> }
    >> if (*src && (tknchar == *src)) src++;

    >
    > Some people might find that test confusing. It is certainly
    > belt-and-braces code.


    Most people grow out of this sort of thing within a few months or so of
    their initial child-like excitement at discovering a language with so
    many side effects. Clarity and ease of debugging become more valuable
    than a transient smug feeling of cleverness.

    Actually I'm pretty tolerant of different people's ways of laying out
    code, indenting and the rest, but CBF really does seem to have total
    anti-taste when it comes to code formatting. Perhaps the most irritating
    thing of all is

    >> } /* tknsplit */


    This seems to me to be about as helpful as the infamous

    i++; /* increment i by one */
    Antoninus Twink, Jun 1, 2008
    #4
  5. Stu Cazzo

    Richard Guest

    Antoninus Twink <> writes:

    > On 31 May 2008 at 23:37, Ben Bacarisse wrote:
    >> CBFalconer <> writes:
    >><snip>
    >>> while (*src && (tknchar != *src)) {

    >><snip body>
    >>> }
    >>> if (*src && (tknchar == *src)) src++;

    >>
    >> Some people might find that test confusing. It is certainly
    >> belt-and-braces code.

    >
    > Most people grow out of this sort of thing within a few months or so of
    > their initial child-like excitement at discovering a language with so
    > many side effects. Clarity and ease of debugging become more valuable
    > than a transient smug feeling of cleverness.


    I have to agree 100% with this statement. Some people seem to take
    pleasure in obfuscating their code.

    >
    > Actually I'm pretty tolerant of different people's ways of laying out
    > code, indenting and the rest, but CBF really does seem to have total
    > anti-taste when it comes to code formatting. Perhaps the most irritating
    > thing of all is
    >
    >>> } /* tknsplit */

    >
    > This seems to me to be about as helpful as the infamous
    >
    > i++; /* increment i by one */


    Again agreed.
    Richard, Jun 1, 2008
    #5
  6. CBFalconer <> writes:

    > Ben Bacarisse wrote:
    >> CBFalconer <> writes:
    >> <snip>
    >>> while (*src && (tknchar != *src)) {

    >> <snip body>
    >>> }
    >>> if (*src && (tknchar == *src)) src++;

    >>
    >> Some people might find that test confusing. It is certainly
    >> belt-and-braces code.

    >
    > You should have left the body.


    It has no bearing on my point. Any loop body that terminates in the
    normal way would do just as well. Maybe I should have said "<snip
    body with no break statement>".

    > That code doesn't have the same
    > effect. Assuming I am correctly interpreting your message.


    I think you missed the point. If you want the function to work when
    tknchar might be 0, then if (tknchar == *src) src++; is enough. If
    you don't want it to work when tknchar is 0 (as seems to be the case)
    then if (*src) src++; is enough.

    It is not in any way wrong, just as if (c == '\n' && c) is not really
    wrong -- it just makes the reader do an unwarranted double take.

    --
    Ben.
    Ben Bacarisse, Jun 1, 2008
    #6
  7. Antoninus Twink <> writes:

    > On 31 May 2008 at 23:37, Ben Bacarisse wrote:
    >> CBFalconer <> writes:
    >><snip>
    >>> while (*src && (tknchar != *src)) {

    >><snip body>
    >>> }
    >>> if (*src && (tknchar == *src)) src++;

    >>
    >> Some people might find that test confusing. It is certainly
    >> belt-and-braces code.

    >
    > Most people grow out of this sort of thing within a few months or so of
    > their initial child-like excitement at discovering a language with so
    > many side effects. Clarity and ease of debugging become more valuable
    > than a transient smug feeling of cleverness.


    I don't know what you mean in this case. I agree with the sentiment,
    but what has it do with this example? How do you write this without
    using side effects?

    The reference to debugging makes me thing you object to the layout.
    Would moving the increment of src down a line make it all OK?

    > Actually I'm pretty tolerant of different people's ways of laying out
    > code,


    Ah. So you were commenting on the layout. What was the reference to
    abusing side effects about?

    --
    Ben.
    Ben Bacarisse, Jun 1, 2008
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Adam Balgach
    Replies:
    2
    Views:
    555
    news-east
    Nov 28, 2004
  2. Alex Vinokur

    strtok() and std::string

    Alex Vinokur, Apr 14, 2005, in forum: C++
    Replies:
    6
    Views:
    4,890
    Pete Becker
    Apr 14, 2005
  3. strtok problem

    , Aug 28, 2003, in forum: C Programming
    Replies:
    4
    Views:
    495
  4. Robert

    strtok trouble

    Robert, Sep 5, 2003, in forum: C Programming
    Replies:
    17
    Views:
    1,209
    Jalapeno
    Sep 6, 2003
  5. Fatih Gey

    segfault on strtok

    Fatih Gey, Oct 23, 2003, in forum: C Programming
    Replies:
    40
    Views:
    1,433
    nobody
    Nov 1, 2003
Loading...

Share This Page