Help with strtok

Discussion in 'C Programming' started by manochavishal@gmail.com, Mar 14, 2006.

  1. Guest

    Hi
    I am writing a Program
    in which i get input as

    #C1012,S,A#C1013,S,U

    I want to get C1012,S,A using strtok and then pass this to function
    CreateVideo
    which will further strtok this (C1012,S,A) and store the required
    values.

    Now here is the piece of that code:

    #define DELIM2 #
    char * field;
    char fieldcopy[20];

    /*Here i have input as #C1012,S,A#C1013,S,U*/
    field = strtok(NULL,DELIM2);
    while(field != NULL)
    {
    strcpy(fieldcopy,field);
    CreateCopies(copy,fieldcopy,NoCopies);
    field = strtok(NULL,DELIM2);
    printf("Field in CreateVideo is %s\n",field);
    }

    Now if I call CreateCopies the strtok doesn't tokenize till the end.
    But if i comment the CreateCopies call it does tokenize till the end.
    In the first case the second time i call strtok field gets a value of
    NULL but works fine if i dont call CreateObjects.

    Why this behaviour???
     
    , Mar 14, 2006
    #1
    1. Advertising

  2. On 2006-03-14, <> wrote:
    > Hi
    > I am writing a Program
    > in which i get input as
    >
    > #C1012,S,A#C1013,S,U
    >
    > I want to get C1012,S,A using strtok and then pass this to function
    > CreateVideo
    > which will further strtok this (C1012,S,A) and store the required
    > values.
    >
    > Now here is the piece of that code:
    >
    > #define DELIM2 #


    you should change this DELIM2 to use a string.

    > char * field;
    > char fieldcopy[20];
    >
    > /*Here i have input as #C1012,S,A#C1013,S,U*/
    > field = strtok(NULL,DELIM2);


    Its not clear if you have called strtok with your field as first
    argument before.


    > while(field != NULL)
    > {
    > strcpy(fieldcopy,field);
    > CreateCopies(copy,fieldcopy,NoCopies);
    > field = strtok(NULL,DELIM2);
    > printf("Field in CreateVideo is %s\n",field);
    > }
    >
    > Now if I call CreateCopies the strtok doesn't tokenize till the end.
    > But if i comment the CreateCopies call it does tokenize till the end.
    > In the first case the second time i call strtok field gets a value of
    > NULL but works fine if i dont call CreateObjects.
    >
    > Why this behaviour???
    >


    You should post all the code, theres not enough info there :
    specifically the set up of "field" and the initial call to strtok which
    should have "field" as the first argument. Since from the look of it,
    and assuming nothing nasty, Createfields doesnt change "field" then
    the *current* problem is nothing to do with Createfields : could be some bad
    pointers because strtok wasnt initialised properly.
     
    Richard G. Riley, Mar 14, 2006
    #2
    1. Advertising

  3. Richard Bos Guest

    "" <> wrote:

    > field = strtok(NULL,DELIM2);
    > while(field != NULL)
    > {
    > strcpy(fieldcopy,field);
    > CreateCopies(copy,fieldcopy,NoCopies);
    > field = strtok(NULL,DELIM2);
    > printf("Field in CreateVideo is %s\n",field);
    > }
    >
    > Now if I call CreateCopies the strtok doesn't tokenize till the end.
    > But if i comment the CreateCopies call it does tokenize till the end.


    Does CreateCopies perhaps also use strtok()? You can't nest uses of
    strtok(), because it keeps only a single, static state.

    Richard
     
    Richard Bos, Mar 14, 2006
    #3
  4. Richard Bos wrote:
    >
    > "" <> wrote:
    >
    > > field = strtok(NULL,DELIM2);
    > > while(field != NULL)
    > > {
    > > strcpy(fieldcopy,field);
    > > CreateCopies(copy,fieldcopy,NoCopies);
    > > field = strtok(NULL,DELIM2);
    > > printf("Field in CreateVideo is %s\n",field);
    > > }
    > >
    > > Now if I call CreateCopies the strtok doesn't tokenize till the end.
    > > But if i comment the CreateCopies call it does tokenize till the end.

    >
    > Does CreateCopies perhaps also use strtok()? You can't nest uses of
    > strtok(), because it keeps only a single, static state.
    >

    Well, you can hardly nest invocations of strtok(), but it is *not*
    difficult to "roll your own" that *will* support nested invocations.

    --
    +----------------------------------------------------------------+
    | Charles and Francis Richmond richmond at plano dot net |
    +----------------------------------------------------------------+
     
    Charles Richmond, Mar 15, 2006
    #4
  5. Richard Bos Guest

    Charles Richmond <> wrote:

    > Richard Bos wrote:
    > >
    > > "" <> wrote:
    > >
    > > > Now if I call CreateCopies the strtok doesn't tokenize till the end.
    > > > But if i comment the CreateCopies call it does tokenize till the end.

    > >
    > > Does CreateCopies perhaps also use strtok()? You can't nest uses of
    > > strtok(), because it keeps only a single, static state.
    > >

    > Well, you can hardly nest invocations of strtok(), but it is *not*
    > difficult to "roll your own" that *will* support nested invocations.


    That is true. While you're at it, allow it to split "name,address,,city"
    into four rather than three fields, as well.

    > --


    That, btw, is not a .sig-sep.

    Richard
     
    Richard Bos, Mar 15, 2006
    #5
  6. CBFalconer Guest

    Charles Richmond wrote:
    > Richard Bos wrote:
    >>

    .... snip ...
    >>
    >> Does CreateCopies perhaps also use strtok()? You can't nest uses
    >> of strtok(), because it keeps only a single, static state.

    >
    > Well, you can hardly nest invocations of strtok(), but it is *not*
    > difficult to "roll your own" that *will* support nested invocations.


    Here's a version I posted a while ago. Public Domain.

    /* ------- file toksplit.h ----------*/
    #ifndef H_toksplit_h
    # define H_toksplit_h

    # ifdef __cplusplus
    extern "C" {
    # endif

    #include <stddef.h>

    /* copy over the next token from an input string, after
    skipping leading blanks (or other whitespace?). The
    token is terminated by the first appearance of tokchar,
    or by the end of the source string.

    The caller must supply sufficient space in token to
    receive any token, Otherwise tokens will be truncated.

    Returns: a pointer past the terminating tokchar.

    This will happily return an infinity of empty tokens if
    called with src pointing to the end of a string. Tokens
    will never include a copy of tokchar.

    released to Public Domain, by C.B. Falconer.
    Published 2006-02-20. Attribution appreciated.
    */

    const char *toksplit(const char *src, /* Source of tokens */
    char tokchar, /* token delimiting char */
    char *token, /* receiver of parsed token */
    size_t lgh); /* length token can receive */
    /* not including final '\0' */

    # ifdef __cplusplus
    }
    # endif
    #endif
    /* ------- end file toksplit.h ----------*/

    /* ------- file toksplit.c ----------*/
    #include "toksplit.h"

    /* copy over the next token from an input string, after
    skipping leading blanks (or other whitespace?). The
    token is terminated by the first appearance of tokchar,
    or by the end of the source string.

    The caller must supply sufficient space in token to
    receive any token, Otherwise tokens will be truncated.

    Returns: a pointer past the terminating tokchar.

    This will happily return an infinity of empty tokens if
    called with src pointing to the end of a string. Tokens
    will never include a copy of tokchar.

    A better name would be "strtkn", except that is reserved
    for the system namespace. Change to that at your risk.

    released to Public Domain, by C.B. Falconer.
    Published 2006-02-20. Attribution appreciated.
    */

    const char *toksplit(const char *src, /* Source of tokens */
    char tokchar, /* token delimiting char */
    char *token, /* receiver of parsed token */
    size_t lgh) /* length token can receive */
    /* not including final '\0' */
    {
    if (src) {
    while (' ' == *src) *src++;

    while (*src && (tokchar != *src)) {
    if (lgh) {
    *token++ = *src;
    --lgh;
    }
    src++;
    }
    if (*src && (tokchar == *src)) src++;
    }
    *token = '\0';
    return src;
    } /* toksplit */

    #ifdef TESTING
    #include <stdio.h>

    #define ABRsize 6 /* length of acceptable token abbreviations */

    int main(void)
    {
    char teststring[] = "This is a test, ,, abbrev, more";

    const char *t, *s = teststring;
    int i;
    char token[ABRsize + 1];

    puts(teststring);
    t = s;
    for (i = 0; i < 4; i++) {
    t = toksplit(t, ',', token, ABRsize);
    putchar(i + '1'); putchar(':');
    puts(token);
    }

    puts("\nHow to detect 'no more tokens'");
    t = s; i = 0;
    while (*t) {
    t = toksplit(t, ',', token, 3);
    putchar(i + '1'); putchar(':');
    puts(token);
    i++;
    }

    puts("\nUsing blanks as token delimiters");
    t = s; i = 0;
    while (*t) {
    t = toksplit(t, ' ', token, ABRsize);
    putchar(i + '1'); putchar(':');
    puts(token);
    i++;
    }
    return 0;
    } /* main */

    #endif
    /* ------- end file toksplit.c ----------*/

    --
    "If you want to post a followup via groups.google.com, don't use
    the broken "Reply" link at the bottom of the article. Click on
    "show options" at the top of the article, then click on the
    "Reply" at the bottom of the article headers." - Keith Thompson
    More details at: <http://cfaj.freeshell.org/google/>
    Also see <http://www.safalra.com/special/googlegroupsreply/>
     
    CBFalconer, Mar 15, 2006
    #6
  7. On 2006-03-15, Richard Bos <> wrote:
    > Charles Richmond <> wrote:
    >> Well, you can hardly nest invocations of strtok(), but it is *not*
    >> difficult to "roll your own" that *will* support nested invocations.

    >
    > That is true. While you're at it, allow it to split "name,address,,city"
    > into four rather than three fields, as well.


    Should it also split "name address city" to 7 fields as well if ' '
    is part of its seperator string?

    Obviously, which one is preferable is highly dependant on what you want
    to do with it.

    --
    John Tsiombikas (Nuclear / Mindlapse)

    http://nuclear.demoscene.gr/
     
    John Tsiombikas (Nuclear / Mindlapse), Mar 16, 2006
    #7
  8. Jordan Abel Guest

    On 2006-03-16, John Tsiombikas (Nuclear / Mindlapse) <> wrote:
    > On 2006-03-15, Richard Bos <> wrote:
    >> Charles Richmond <> wrote:
    >>> Well, you can hardly nest invocations of strtok(), but it is *not*
    >>> difficult to "roll your own" that *will* support nested invocations.

    >>
    >> That is true. While you're at it, allow it to split "name,address,,city"
    >> into four rather than three fields, as well.

    >
    > Should it also split "name address city" to 7 fields as well if ' '
    > is part of its seperator string?
    >
    > Obviously, which one is preferable is highly dependant on what you want
    > to do with it.


    So there should be two functions. or maybe a flag to be passed to the
    single function.
     
    Jordan Abel, Mar 16, 2006
    #8
  9. CBFalconer Guest

    Jordan Abel wrote:
    > John Tsiombikas (Nuclear / Mindlapse) <> wrote:
    >> On 2006-03-15, Richard Bos <> wrote:
    >>> Charles Richmond <> wrote:
    >>>
    >>>> Well, you can hardly nest invocations of strtok(), but it is *not*
    >>>> difficult to "roll your own" that *will* support nested invocations.
    >>>
    >>> That is true. While you're at it, allow it to split "name,address,,city"
    >>> into four rather than three fields, as well.

    >>
    >> Should it also split "name address city" to 7 fields as well if ' '
    >> is part of its seperator string?
    >>
    >> Obviously, which one is preferable is highly dependant on what you want
    >> to do with it.

    >
    > So there should be two functions. or maybe a flag to be passed to the
    > single function.


    If you simply say that leading blanks in a token are absorbed, and
    that the token char is a separator, the conflicts disappear. See
    my toksplit routine elsethread. Which, BTW, is re-entrant.

    --
    "If you want to post a followup via groups.google.com, don't use
    the broken "Reply" link at the bottom of the article. Click on
    "show options" at the top of the article, then click on the
    "Reply" at the bottom of the article headers." - Keith Thompson
    More details at: <http://cfaj.freeshell.org/google/>
    Also see <http://www.safalra.com/special/googlegroupsreply/>
     
    CBFalconer, Mar 16, 2006
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Adam Balgach
    Replies:
    2
    Views:
    565
    news-east
    Nov 28, 2004
  2. Alex Vinokur

    strtok() and std::string

    Alex Vinokur, Apr 14, 2005, in forum: C++
    Replies:
    6
    Views:
    4,922
    Pete Becker
    Apr 14, 2005
  3. ern

    strtok ( ) help

    ern, Jan 20, 2006, in forum: C Programming
    Replies:
    13
    Views:
    838
  4. Help With strtok

    , Mar 14, 2006, in forum: C Programming
    Replies:
    2
    Views:
    436
  5. Glen

    strtok help

    Glen, Aug 10, 2005, in forum: C++
    Replies:
    1
    Views:
    394
Loading...

Share This Page