cannot read after \n using strtok()

Discussion in 'C Programming' started by ohaqqi@gmail.com, Nov 4, 2007.

  1. Guest

    Hi guys, I'm trying to figure out the problem with my C code below
    implementing strtok(). I am trying to read in a CSV text file which
    contains information separated by commas and \n newlines. In the CSV
    file, the first line is filled with header info that I don't want to
    store in my array. After the header info is the useful info, for
    example:

    loan_id,lender_id,borrower_id,principle_amt,interest_rate,loan_status,loan_balance,fco_discount,loan_amt_type,int_rate_type,risk,property_addr,property_city,property_st,property_zip,settlement_date
    2445,44490,3332000,344123.22,5.9,default,301223.89,1.2,Regular,Fixed,
    22,204 Main Street,Baltimore,MD,21076,Mar 30 2009
    10446,1490,367000,246128.15,7.65,good,199009.43,3.8,regular,ARM,
    12,23309 Pinetree Road,Birmingham,AL,78995,Sep 09 2012
    433905,33,277705,566909.04,7.65,default,209880.43,3.8,Jumbo,ARM,33,905
    Pacific Coast Highway,Malibu,CA,90254,Dec 01 2021

    Each line of comma-separated info is separated by a \n character. Here
    is my code:
    **************************************************************************
    filecontent_p = filecontent; //set the pointer to index 0 of array

    filecontent_p = strtok(filecontent, "\n"); //skip
    first line
    printf("filecontent_p = %s\n", filecontent_p);

    n=0;

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].loan_id = atoi(filecontent_p);
    printf("loan[%i].loan_id = %i\n", n,
    newloan[n].loan_id);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].lender_id = atoi(filecontent_p);
    printf("loan[%i].lender_id = %i\n", n,
    newloan[n].lender_id);

    AND ON AND ON...
    ****************************************************************************
    filecontent_p is a pointer to my large array that will store the
    information. I am able to read the first line containing the header
    information using the first strtok() command. Any subsequent reads are
    not possible. All the following filecontent_p reads after I read in
    the header line result in NULLs. Where is the pointer pointing to that
    it results in NULL reads? I thought it should point to the first
    character of the next line that contains info. Any help would be
    great. thanks!
    , Nov 4, 2007
    #1
    1. Advertising

  2. "" <> writes:

    > Hi guys, I'm trying to figure out the problem with my C code below
    > implementing strtok().


    "using" not "implementing".

    > filecontent_p is a pointer to my large array that will store the
    > information. I am able to read the first line containing the header
    > information using the first strtok() command. Any subsequent reads are
    > not possible.


    Then you probably did not post the part that matters. You showed one
    line being processed -- I'd want to see how the data was read and how
    you move on to the next line. In fact, I'd prefer to see a small,
    compilable, example that shows the problem. Just writing such a test
    case might allow you to find the error you self.

    --
    Ben.
    Ben Bacarisse, Nov 4, 2007
    #2
    1. Advertising

  3. Guest

    Sure, here is the compilable program. If you save the info I provided
    above (starting with "loan_id") into a CSV file and rename it
    newloans.csv, the program will open and attempt to process it.
    *********************************************
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>

    int main(int argc, char *argv[])
    {
    //VARIABLE DECLARATIONS

    char input; //command line input by the user
    char filecontent[1000]; //contents read from file
    char * filecontent_p; //pointer to filecontent[] array
    char * filename; //pointer filename entered by user (used by
    strtok())

    int * loan_id_p;

    int n;

    //struct loan containts all the loan information retrieved from
    file
    struct loandata
    {
    int loan_id;
    int lender_id;
    int borrower_id;
    float principle_amt;
    float interest_rate;
    char loan_status[10];
    float loan_balance;
    float fco_discount;
    char loan_amt_type[20];
    char int_rate_type[6];
    unsigned short int risk;
    char property_addr[50];
    char property_city[25];
    char property_state[2];
    int property_zip;
    char settlement_date[30];
    unsigned short int inbundle;
    };

    struct loandata newloan[50]; //array of structs for new
    loans
    struct loandata updateloan[50]; //array of structs for loan
    updates

    for(;;)
    {
    printf("***************************************************\n");
    printf("LAS LoanReport.exe\n");
    printf("Unauthorized Access into LAS System is prohibited\n");
    printf("***************************************************\n");
    printf("n = Create new loans\nu = Update existing loans\nx = Exit
    \n");
    printf("Please enter n, u, or x: ");

    input = fgetc(stdin);

    if(input != ' ')
    {

    if (input == 'X' || input == 'x')
    {
    printf("EXITING program...\n");
    return(1);
    }

    else if(input == 'N' || input == 'n')
    {
    printf("\nAttempting to open file newloans.csv\n", filename);

    FILE * f_read;

    //attempt to open file in READ ONLY mode
    f_read = fopen("newloans.csv", "r");

    if(!f_read) //Open operation failed
    {
    printf("Cannot open file newloans.csv...\n");
    printf("Please make sure file is in current directory\n");
    }

    else if(f_read)
    {
    printf("File newloans.csv opened successfully.");
    printf("Extracting data...\n");

    while(fgets(filecontent, sizeof(filecontent), f_read) !=
    NULL)
    {
    printf("INITIAL filecontent_p = %s\n", filecontent_p);

    filecontent_p = filecontent; //set the pointer to
    index 0 of array

    filecontent_p = strtok(filecontent, "\n"); //skip
    first line
    printf("filecontent_p = %s\n", filecontent_p);

    n=0;
    //for(n = 0; n < 3; n++)
    //{
    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].loan_id = atoi(filecontent_p);
    printf("loan[%i].loan_id = %i\n", n,
    newloan[n].loan_id);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].lender_id = atoi(filecontent_p);
    printf("loan[%i].lender_id = %i\n", n,
    newloan[n].lender_id);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].borrower_id = atoi(filecontent_p);
    printf("loan[%i].borrower_id = %i\n", n,
    newloan[n].borrower_id);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].principle_amt = atof(filecontent_p);
    printf("loan[%i].principle_amt = %8.2f\n", n,
    newloan[n].principle_amt);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].interest_rate = atof(filecontent_p);
    printf("loan[%i].interest_rate = %2.2f\n", n,
    newloan[n].interest_rate);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    strcpy(newloan[n].loan_status, filecontent_p);
    printf("loan[%i].loan_status = %s\n", n,
    newloan[n].loan_status);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].loan_balance = atof(filecontent_p);
    printf("loan[%i].loan_balance = %8.2f\n", n,
    newloan[n].loan_balance);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].fco_discount = atof(filecontent_p);
    printf("loan[%i].fco_discount = %2.2f\n", n,
    newloan[n].fco_discount);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    strcpy(newloan[n].loan_amt_type, filecontent_p);
    printf("loan[%i].loan_amt_type = %s\n", n,
    newloan[n].loan_amt_type);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    strcpy(newloan[n].int_rate_type, filecontent_p);
    printf("loan[%i].int_rate_type = %s\n", n,
    newloan[n].int_rate_type);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].risk = atoi(filecontent_p);
    printf("loan[%i].risk = %i\n", n, newloan[n].risk);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    strcpy(newloan[n].property_addr, filecontent_p);
    printf("loan[%i].property_addr = %s\n", n,
    newloan[n].property_addr);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    strcpy(newloan[n].property_city, filecontent_p);
    printf("loan[%i].property_city = %s\n", n,
    newloan[n].property_city);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    strcpy(newloan[n].property_state, filecontent_p);
    printf("loan[%i].property_state = %s\n", n,
    newloan[n].property_state);

    filecontent_p = strtok(NULL, ",");
    printf("filecontent_p = %s\n", filecontent_p);
    newloan[n].property_zip = atoi(filecontent_p);
    printf("loan[%i].property_zip = %i\n", n,
    newloan[n].property_zip);

    filecontent_p = strtok(NULL, ",\n");
    printf("filecontent_p = %s\n", filecontent_p);
    strcpy(newloan[n].settlement_date, filecontent_p);
    printf("loan[%i].settlement_date = %s\n", n,
    newloan[n].settlement_date);

    //} //end of for loop

    //if ((strcmp(filecontent_p,"loan_id")) == 0)
    //{
    //filecontent_p = strtok(NULL, "\n");
    //loan.loan_id = atoi(filecontent_p);
    //}

    } //end of while loop (file read)

    fclose(f_read); //Close the file after reading it


    }
    } //end of forever loop
    }
    }
    }
    , Nov 4, 2007
    #3
  4. "" <> writes:

    > Sure, here is the compilable program. If you save the info I provided
    > above (starting with "loan_id") into a CSV file and rename it
    > newloans.csv, the program will open and attempt to process it.


    You are not making it easy. Both the code and the data have wrapped
    lines. Both need fixing so I can compiler and test...

    > *********************************************
    > #include <stdio.h>
    > #include <stdlib.h>
    > #include <string.h>
    >
    > int main(int argc, char *argv[])
    > {
    > //VARIABLE DECLARATIONS
    >
    > char input; //command line input by the user
    > char filecontent[1000]; //contents read from file
    > char * filecontent_p; //pointer to filecontent[] array
    > char * filename; //pointer filename entered by user (used by
    > strtok())
    >
    > int * loan_id_p;
    >
    > int n;
    >
    > //struct loan containts all the loan information retrieved from
    > file
    > struct loandata
    > {
    > int loan_id;
    > int lender_id;
    > int borrower_id;
    > float principle_amt;
    > float interest_rate;
    > char loan_status[10];
    > float loan_balance;
    > float fco_discount;
    > char loan_amt_type[20];
    > char int_rate_type[6];
    > unsigned short int risk;
    > char property_addr[50];
    > char property_city[25];
    > char property_state[2];
    > int property_zip;
    > char settlement_date[30];
    > unsigned short int inbundle;
    > };
    >
    > struct loandata newloan[50]; //array of structs for new
    > loans
    > struct loandata updateloan[50]; //array of structs for loan
    > updates
    >
    > for(;;)
    > {
    > printf("***************************************************\n");
    > printf("LAS LoanReport.exe\n");
    > printf("Unauthorized Access into LAS System is prohibited\n");
    > printf("***************************************************\n");
    > printf("n = Create new loans\nu = Update existing loans\nx = Exit
    > \n");
    > printf("Please enter n, u, or x: ");
    >
    > input = fgetc(stdin);


    You need to read any remaining input up to a newline character.

    >
    > if(input != ' ')
    > {
    >
    > if (input == 'X' || input == 'x')
    > {
    > printf("EXITING program...\n");
    > return(1);
    > }
    >
    > else if(input == 'N' || input == 'n')
    > {
    > printf("\nAttempting to open file newloans.csv\n", filename);


    No %s for the filename!

    >
    > FILE * f_read;
    >
    > //attempt to open file in READ ONLY mode
    > f_read = fopen("newloans.csv", "r");
    >
    > if(!f_read) //Open operation failed
    > {
    > printf("Cannot open file newloans.csv...\n");
    > printf("Please make sure file is in current directory\n");
    > }
    >
    > else if(f_read)
    > {
    > printf("File newloans.csv opened successfully.");
    > printf("Extracting data...\n");
    >
    > while(fgets(filecontent, sizeof(filecontent), f_read) !=
    > NULL)
    > {
    > printf("INITIAL filecontent_p = %s\n", filecontent_p);


    Boom. filecontent_p not initialised. Is this really the program you
    run? If so, you are unlucky that this ran. Al bets are off from here
    on.

    >
    > filecontent_p = filecontent; //set the pointer to
    > index 0 of array
    >
    > filecontent_p = strtok(filecontent, "\n"); //skip
    > first line
    > printf("filecontent_p = %s\n", filecontent_p);
    >
    > n=0;
    > //for(n = 0; n < 3; n++)
    > //{
    > filecontent_p = strtok(NULL, ",");


    You are misunderstanding what strtok does. Having found token
    delimited by a '\n' you won't now be able to find an earlier on
    delimited by ','. Delete the earlier call and replace the NULL with
    filecontent_p and you will get much further.

    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].loan_id = atoi(filecontent_p);


    atoi is unreliable on overflow. strtol (and friends) are to be
    preferred.

    > printf("loan[%i].loan_id = %i\n", n,
    > newloan[n].loan_id);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].lender_id = atoi(filecontent_p);
    > printf("loan[%i].lender_id = %i\n", n,
    > newloan[n].lender_id);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].borrower_id = atoi(filecontent_p);
    > printf("loan[%i].borrower_id = %i\n", n,
    > newloan[n].borrower_id);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].principle_amt = atof(filecontent_p);
    > printf("loan[%i].principle_amt = %8.2f\n", n,
    > newloan[n].principle_amt);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].interest_rate = atof(filecontent_p);
    > printf("loan[%i].interest_rate = %2.2f\n", n,
    > newloan[n].interest_rate);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > strcpy(newloan[n].loan_status, filecontent_p);
    > printf("loan[%i].loan_status = %s\n", n,
    > newloan[n].loan_status);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].loan_balance = atof(filecontent_p);
    > printf("loan[%i].loan_balance = %8.2f\n", n,
    > newloan[n].loan_balance);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].fco_discount = atof(filecontent_p);
    > printf("loan[%i].fco_discount = %2.2f\n", n,
    > newloan[n].fco_discount);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > strcpy(newloan[n].loan_amt_type, filecontent_p);
    > printf("loan[%i].loan_amt_type = %s\n", n,
    > newloan[n].loan_amt_type);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > strcpy(newloan[n].int_rate_type, filecontent_p);
    > printf("loan[%i].int_rate_type = %s\n", n,
    > newloan[n].int_rate_type);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].risk = atoi(filecontent_p);
    > printf("loan[%i].risk = %i\n", n, newloan[n].risk);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > strcpy(newloan[n].property_addr, filecontent_p);
    > printf("loan[%i].property_addr = %s\n", n,
    > newloan[n].property_addr);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > strcpy(newloan[n].property_city, filecontent_p);
    > printf("loan[%i].property_city = %s\n", n,
    > newloan[n].property_city);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > strcpy(newloan[n].property_state, filecontent_p);
    > printf("loan[%i].property_state = %s\n", n,
    > newloan[n].property_state);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].property_zip = atoi(filecontent_p);
    > printf("loan[%i].property_zip = %i\n", n,
    > newloan[n].property_zip);
    >
    > filecontent_p = strtok(NULL, ",\n");
    > printf("filecontent_p = %s\n", filecontent_p);
    > strcpy(newloan[n].settlement_date, filecontent_p);
    > printf("loan[%i].settlement_date = %s\n", n,
    > newloan[n].settlement_date);
    >
    > //} //end of for loop
    >
    > //if ((strcmp(filecontent_p,"loan_id")) == 0)
    > //{
    > //filecontent_p = strtok(NULL, "\n");
    > //loan.loan_id = atoi(filecontent_p);
    > //}
    >
    > } //end of while loop (file read)
    >
    > fclose(f_read); //Close the file after reading it
    >
    >
    > }
    > } //end of forever loop
    > }
    > }
    > }


    You need to break this up into helpful functions or you will find your
    self with a huge unmanageable mess very soon. Using consistent
    indentation will help your readers (and yourself).

    --
    Ben.
    Ben Bacarisse, Nov 4, 2007
    #4
  5. CBFalconer Guest

    Ben Bacarisse wrote:
    > "" <> writes:
    >
    >> Sure, here is the compilable program. If you save the info I
    >> provided above (starting with "loan_id") into a CSV file and
    >> rename it newloans.csv, the program will open and attempt to
    >> process it.

    >
    > You are not making it easy. Both the code and the data have
    > wrapped lines. Both need fixing so I can compiler and test...


    He also needs to correct the comments for Usenet publication. //
    comments don't withstand line wrapping.

    --
    Chuck F (cbfalconer at maineline dot net)
    <http://cbfalconer.home.att.net>
    Try the download section.



    --
    Posted via a free Usenet account from http://www.teranews.com
    CBFalconer, Nov 4, 2007
    #5
  6. On Sun, 04 Nov 2007 00:24:30 -0000, ""
    <> wrote:

    >Hi guys, I'm trying to figure out the problem with my C code below
    >implementing strtok(). I am trying to read in a CSV text file which
    >contains information separated by commas and \n newlines. In the CSV
    >file, the first line is filled with header info that I don't want to
    >store in my array. After the header info is the useful info, for
    >example:
    >
    >loan_id,lender_id,borrower_id,principle_amt,interest_rate,loan_status,loan_balance,fco_discount,loan_amt_type,int_rate_type,risk,property_addr,property_city,property_st,property_zip,settlement_date
    >2445,44490,3332000,344123.22,5.9,default,301223.89,1.2,Regular,Fixed,
    >22,204 Main Street,Baltimore,MD,21076,Mar 30 2009
    >10446,1490,367000,246128.15,7.65,good,199009.43,3.8,regular,ARM,
    >12,23309 Pinetree Road,Birmingham,AL,78995,Sep 09 2012
    >433905,33,277705,566909.04,7.65,default,209880.43,3.8,Jumbo,ARM,33,905
    >Pacific Coast Highway,Malibu,CA,90254,Dec 01 2021
    >
    >Each line of comma-separated info is separated by a \n character. Here
    >is my code:
    >**************************************************************************
    >filecontent_p = filecontent; //set the pointer to index 0 of array
    >
    > filecontent_p = strtok(filecontent, "\n"); //skip
    >first line
    > printf("filecontent_p = %s\n", filecontent_p);
    >
    > n=0;
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].loan_id = atoi(filecontent_p);
    > printf("loan[%i].loan_id = %i\n", n,
    >newloan[n].loan_id);
    >
    > filecontent_p = strtok(NULL, ",");
    > printf("filecontent_p = %s\n", filecontent_p);
    > newloan[n].lender_id = atoi(filecontent_p);
    > printf("loan[%i].lender_id = %i\n", n,
    >newloan[n].lender_id);
    >
    >AND ON AND ON...
    >****************************************************************************
    >filecontent_p is a pointer to my large array that will store the
    >information. I am able to read the first line containing the header
    >information using the first strtok() command. Any subsequent reads are
    >not possible. All the following filecontent_p reads after I read in
    >the header line result in NULLs. Where is the pointer pointing to that


    I copied your code into a main function and initialized the array as
    you described. All three calls to strtok worked as expected,
    producing one long string of text and two short strings of digits.
    Your problem has to be in something you are not showing us.

    >it results in NULL reads? I thought it should point to the first


    strtok did not return NULL to me. Are you sure your array contains
    all the lines you think it does? In my test I just initialized it
    with the data you provided using adjacent string literals and adding
    in the implied '\n' whenever a ',' was missing. I expect in your code
    you are reading the data from a file. Why don't you print out the
    entire array before the first call to strtok to be sure you have
    everything.

    >character of the next line that contains info. Any help would be


    On the first call to strtok, the return value points to the beginning
    of the array. On a subsequent call, it points to the first
    non-delimiter following the delimiter which terminated the previous
    call.

    >great. thanks!



    Remove del for email
    Barry Schwarz, Nov 5, 2007
    #6
  7. Guest

    Thanks for the tips guys. I'll try these suggestions and let you know
    how it goes.

    Just to clarify, I would like to take each line in the CSV file (after
    the first line, hence I strtok() to skip it via the \n character) and
    store it in a separate data structure. Thus I use an array of structs.
    Each line of text represents a separate loan and is stored into a
    separate struct.
    , Nov 5, 2007
    #7
  8. "" <> writes:

    > Thanks for the tips guys. I'll try these suggestions and let you know
    > how it goes.
    >
    > Just to clarify, I would like to take each line in the CSV file (after
    > the first line, hence I strtok() to skip it via the \n character) and
    > store it in a separate data structure.


    Using strtok() to skip the first line won't work since you only have
    one line. fgets just reads a line -- sometimes less but never more.
    If you read the whole data file into a buffer, then something like your
    pattern will work (though I'd still use a simpler method to skip over
    the first line). It did not occur to me that it was fgets not strtok that
    you had misused!

    I would advice that you revert to the pattern I *thought* you were
    trying to write: read a line with fgets(), split it using strtok(),
    repeat. This is much more normal in C programs.

    --
    Ben.
    Ben Bacarisse, Nov 5, 2007
    #8
  9. CBFalconer Guest

    Ben Bacarisse wrote:
    > "" <> writes:
    >
    >> Thanks for the tips guys. I'll try these suggestions and let you
    >> know how it goes.
    >>
    >> Just to clarify, I would like to take each line in the CSV file
    >> (after the first line, hence I strtok() to skip it via the \n
    >> character) and store it in a separate data structure.

    >
    > Using strtok() to skip the first line won't work since you only
    > have one line. fgets just reads a line -- sometimes less but
    > never more. If you read the whole data file into a buffer, then
    > something like your pattern will work (though I'd still use a
    > simpler method to skip over the first line). It did not occur to
    > me that it was fgets not strtok that you had misused!
    >
    > I would advice that you revert to the pattern I *thought* you
    > were trying to write: read a line with fgets(), split it using
    > strtok(), repeat. This is much more normal in C programs.


    Another option is the following routine, which avoids some of the
    awkwardness of strtok. The source includes a testing routine. All
    in standard C. Note that tknsplit is NOT a direct replacement for
    strtok.

    /* ------- file tknsplit.c ----------*/
    #include "tknsplit.h"

    /* copy over the next tkn from an input string, after
    skipping leading blanks (or other whitespace?). The
    tkn is terminated by the first appearance of tknchar,
    or by the end of the source string.

    The caller must supply sufficient space in tkn to
    receive any tkn, Otherwise tkns will be truncated.

    Returns: a pointer past the terminating tknchar.

    This will happily return an infinity of empty tkns if
    called with src pointing to the end of a string. Tokens
    will never include a copy of tknchar.

    A better name would be "strtkn", except that is reserved
    for the system namespace. Change to that at your risk.

    released to Public Domain, by C.B. Falconer.
    Published 2006-02-20. Attribution appreciated.
    Revised 2006-06-13 2007-05-26 (name)
    */

    const char *tknsplit(const char *src, /* Source of tkns */
    char tknchar, /* tkn delimiting char */
    char *tkn, /* receiver of parsed tkn */
    size_t lgh) /* length tkn can receive */
    /* not including final '\0' */
    {
    if (src) {
    while (' ' == *src) src++;

    while (*src && (tknchar != *src)) {
    if (lgh) {
    *tkn++ = *src;
    --lgh;
    }
    src++;
    }
    if (*src && (tknchar == *src)) src++;
    }
    *tkn = '\0';
    return src;
    } /* tknsplit */

    #ifdef TESTING
    #include <stdio.h>

    #define ABRsize 6 /* length of acceptable tkn abbreviations */

    /* ---------------- */

    static void showtkn(int i, char *tok)
    {
    putchar(i + '1'); putchar(':');
    puts(tok);
    } /* showtkn */

    /* ---------------- */

    int main(void)
    {
    char teststring[] = "This is a test, ,, abbrev, more";

    const char *t, *s = teststring;
    int i;
    char tkn[ABRsize + 1];

    puts(teststring);
    t = s;
    for (i = 0; i < 4; i++) {
    t = tknsplit(t, ',', tkn, ABRsize);
    showtkn(i, tkn);
    }

    puts("\nHow to detect 'no more tkns' while truncating");
    t = s; i = 0;
    while (*t) {
    t = tknsplit(t, ',', tkn, 3);
    showtkn(i, tkn);
    i++;
    }

    puts("\nUsing blanks as tkn delimiters");
    t = s; i = 0;
    while (*t) {
    t = tknsplit(t, ' ', tkn, ABRsize);
    showtkn(i, tkn);
    i++;
    }
    return 0;
    } /* main */

    #endif
    /* ------- end file tknsplit.c ----------*/

    --
    Chuck F (cbfalconer at maineline dot net)
    <http://cbfalconer.home.att.net>
    Try the download section.



    --
    Posted via a free Usenet account from http://www.teranews.com
    CBFalconer, Nov 5, 2007
    #9
  10. On Mon, 05 Nov 2007 17:37:09 +0000, Ben Bacarisse
    <> wrote:

    > "" <> writes:
    >
    > > Thanks for the tips guys. I'll try these suggestions and let you know
    > > how it goes.
    > >
    > > Just to clarify, I would like to take each line in the CSV file (after
    > > the first line, hence I strtok() to skip it via the \n character) and
    > > store it in a separate data structure.

    >
    > Using strtok() to skip the first line won't work since you only have
    > one line. fgets just reads a line -- sometimes less but never more.


    Agree.

    > If you read the whole data file into a buffer, then something like your
    > pattern will work (though I'd still use a simpler method to skip over


    Depends on what you mean by 'like'. Something like:
    /* outline only, 'obvious' decls and error checking omitted */
    bigbuf [ fread (bigbuf, 1, ALL, fp) ] = '\0';
    for( line=strtok(bigbuf,"\n"); line; line=strtok(NULL,"\n") ){
    for( tkn=strtok(line,","); tkn; tkn=strtok(NULL,",") ){
    ...
    }
    }
    WON'T WORK; the two 'levels' of strtok try to use the same, shared,
    static, state. You can use strtok_r with explicit state, or since here
    we're looking for only one char at each level, just strchr, e.g.:
    for( line=bigbuf; nl=strchr(line,'\n'); line=nl+1 ){
    *nl = '\0';
    for( tkn=line; nt=strchr(tkn,','); tkn=nt+1 ){
    *nt = '\0'; process tkn
    }
    process last tkn
    }
    process last partial line if supported
    or to make last partial line easier:
    for( line=bigbuf; *line; line=nl ){
    nl=strchr(line,'\n');
    if( nl ) *nl++ = '\0';
    else nl=line+strlen(line);
    process line as above or otherwise
    }
    or a mixture e.g. strchr for lines and strtok for tokens within line.
    (Personally, given that the lengths can be available when needed, I
    would probably make use of them with memchr, but that's minor.)

    > the first line). It did not occur to me that it was fgets not strtok that
    > you had misused!
    >
    > I would advice that you revert to the pattern I *thought* you were
    > trying to write: read a line with fgets(), split it using strtok(),
    > repeat. This is much more normal in C programs.


    Agree again. This is not only more common, it avoids the need to have,
    or allocate, or otherwise manage, a buffer large enough for the whole
    file. You do need a whole line, but for many reasonable(?) data
    formats, lines are never hugely long but files may be.

    However, fgets includes the terminating \n in the buffer (if it
    reached it). If you don't want that character included in the last
    token either trim the line (just) after reading:
    while( fgets (line, 1, max, fp) != NULL ){
    /* note strlen(line) always > 0 here, but assert if you like */
    if( line[strlen(line)-1] == '\n' ) line[strlen(line)-1] = '\0';
    else line_overlong_or_lastline_partial_do_something;
    ...
    }
    or if using strtok within the line include \n in the delim set.

    - formerly david.thompson1 || achar(64) || worldnet.att.net
    David Thompson, Nov 19, 2007
    #10
  11. CBFalconer Guest

    David Thompson wrote:
    >

    .... snip ...
    >
    > However, fgets includes the terminating \n in the buffer (if it
    > reached it). If you don't want that character included in the
    > last token either trim the line (just) after reading:


    Again, here is a place where ggets [non-std] shines. Since it
    always reads a complete line, it can afford to always delete the
    terminal '\n'. The only possible anomaly is on the last line of a
    file, if the file has been constructed without a final '\n'. The
    action effectively inserts it on read.

    --
    Chuck F (cbfalconer at maineline dot net)
    <http://cbfalconer.home.att.net>
    Try the download section.



    --
    Posted via a free Usenet account from http://www.teranews.com
    CBFalconer, Nov 20, 2007
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ern

    Using strcmp() after using strtok()

    ern, Sep 21, 2005, in forum: C Programming
    Replies:
    3
    Views:
    907
    Default User
    Sep 22, 2005
  2. Replies:
    20
    Views:
    3,124
    Ben Bacarisse
    Feb 18, 2006
  3. BGP
    Replies:
    12
    Views:
    692
    Default User
    Jun 21, 2005
  4. Replies:
    5
    Views:
    454
  5. wreckingcru
    Replies:
    11
    Views:
    1,205
    red floyd
    Feb 1, 2006
Loading...

Share This Page