reading the buffer in chunks

Discussion in 'C++' started by Sean, Feb 1, 2007.

  1. Sean

    Sean Guest

    Hi,
    I have a variable declared as the following:

    char * buf;

    I then get a size of a file and allocate memory and point the buf to
    it. Here is where I am having a problem. I want to know how I can read
    the buf in chunks... so here is a description of what i am looking
    for.:

    while the end of the buffer hasn't been reached{
    send 100 characters of the buf to some function
    then send the next 100 ...
    }repeat this until the end of the buf

    I was originally trying to do:
    while (!feof(filename)){
    read(filename, buf, 100,0);
    process(buf);
    }

    but for some reason the look never ended. I work with both binary and
    ascii files so I am not sure if that has anything to do with.

    Any suggestions for the buffer problem??

    Thanks

    J
     
    Sean, Feb 1, 2007
    #1
    1. Advertising

  2. Sean

    Guest

    Your pseudocode looks fine, at least. Could you post some of your real
    code?

    This is how I would do it in C:

    FILE* fp = fopen("file.txt","rt");

    while (!feof(fp)) {
    char* buffer = new char[100];
    size_t count = fread(buffer,sizeof(char),100,fp);

    //if count < 100, then the end of file was reached, for sure.
    process(buffer);
    delete buffer;
    }

    It could be that the order of your parameters are wrong. Another
    technique is to see if the file pointer has reached the end of the
    file using ftell() and comparing that to the calculated file size:

    fseek(fp,0,SEEK_END);
    file_size = ftell(fp);
    //then rewind to go back to the beginning of the file
    rewind(fp);


    Cheers,
    Henry

    --------
    http://hamath.blogspot.com - Science, technology, and interesting
    stuff Blog
     
    , Feb 1, 2007
    #2
    1. Advertising

  3. Sean

    Jerry Coffin Guest

    In article <>,
    says...
    > Hi,
    > I have a variable declared as the following:
    >
    > char * buf;
    >
    > I then get a size of a file and allocate memory and point the buf to
    > it. Here is where I am having a problem. I want to know how I can read
    > the buf in chunks... so here is a description of what i am looking
    > for.:


    If you're going to read chunks of fixed size, you might as well define
    buf as an array so you don't have to allocate and free the memory
    manually.

    > while the end of the buffer hasn't been reached{
    > send 100 characters of the buf to some function
    > then send the next 100 ...
    > }repeat this until the end of the buf
    >
    > I was originally trying to do:
    > while (!feof(filename)){


    A loop like 'while (!feof...' is almost always a mistake -- this should
    be covered in the FAQ. Your use of "filename" implies that you're
    passing the name of the file. When you're working with a file, you
    almost always need to open the file, then work with the opened file
    instead of the file name. You can use either an ifstream or use fopen to
    open the file (it returns a FILE *).

    > read(filename, buf, 100,0);
    > process(buf);


    This part looks perfectly reasonable.

    --
    Later,
    Jerry.

    The universe is a figment of its own imagination.
     
    Jerry Coffin, Feb 1, 2007
    #3
  4. On Feb 1, 5:03 am, "" <>
    wrote:
    > Your pseudocode looks fine, at least. Could you post some of your real
    > code?
    >
    > This is how I would do it in C:
    >
    > FILE* fp = fopen("file.txt","rt");
    >
    > while (!feof(fp)) {
    > char* buffer = new char[100];
    > size_t count = fread(buffer,sizeof(char),100,fp);
    >
    > //if count < 100, then the end of file was reached, for sure.
    > process(buffer);


    You might want to change this to 'process(buffer, count);' to make
    sure you don't process invalid data. While the buffer will always be
    of the same size it might only be the first byte that is valid, the
    rest is junk left from the last read.

    > delete buffer;
    >
    > }


    --
    Erik Wikström
     
    =?iso-8859-1?q?Erik_Wikstr=F6m?=, Feb 1, 2007
    #4
  5. Jerry Coffin schrieb:
    > In article <>,
    > says...

    [...]
    >> while the end of the buffer hasn't been reached{
    >> send 100 characters of the buf to some function
    >> then send the next 100 ...
    >> }repeat this until the end of the buf
    >>
    >> I was originally trying to do:
    >> while (!feof(filename)){

    >
    > A loop like 'while (!feof...' is almost always a mistake -- this should
    > be covered in the FAQ.


    It is:
    http://www.parashift.com/c -faq-lite/input-output.html#faq-15.5

    while (!std::cin.eof()) // wrong way
    {
    std::cin >> x;
    // Work with x ...
    }

    The problem is, that the eof state is set only after trying to read past
    the end of file.

    I guess it's the same with the C functions.

    --
    Thomas
    http://www.netmeister.org/news/learn2quote.html
     
    Thomas J. Gritzan, Feb 1, 2007
    #5
  6. Sean

    Default User Guest

    Thomas J. Gritzan wrote:

    > Jerry Coffin schrieb:
    > > In article <>,
    > > says...

    > [...]
    > >> while the end of the buffer hasn't been reached{
    > >> send 100 characters of the buf to some function
    > >> then send the next 100 ...
    > >> }repeat this until the end of the buf
    > > >
    > >> I was originally trying to do:
    > >> while (!feof(filename)){

    > >
    > > A loop like 'while (!feof...' is almost always a mistake -- this
    > > should be covered in the FAQ.

    >
    > It is:
    > http://www.parashift.com/c -faq-lite/input-output.html#faq-15.5
    >
    > while (!std::cin.eof()) // wrong way
    > {
    > std::cin >> x;
    > // Work with x ...
    > }
    >
    > The problem is, that the eof state is set only after trying to read
    > past the end of file.
    >
    > I guess it's the same with the C functions.


    Yes, here's the corresponding C FAQ entry:

    <http://c-faq.com/stdio/feof.html>




    Brian
     
    Default User, Feb 1, 2007
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Martin Marcher

    reading file objects in chunks

    Martin Marcher, Nov 12, 2007, in forum: Python
    Replies:
    1
    Views:
    311
    Marc 'BlackJack' Rintsch
    Nov 12, 2007
  2. Replies:
    4
    Views:
    861
    Tom Anderson
    Jul 11, 2008
  3. ColdStart
    Replies:
    0
    Views:
    575
    ColdStart
    Jul 12, 2010
  4. Ben Johnson
    Replies:
    1
    Views:
    150
    Brian Candler
    Feb 28, 2007
  5. Replies:
    1
    Views:
    365
    Robert Klemme
    Jan 29, 2009
Loading...

Share This Page