reading the buffer in chunks

[Sean]

Hi,
I have a variable declared as the following:

char * buf;

I then get a size of a file and allocate memory and point the buf to
it. Here is where I am having a problem. I want to know how I can read
the buf in chunks... so here is a description of what i am looking
for.:

while the end of the buffer hasn't been reached{
send 100 characters of the buf to some function
then send the next 100 ...
}repeat this until the end of the buf

I was originally trying to do:
while (!feof(filename)){
read(filename, buf, 100,0);
process(buf);
}

but for some reason the look never ended. I work with both binary and
ascii files so I am not sure if that has anything to do with.

Any suggestions for the buffer problem??

Thanks

J

 

[gamediaceo]

Your pseudocode looks fine, at least. Could you post some of your real
code?

This is how I would do it in C:

FILE* fp = fopen("file.txt","rt");

while (!feof(fp)) {
char* buffer = new char[100];
size_t count = fread(buffer,sizeof(char),100,fp);

//if count < 100, then the end of file was reached, for sure.
process(buffer);
delete buffer;
}

It could be that the order of your parameters are wrong. Another
technique is to see if the file pointer has reached the end of the
file using ftell() and comparing that to the calculated file size:

fseek(fp,0,SEEK_END);
file_size = ftell(fp);
//then rewind to go back to the beginning of the file
rewind(fp);


Cheers,
Henry

 

[Jerry Coffin]

Hi,
I have a variable declared as the following:

char * buf;

I then get a size of a file and allocate memory and point the buf to
it. Here is where I am having a problem. I want to know how I can read
the buf in chunks... so here is a description of what i am looking
for.:

If you're going to read chunks of fixed size, you might as well define
buf as an array so you don't have to allocate and free the memory
manually.

while the end of the buffer hasn't been reached{
send 100 characters of the buf to some function
then send the next 100 ...
}repeat this until the end of the buf

I was originally trying to do:
while (!feof(filename)){

A loop like 'while (!feof...' is almost always a mistake -- this should
be covered in the FAQ. Your use of "filename" implies that you're
passing the name of the file. When you're working with a file, you
almost always need to open the file, then work with the opened file
instead of the file name. You can use either an ifstream or use fopen to
open the file (it returns a FILE *).

read(filename, buf, 100,0);
process(buf);

This part looks perfectly reasonable.

 

[=?iso-8859-1?q?Erik_Wikstr=F6m?=]

Your pseudocode looks fine, at least. Could you post some of your real
code?

This is how I would do it in C:

FILE* fp = fopen("file.txt","rt");

while (!feof(fp)) {
char* buffer = new char[100];
size_t count = fread(buffer,sizeof(char),100,fp);

//if count < 100, then the end of file was reached, for sure.
process(buffer);

You might want to change this to 'process(buffer, count);' to make
sure you don't process invalid data. While the buffer will always be
of the same size it might only be the first byte that is valid, the
rest is junk left from the last read.

 

[Thomas J. Gritzan]

(e-mail address removed) says... [...]

while the end of the buffer hasn't been reached{
send 100 characters of the buf to some function
then send the next 100 ...
}repeat this until the end of the buf

I was originally trying to do:
while (!feof(filename)){

A loop like 'while (!feof...' is almost always a mistake -- this should
be covered in the FAQ.

It is:
http://www.parashift.com/c++-faq-lite/input-output.html#faq-15.5

while (!std::cin.eof()) // wrong way
{
std::cin >> x;
// Work with x ...
}

The problem is, that the eof state is set only after trying to read past
the end of file.

I guess it's the same with the C functions.

 

[Default User]

(e-mail address removed) says... [...]

while the end of the buffer hasn't been reached{
send 100 characters of the buf to some function
then send the next 100 ...
}repeat this until the end of the buf

I was originally trying to do:
while (!feof(filename)){

A loop like 'while (!feof...' is almost always a mistake -- this
should be covered in the FAQ.

It is:
http://www.parashift.com/c++-faq-lite/input-output.html#faq-15.5

while (!std::cin.eof()) // wrong way
{
std::cin >> x;
// Work with x ...
}

The problem is, that the eof state is set only after trying to read
past the end of file.

I guess it's the same with the C functions.

Yes, here's the corresponding C FAQ entry:

<http://c-faq.com/stdio/feof.html>




Brian