Fread problem

J

Jack

Hi,

I got a strange problem with Fread:

It crashes my program from time to time, in one of its internal memcpy
calls.

I managed to bypass this bug one time by creating a bigger file buffer,
but it's just a patch...

Does anyone knows anything about it?
Is there a clean way to reset the file buffers?
Can this be compiler dependent (gcc 4.1) - maybe a compiler bug?

Thanks for any help
 
I

Ian Collins

Hi,

I got a strange problem with Fread:

It crashes my program from time to time, in one of its internal memcpy
calls.

Assuming it is fread(), it is very unlikely the function is the cause of
the problem. The much more likely cause is something else in your
program stomping on the bugger space.

Run the program with an bounds checker (such is Valgrind in Linux).
 
J

John Gordon

In said:
I got a strange problem with Fread:
It crashes my program from time to time, in one of its internal memcpy
calls.

When things like this happen it is nearly always because of a memory
corruption bug elsewhere in your program, not because of a bug in a
library function like fread().

C is, unfortunately, a fragile language. If you copy more to a buffer
than it can hold, or call free() on a bad value, or call free() twice on
the same value, you've just corrupted the memory space of your program and
anything can happen from that point onward, including crashes in library
functions such as fread().

If your program is a simple one, go over it again and try to identify any
areas where you might have committed the above errors. If it's a complex
program, you might use a memory analyzer such as efence or valgrind.
 
J

Jack

When things like this happen it is nearly always because of a memory
corruption bug elsewhere in your program, not because of a bug in a
library function like fread().

C is, unfortunately, a fragile language. If you copy more to a buffer
than it can hold, or call free() on a bad value, or call free() twice on
the same value, you've just corrupted the memory space of your program
and anything can happen from that point onward, including crashes in
library functions such as fread().

If your program is a simple one, go over it again and try to identify
any areas where you might have committed the above errors. If it's a
complex program, you might use a memory analyzer such as efence or
valgrind.

Yes...
Actually, I think it's a memory error somewhere that causes really
indirectly this bug.
But I find it strange that even a memory error makes a fread with proper
parameters ( I do check this ) fail.

I've heard somewhere one can play with the file buffer, so I tried it,
and put MY file buffer, with the Setbuff() function. And it didn't crash
anymore, but the read couldn't read all the files. Some bytes in the
middle were missing (I think it is when I resume the reading, after
having processed the first part)...

Last thing, this Fread occurs in a big source file... Created by lex /
yacc.

The file is big, not easy to understand, so difficult to debug
: (
 
D

Dr Nick

Jack said:
Yes...
Actually, I think it's a memory error somewhere that causes really
indirectly this bug.
But I find it strange that even a memory error makes a fread with proper
parameters ( I do check this ) fail.

Somewhere the library and the operating system have to store things like
the details of the underlying file, the position in it etc. When you
trample over memory you can overwrite this.

In particular, many systems get the buffers for the stdio streams from
the same memory pool as you get with malloc. So do something wrong with
a block you've got from malloc and you can really mess the streams up.

And vice versa. I once managed to get malloc to return "null" by
previously fclosing the same file pointer twice.
I've heard somewhere one can play with the file buffer, so I tried it,
and put MY file buffer, with the Setbuff() function. And it didn't crash
anymore, but the read couldn't read all the files. Some bytes in the
middle were missing (I think it is when I resume the reading, after
having processed the first part)...

That's not incompatible with what I wrote above.
 
I

Ian Collins

Yes...
Actually, I think it's a memory error somewhere that causes really
indirectly this bug.

So try valgrind...
But I find it strange that even a memory error makes a fread with proper
parameters ( I do check this ) fail.

Why strange? All the memory is in one address space, so any part of the
program can mess with the buffers being used by stdio.
 
J

Joe Pfeiffer

Jack said:
Yes...
Actually, I think it's a memory error somewhere that causes really
indirectly this bug.
But I find it strange that even a memory error makes a fread with proper
parameters ( I do check this ) fail.

Why should this be any surprise at all? It looks like you're corrupting
memory, so the buffers fread() depends on are failing.
I've heard somewhere one can play with the file buffer, so I tried it,
and put MY file buffer, with the Setbuff() function. And it didn't crash
anymore, but the read couldn't read all the files. Some bytes in the
middle were missing (I think it is when I resume the reading, after
having processed the first part)...

Sounds more and more like a wild pointer someplace.
Last thing, this Fread occurs in a big source file... Created by lex /
yacc.

The file is big, not easy to understand, so difficult to debug
: (

Start by assuming yacc and lex are working correctly, do your debugging
on the yacc and lex source files, and on whatever utility C functions
you've defined. lex and yacc output isn't meant to be human readable;
trying to understand what they're putting out is really useless for
debugging purposes.
 
J

John Gordon

In said:
But I find it strange that even a memory error makes a fread with proper
parameters ( I do check this ) fail.

Unfortunately, it has nothing to do with the parameters to fread() being
right or wrong. Once you corrupt the underlying memory, any function can
go wrong.

Here's an analogy:

Imagine you're a cook at a restaurant. The restaurant has lots of recipes
that have been tested and you know they work.

One day, a customer orders pancakes. You cook them up and serve them.
However, the cook on the previous shift didn't seal the flour container
properly, so the flour had weevils in it. The pancakes are rotten and
the customer gets sick.

Nothing was wrong with the recipe (the function arguments); you did
everything right. But an ingredient (memory) was corrupted and so the
function crashes (pancakes are rotten).
 
J

Jack

Somewhere the library and the operating system have to store things like
the details of the underlying file, the position in it etc. When you
trample over memory you can overwrite this.

In particular, many systems get the buffers for the stdio streams from
the same memory pool as you get with malloc. So do something wrong with
a block you've got from malloc and you can really mess the streams up.

And vice versa. I once managed to get malloc to return "null" by
previously fclosing the same file pointer twice.


That's not incompatible with what I wrote above.

Thanks for your help. I think I've now found a solution.

After the file is first Fopen'd, I make a backup of the internal buffers.
fp=Fopen(...);
FILE fback;
memcpy(&fback,fp,sizeof(FILE));

then before the fread I restore this buffer,
memcpy(fp,&fback,sizeof(FILE));
Fread(...,fp);

Now there is no segmentation fault, job done.
 
I

Ian Collins

Thanks for your help. I think I've now found a solution.

No, you have simply moved the problem elsewhere.
After the file is first Fopen'd, I make a backup of the internal buffers.
fp=Fopen(...);
FILE fback;
memcpy(&fback,fp,sizeof(FILE));

then before the fread I restore this buffer,
memcpy(fp,&fback,sizeof(FILE));
Fread(...,fp);

Now there is no segmentation fault, job done.

Until the bug you have yet to fix pops up in another random place.
 
J

John Gordon

In said:
Thanks for your help. I think I've now found a solution.
After the file is first Fopen'd, I make a backup of the internal buffers.
fp=Fopen(...);
FILE fback;
memcpy(&fback,fp,sizeof(FILE));
then before the fread I restore this buffer,
memcpy(fp,&fback,sizeof(FILE));
Fread(...,fp);
Now there is no segmentation fault, job done.

If that works, it's purely accidental and I wouldn't count on it.

Your code even has a bug: you're copying sizeof(FILE) bytes, but
fp is a pointer-to-FILE which is probably not the same size.
 
E

Eric Sosman

[...]
Thanks for your help. I think I've now found a solution.

After the file is first Fopen'd, I make a backup of the internal buffers.
fp=Fopen(...);
FILE fback;
memcpy(&fback,fp,sizeof(FILE));

then before the fread I restore this buffer,
memcpy(fp,&fback,sizeof(FILE));
Fread(...,fp);

Now there is no segmentation fault, job done.

When your smoke alarm jolts you awake in the middle of the
night, do you fix the problem by stuffing cotton in your ears
and going back to sleep?

When the BRAKE TROUBLE light on your car's dashboard refuses
to go out, do you put a piece of tape over it?

When you're eating the leftovers from last week's sushi and
find that it smells and tastes rotten, do you pile on extra
wasabi to stun your taste buds?

Do you wear Joo Janta Peril-Sensitive Sunglasses?
 
J

Joe Pfeiffer

Jack said:
Thanks for your help. I think I've now found a solution.

After the file is first Fopen'd, I make a backup of the internal buffers.
fp=Fopen(...);
FILE fback;
memcpy(&fback,fp,sizeof(FILE));

then before the fread I restore this buffer,
memcpy(fp,&fback,sizeof(FILE));
Fread(...,fp);

Now there is no segmentation fault, job done.

That's not a fix. Your underlying bug is still present; your code is
still broken. Anything from adding a new line of code to a new compiler
version to a new version of libc to slightly different input than what
you've tested on so far could make it manifest again at any time.
 
G

Geoff

After the file is first Fopen'd, I make a backup of the internal buffers.
fp=Fopen(...);
FILE fback;
memcpy(&fback,fp,sizeof(FILE));

then before the fread I restore this buffer,
memcpy(fp,&fback,sizeof(FILE));
Fread(...,fp);

Now there is no segmentation fault, job done.

This is not a correct solution. This is a kludge. Your code is still
bugged and it WILL fail again.

Why do you keep capitalizing Fopen and Fread? The standard library
calls are fopen and fread. Are you using someone else's wrapper or
libraries?

You have been advised to use valgrind to find your bug. It appears you
haven't followed that advice and you still have no clue why your code
is failing. Learn to use the toolset. At a minimum you should have
been able to trace back your call stack and find the error in a
debugger.

Post code, your methods are suspect.
 
J

John Gordon

In said:
At a minimum you should have been able to trace back your call stack
and find the error in a debugger.

A call trace won't really help, will it? The memory corruption likely
happened in some previous call and isn't on the stack anymore, which is
exactly why errors like this are hard to find.
Post code, your methods are suspect.

He said it's a large program, so posting it may be problematic.
 
J

James Kuyper

If that works, it's purely accidental and I wouldn't count on it.

Your code even has a bug: you're copying sizeof(FILE) bytes, but
fp is a pointer-to-FILE which is probably not the same size.

If he wanted to copy the pointer, that would imply that he's got several
different errors, not just one. What you're thinking of would be written
as follows:

FILE *fback;
memcpy(&fback, &fp, sizeof fp);

Which is almost (but not quite) the same as the much simpler:

FILE *fback = fp;

He's not copying fp, he's copying the FILE object that fp points at. It
can't be the correct solution to his problem, but it is done correctly.
 
J

John Gordon

In said:
He's not copying fp, he's copying the FILE object that fp points at. It
can't be the correct solution to his problem, but it is done correctly.

Why so it is. My mistake.
 
K

Kleuskes & Moos

Thanks for your help. I think I've now found a solution.

After the file is first Fopen'd, I make a backup of the internal buffers.
fp=Fopen(...);
FILE fback;
memcpy(&fback,fp,sizeof(FILE));

then before the fread I restore this buffer,
memcpy(fp,&fback,sizeof(FILE));
Fread(...,fp);

Now there is no segmentation fault, job done.

In addition to what everyone else has told you, here's a few rules of
thumb:

* A bug is more likely to reside in the code most recently written
instead of a widely used library.
* A bug isn't solved until you know what you did wrong.
* It's always your fault and the mistake you made is most likely very
silly, once you spot it.
 
J

Jack

That's not a fix. Your underlying bug is still present; your code is
still broken. Anything from adding a new line of code to a new compiler
version to a new version of libc to slightly different input than what
you've tested on so far could make it manifest again at any time.

I sure do understand the point everyone's making, and in an ideal world
I'd get right to the heart of this bug. BUT... the customer pays for
their support contract annually in advance, any time we have to spend on
maintenance from that contract is just a cost to our company, so we get
strong pressure from management to get quick 'n' easy fixes out the door.

Anyways, if the problem reoccurs in future, with any luck it'll end up on
some other maintenance programmer's books not mine. :)
 
I

Ian Collins

I sure do understand the point everyone's making, and in an ideal world
I'd get right to the heart of this bug. BUT... the customer pays for
their support contract annually in advance, any time we have to spend on
maintenance from that contract is just a cost to our company, so we get
strong pressure from management to get quick 'n' easy fixes out the door.

Wouldn't the quickest solution have been to use the appropriate tool to
find the bug?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top