fread breaks file descriptors opened in "w" mode.

L

Lénaïc Huard

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all,

For some reasons, somewhere in a program, I'd like, if possible, to quickly
parse a whole file before rewinding it and letting the full analysis start.
My problem is that the FILE* I want do parse has been fopen'ed far away
from where I am and I don't know in which MODE my FILE* has been opened.
And additionally, my FILE* may not be a regular file, but a continuous
stream (pipe), in which case it is not rewindable.

So my program basically behaves like this:

void PreParsing( FILE *my_file )
{
/*
* Test if my FILE * is seekable
*/
if( fseek( my_file, 0, SEEK_SET ) == 0 ) {

/*
* If so, go on for the pre-parsing.
*/
//...
fread( buf, BUF_SIZE, 1, my_file );
//...

/*
* Rewind the file.
*/
if( fseek( my_file, 0, SEEK_SET ) != 0 ) {
// unexpected error since my FILE * was expected to be seekable.
}
}
}

* On both Linux 2.6.xx and Solaris 5.10,

when the my_file is a regular file that has been fopen'ed with "w" as mode,
- the first fseek returns 0 meaning success: my_file is seekable
- the fread returns 0 meaning that there was nothing to read: expected
since a file opened in "w" is truncated, hence is empty, and anyhow, cannot
be read!
- the second fseek returns 0 meaning success: my_file could be rewound.

when the my_file is a pipe that has been fopen'ed with "w" as mode,
- the first fseek returns -1 and errno is set to 29 (Illegal seek).
That's exactly what I expected since a pipe cannot be sought.

Ok. So, on Linux 2.6.xx and Solaris 5.10, my code behaves like I expected
for both regular files and pipes.

* On AIX 5.2,

when the my_file is a pipe that has been fopen'ed with "w" as mode,
- the first fseek returns -1 and errno is set to 29 (Illegal seek).
I'm still OK with that.

but...
when the my_file is a regular file that has been fopen'ed with "w" as mode,
- the first fseek returns 0 meaning success: my_file is seekable
- the fread returns 0 meaning that there was nothing to read: expected
since a file opened in "w" is truncated, hence is empty, and anyhow, cannot
be read!
- the second fseek returns -1 and sets errno to 9 (Bad file number)
I'm a little bit surprised by this error.

In fact, after having had a deeper look on that, it appears to me that a
fread attempt on a FILE * opened in "w" mode breaks it since any subsequent
operation (fseek, fwrite and even fclose !) fails with the error 9 (Bad
file number).
So, on AIX , my function fails to restore the FILE * state at its end.


So, I have a few questions:

- a fread on a FILE * opened in "w" will for sure return 0 item, but is it
really expected that it makes the given FILE * totally unusable even for
fclose !?! This behavior has been observed only on AIX. Linux and Solaris
works.
- given a FILE *, is there a better way to guess in which mode it has been
opened than attempting to read or write it and look at errors ?
- given a FILE *, is there a better way to guess if it can be sought than
attempting a fseek on it ?


PS: I Xposted my problem on both comp.lang.c and comp.unix.aix because I
have no idea whether this is an AIX specific problem or if the C norm
specifies explicitly that fread have unpredictable effects on a write-only
file descriptor. Anyhow, my goal is to find a portable solution that uses
as less platform specific stuff as possible.

Thanks for your advices.
Lénaïc ...still puzzled by AIX behavior...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkkjPpUACgkQjYEjJATS6Bi55ACgj+RbzOhyjDj63G+ciKo0Iy1B
FuoAn3WsnRc69XXDi2KV0Wt4aOFo0Bpf
=In7u
-----END PGP SIGNATURE-----
 
V

vippstar

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Please don't include that crap in your messages...
Hello all,

For some reasons, somewhere in a program, I'd like, if possible, to quickly
parse a whole file before rewinding it and letting the full analysis start.
My problem is that the FILE* I want do parse has been fopen'ed far away
from where I am and I don't know in which MODE my FILE* has been opened.

Well that's a bad thing, if you want truly meaningful results you
should enforce just one mode, binary or text.
And additionally, my FILE* may not be a regular file, but a continuous
stream (pipe), in which case it is not rewindable.

Load it in memory once, and don't bother with the actual file
rewinding.
So my program basically behaves like this:

void PreParsing( FILE *my_file )
{
/*
* Test if my FILE * is seekable
*/
if( fseek( my_file, 0, SEEK_SET ) == 0 ) {

That isn't testing anything really, except for that very fseek call.
/*
* If so, go on for the pre-parsing.
*/
//...
fread( buf, BUF_SIZE, 1, my_file );

I think you probably want fread(buf, 1, BUF_SIZE, my_file);
You should also observe the return value.

* On AIX 5.2,
when the my_file is a regular file that has been fopen'ed with "w" as mode,
- the first fseek returns 0 meaning success: my_file is seekable
- the fread returns 0 meaning that there was nothing to read: expected
since a file opened in "w" is truncated, hence is empty, and anyhow, cannot
be read!
- the second fseek returns -1 and sets errno to 9 (Bad file number)
I'm a little bit surprised by this error.

Why? That's allowed by the standard.
In fact, after having had a deeper look on that, it appears to me that a
fread attempt on a FILE * opened in "w" mode breaks it since any subsequent
operation (fseek, fwrite and even fclose !) fails with the error 9 (Bad
file number).
So, on AIX , my function fails to restore the FILE * state at its end.

Well, that's also allowed by the standard (it does however sound like
your implementation has a bug)
So, I have a few questions:

- a fread on a FILE * opened in "w" will for sure return 0 item, but is it
really expected that it makes the given FILE * totally unusable even for
fclose !?! This behavior has been observed only on AIX. Linux and Solaris
works.

The standard allows any function of the standard library to set errno
for any reason. (there are exceptions)
- given a FILE *, is there a better way to guess in which mode it has been
opened than attempting to read or write it and look at errors ?

There's no way to do that in standard C.
- given a FILE *, is there a better way to guess if it can be sought than
attempting a fseek on it ?

No way in standard C.
PS: I Xposted my problem on both comp.lang.c and comp.unix.aix because I
have no idea whether this is an AIX specific problem or if the C norm
specifies explicitly that fread have unpredictable effects on a write-only
file descriptor. Anyhow, my goal is to find a portable solution that uses
as less platform specific stuff as possible.

Well, the standard is okay with this behavior. The implementation
probably has a bug. It's topical in both groups I believe.
 
R

Richard Tobin

Lénaïc Huard said:
For some reasons, somewhere in a program, I'd like, if possible, to quickly
parse a whole file before rewinding it and letting the full analysis start.
My problem is that the FILE* I want do parse has been fopen'ed far away
from where I am and I don't know in which MODE my FILE* has been opened.
And additionally, my FILE* may not be a regular file, but a continuous
stream (pipe), in which case it is not rewindable.

I'm at a loss as to why you want to handle the case where the file
has been opened for writing. If you want to read from a file, open
it for reading.

The other aspect - seekability - is entirely reasonable, but I don't
think you can handle it in standard C. If you are (as you appear to
be) using Posix systems, you can stat the underlying file descriptor
and seek only if it is a regular file.

I seem to recall using at least one system where fseek() succeeded
even on pipes provided the seek was within the existing stdio buffer,
so just because one fseek() succeeds, it doesn't mean others will.

-- Richard
 
V

vippstar

On Nov 19, 12:43 am, Mark McIntyre <[email protected]>
wrote:
AFAIK reading from a write-only stream is undefined behaviour.

I've recently asked this (or remember asking it), but I believe I
received no answers.
I've also searched the standard for the answer, with no luck. Anyone?
 
L

Lénaïc Huard

(e-mail address removed) a écrit :
Well that's a bad thing, if you want truly meaningful results you
should enforce just one mode, binary or text.

Sorry, I wasn't clear enough. My problem is not between binary or text. I
deal only with binary files. The problem is between read or write.

I want to write a kind of « decorator » around a FILE *. And the constructor
of that decorator should parse the file is possible (i.e. if it is a
regular file opened for reading). But unfortunately, I don't know whether
those conditions are met when the constructor is invoked.
Load it in memory once, and don't bother with the actual file
rewinding.

But sometimes, this piece of code is used in programs working on continuous
streams. The data must me processed as they come. We can't wait for the
last item before starting to process the first one. And when used in this
context, the total amount of data that will have been read at the end of
the program is too big to fit into memory.
I would like to be able to detect such contexts and to skip the initial
parsing in those cases.
And of course, as I believe in Santa Claus, I try to do this without
modifying the prototypes of a lot of functions to add a read vs. write
information.
That isn't testing anything really, except for that very fseek call.

Indeed, but this was already a workaround. My initial wish was to know
whether I have a chance that the second fseek works or if there is no
chance. In my case, this is nearly equivalent to know whether FILE * is a
regular file or a pipe.
But... I realize that may be such question ought to be put in
comp.unix.programmer instead...
I think you probably want fread(buf, 1, BUF_SIZE, my_file);
You should also observe the return value.

Indeed. Sorry, the real program is better and checks the return. Promise!
Why? That's allowed by the standard.

I have no problem with the fact that fseek returns an error. But the kind of
error make me feel that the fread didn't simply return 0 item, it also
closed the file descriptor. Further in the program, fclose also issues an
EBADF (Bad file number) error.

My concern is not « fseek shouldn't fail ». It's more : « Should fread
executed on an un-readable stream just return an error or is its behavior
completely undefined (including closing the file descriptor, corrupting
memory, crashing or potentially anything else.) »
Well, that's also allowed by the standard (it does however sound like
your implementation has a bug)

The standard allows any function of the standard library to set errno
for any reason. (there are exceptions)

I understand your point: the standard library functions can, according to
the standard, legally fail at any call and return an appropriate error.
But I would expect that any illegal operation (like reading an un-readable
stream) would return an error, set the errno, and leave the stream
un-modified. And instead of this, I have the feeling that my file
descriptor is closed.
 
L

Lénaïc Huard

Richard Tobin a écrit :
The other aspect - seekability - is entirely reasonable, but I don't
think you can handle it in standard C. If you are (as you appear to
be) using Posix systems, you can stat the underlying file descriptor
and seek only if it is a regular file.

Sounds to be exactly what I was looking for... I'm still wondering why I
didn't think about stat before.
Thanks!
I seem to recall using at least one system where fseek() succeeded
even on pipes provided the seek was within the existing stdio buffer,
so just because one fseek() succeeds, it doesn't mean others will.

Sounds to be an excellent reason for not testing the seekability with fseek!
 
J

jameskuyper

Lénaïc Huard said:
(e-mail address removed) a �crit :


Sorry, I wasn't clear enough. My problem is not between binary or text. I
deal only with binary files. The problem is between read or write.

If you don't know whether a given file has been opened for reading or
writing, you need to re-design your code so that you do know.
I want to write a kind of � decorator � around a FILE *. And the constructor
of that decorator should parse the file is possible (i.e. if it is a
regular file opened for reading). But unfortunately, I don't know whether
those conditions are met when the constructor is invoked.

C doesn't have constructors. Are you actually working in C++? in that
case your question should be re-directed to comp.lang.c++.

....
And of course, as I believe in Santa Claus, I try to do this without
modifying the prototypes of a lot of functions to add a read vs. write
information.

Well, I don't see what Santa has to do with that, but if there's some
legitimate reason why the most obvious fix is s unacceptable, you'll
need to find some other way of passing the information around. One
option is to create a structure with at least two members: a FILE *
and something that keeps track of what mode the file was opened in.
 
B

bennett.tony

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all,

For some reasons, somewhere in a program, I'd like, if possible, to quickly
parse a whole file before rewinding it and letting the full analysis start.
My problem is that the FILE* I want do parse has been fopen'ed far away
from where I am and I don't know in which MODE my FILE* has been opened.
And additionally, my FILE* may not be a regular file, but a continuous
stream (pipe), in which case it is not rewindable.

So my program basically behaves like this:

void PreParsing( FILE *my_file )
{
  /*
   * Test if my FILE * is seekable
   */
  if( fseek( my_file, 0, SEEK_SET ) == 0 ) {

    /*
     * If so, go on for the pre-parsing.
     */
    //...
    fread( buf, BUF_SIZE, 1, my_file );
    //...

    /*
     * Rewind the file.
     */
    if( fseek( my_file, 0, SEEK_SET ) != 0 ) {
      // unexpected error since my FILE * was expected to be seekable.
    }
  }

}

* On both Linux 2.6.xx and Solaris 5.10,

when the my_file is a regular file that has been fopen'ed with "w" as mode,
 - the first fseek returns 0 meaning success: my_file is seekable
 - the fread returns 0 meaning that there was nothing to read: expected
since a file opened in "w" is truncated, hence is empty, and anyhow, cannot
be read!
 - the second fseek returns 0 meaning success: my_file could be rewound..

when the my_file is a pipe that has been fopen'ed with "w" as mode,
 - the first fseek returns -1 and errno is set to 29 (Illegal seek).
That's exactly what I expected since a pipe cannot be sought.

Ok. So, on Linux 2.6.xx and Solaris 5.10, my code behaves like I expected
for both regular files and pipes.

* On AIX 5.2,

when the my_file is a pipe that has been fopen'ed with "w" as mode,
 - the first fseek returns -1 and errno is set to 29 (Illegal seek).
I'm still OK with that.

but...
when the my_file is a regular file that has been fopen'ed with "w" as mode,
 - the first fseek returns 0 meaning success: my_file is seekable
 - the fread returns 0 meaning that there was nothing to read: expected
since a file opened in "w" is truncated, hence is empty, and anyhow, cannot
be read!
 - the second fseek returns -1 and sets errno to 9 (Bad file number)
I'm a little bit surprised by this error.

In fact, after having had a deeper look on that, it appears to me that a
fread attempt on a FILE * opened in "w" mode breaks it since any subsequent
operation (fseek, fwrite and even fclose !) fails with the error 9 (Bad
file number).
So, on AIX , my function fails to restore the FILE * state at its end.

So, I have a few questions:

 - a fread on a FILE * opened in "w" will for sure return 0 item, but is it
really expected that it makes the given FILE * totally unusable even for
fclose !?! This behavior has been observed only on AIX. Linux and Solaris
works.
 - given a FILE *, is there a better way to guess in which mode it has been
opened than attempting to read or write it and look at errors ?
 - given a FILE *, is there a better way to guess if it can be sought than
attempting a fseek on it ?

PS: I Xposted my problem on both comp.lang.c and comp.unix.aix because I
have no idea whether this is an AIX specific problem or if the C norm
specifies explicitly that fread have unpredictable effects on a write-only
file descriptor. Anyhow, my goal is to find a portable solution that uses
as less platform specific stuff as possible.

Thanks for your advices.
Lénaïc ...still puzzled by AIX behavior...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkkjPpUACgkQjYEjJATS6Bi55ACgj+RbzOhyjDj63G+ciKo0Iy1B
FuoAn3WsnRc69XXDi2KV0Wt4aOFo0Bpf
=In7u
-----END PGP SIGNATURE-----

Partial solution - you can discover if it is a PIPE or a Special
Device
by doing a "stat" on it and checking the
mode:

Passed in as a parm: FILE *fp;

struct stat my_stat;
mode_t the_type;

fstat ( fileno(fp), &my_stat);

/* apply mask to get just file_type */
the_type = (my_stat.st_mode & _S_IFMT )

if ( the_type == _S_IFIFO ) {
printf("Its a pipe\n");
}else{
if ( the_type == _S_IFBLK ) {
printf("It is a Block Special device\n");
}else{
if ( the_type == _S_IFCHR ) {
printf("It is a Character Special device\n");
....... etc....

-tony
 
M

Martien Verbruggen

Partial solution - you can discover if it is a PIPE or a Special
Device
by doing a "stat" on it and checking the
mode:

[snip code using fstat() and its associated macros]

Of course, this is not standard C, but POSIX. If you are indeed using a
POSIX or POSIX-like system, and you want to use that API, you
should probably be discussing this in comp.unix.programmer, as the
people there tend to know a lot more about POSIX, on average, than the
people here.

Martien
 
C

CBFalconer

Lénaïc Huard said:
Richard Tobin a écrit :


Sounds to be exactly what I was looking for. I'm still
wondering why I didn't think about stat before. Thanks!


Sounds to be an excellent reason for not testing the seekability
with fseek!

And it is entirely off-topic on c.l.c, so all such answers are
necessarily suspect. If you ask on comp.unix.programmer, however,
you will probably be on-target and get valid answers and criticisms
of those answers.

As an aside, if a file is opened in "w" mode, you can't do many
things to it. Read the C standard.
 
L

lawrence.jones

On Nov 19, 12:43 am, Mark McIntyre <[email protected]>
wrote:


I've recently asked this (or remember asking it), but I believe I
received no answers.
I've also searched the standard for the answer, with no luck. Anyone?

It's undefined behavior by virture of the fact that all I/O takes place
as if by calls to fgetc() and fputc(), and their behavior is only
defined for input and output streams, respectively.
 
V

vippstar

It's undefined behavior by virture of the fact that all I/O takes place
as if by calls to fgetc() and fputc(), and their behavior is only
defined for input and output streams, respectively.

Ah, I suspected something like that. Thanks!
 
G

Gary R. Hook

Lénaïc Huard said:
So, I have a few questions:

- a fread on a FILE * opened in "w" will for sure return 0 item, but is it
really expected that it makes the given FILE * totally unusable even for
fclose !?! This behavior has been observed only on AIX. Linux and Solaris
works.
- given a FILE *, is there a better way to guess in which mode it has been
opened than attempting to read or write it and look at errors ?

Once you've eliminated the pipe possibility, dup() the descriptor and
fdopen() the duplicate in r-w mode. Close the old one.

You shouldn't be attempting to read from a stream opened for writing.
That makes no sense and getting undefined, inconsistent results seems
reasonable.
 
K

Keith Thompson

There's no portable way to do it.
Once you've eliminated the pipe possibility, dup() the descriptor and
fdopen() the duplicate in r-w mode. Close the old one.

dup() and fdopen() are defined by POSIX. Using them is ok as long as
you don't mind limiting your code's portability to POSIX
implementations.
You shouldn't be attempting to read from a stream opened for
writing. That makes no sense and getting undefined, inconsistent
results seems reasonable.

Right, reading from a file opened in "w" mode invokes undefined
behavior.
 
R

Richard Tobin

Once you've eliminated the pipe possibility, dup() the descriptor and
fdopen() the duplicate in r-w mode. Close the old one.

What is this supposed to do? If the underlying file descriptor is
opened for writing only, fdopen()ing it read-write won't do you
any good.

-- Richard
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top