P
pete
pete said:
I meant "newline" instead of "null"
I meant "newline" instead of "null"
I meant "newline" instead of "null"
I meant "newline" instead of "null"
R
Randy Howard
es728 said:On Fri, 7 Nov 2003, CBFalconer wrote:
[Talking about Dan Pop.]
I think his diplomatic skills have improved significantly
during the past two years.
Tak-Shing, biased sampling
Now why in the world did you have to say that? He's going to read it
and adjust his style accordingly. *sigh*
A
Anupam
pete said:
Hi everyone, while going through this I was just brushing over that
section of the standard and am quite interested in your comments
regarding the following
citation :
7.19.2 Streams
3 A binary stream is an ordered sequence of characters that can
transparently record internal data. Data read in from a binary stream
shall compare equal to the data that were earlier written out to that
stream, under the same implementation. Such a stream may,however, have
an *implementation-defined number of null characters* appended to the
end of the stream.
Does this mean that a strictly conforming program is unable to
create or use a binary stream because if it assumes a certain no. of
null characters at the end of the stream, this no. may change from
implementation to implementation? Of course fseek can always be used
..... but we cannot rely on the no. of nulls at the end of the stream?
Am I correct? Or am I missing something.
Thanks everyone.. I'll be eagerly awaiting your comments. Actually
This was because even the limitation regarding a newline at the end of
a stream was new to me and I enjoyed this beautiful piece of
information. Also I discovered something else I did not know. Just
sharing it with you all (Though many of you must be already knowing
this fact). It's that as per #2 of 1.19.2 , a space before a newline
is not guaranteed to be read in correctly. That too was news to me.
Thanks and regards,
Anupam
PS. I'd like to suggest all the newcomers to refrain from making
strong comments too fast. I'm also not too old to this group. I find
that I enjoy it immensely, because here people really do know. Use
their expertise and give of whatever you have but dont get into a
slanging contest. My first posting was a bad off-topic one .. albeit I
was thinking that I was doing a right thing by correcting the trolls
... but its no good to feed them.. better by far to ignore them all ...
hang around for a while and then it becomes easy to understand about
the internalisations of the between-the-lines facts of the group.
Thanks
C
Chris Torek
A strictly conforming program can still create or use a binary
stream, provided it makes no assumptions about such null bytes.
This restriction comes from real-world operating systems, including
both VMS and MS-DOS. On these systems, some files come in "fixed
length record" format, typically an exact multiple of 512 bytes
long[%]. If you write 510 bytes to such a binary file, then close
it, then open it and read it back, you will see that the file
contains 512 bytes. The last two, as long as the file is both
written and then read-back in C, must be '\0' (in other languages
they could conceivably have other values). Write 513 bytes, and
this file will be 1024 bytes long, the last 511 of them being '\0'.
-----
[%] The fixed record size is not necessarily 512 -- 1024 and
2048 are not terribly uncommon, and 256 and 128 did occur;
and even non-powers-of-two are possible sometimes. But 512
is by far the most common number.
-----
On some of these file systems, text files are (or were once) also
exact multiples of 512 bytes long. To tell where a text file should
*appear* to end, the I/O library appends a special "end-of-file"
marker character, often a control-Z ('\034') byte. Reading any
file -- including a binary file -- "as if" it were text then returns
EOF when the marker is encountered. Contrariwise, reading a text
file in binary allows you to "read past the end". (VMS's RMS is
too smart to allow you to read a file in the wrong mode by mistake.
MS-DOS is pretty stupid and will let you do whatever you like.
It inherited this from CP/M and QDOS, from which it was cloned.
Current DOS file system formats as used on Windows do specify file
sizes in bytes, but the now-pointless control-Z protocol is often
retained anyway.)
As a quick (if not terribly efficient) illustration of how one can
deal with '\0' padding in binary files, imagine that your C code
to write a block of binary data to a binary file is coded as:
void write_a_block(unsigned char *data, size_t len) {
size_t written;
written = fwrite(data, 1, size, output_file);
if (written != size)
... handle output error ...
}
If you fopen some file for "wb", call write_a_block several times,
then fclose the file, the file may have '\0' bytes appended, so
there is no way to read the file back without possibly getting
those extra '\0' bytes. But suppose we change write_a_block() to
read:
void write_a_block(unsigned char *data, size_t len) {
size_t written;
unsigned long l;
unsigned char b[4];
if (len == 0)
... handle "asked to write 0 bytes" error ...
if (len > 0xffffffffU)
... handle block-too-big error ...
l = len; /* we use l in case size_t is < 32 bits long */
b[0] = l >> 24; /* mask not necessary: we know l <= 0xffffffff */
b[1] = (l >> 16) & 0xff;
b[2] = (l >> 8) & 0xff;
b[3] = l & 0xff;
written = fwrite(b, 1, 4, output_file);
if (written != 4)
... handle output error ...
written = fwrite(data, 1, size, output_file);
if (written != size)
... handle output error ...
}
Now each block in the file is prefixed with a (nonzero) big-endian
count of the number of bytes in the block. To read the file back,
first read four "unsigned char"s, then assemble the desired length:
nread = fread(b, 1, 4, input_file);
if (nread != 4)
... handle input eof/error ...
l = ((unsigned long)b[0] << 24) +
((unsigned long)b[1] << 16) +
((unsigned long)b[2] << 8) +
(unsigned long)b[3];
if (l > (size_t)-1)
... handle block-too-big error ...
... now fread "l" bytes, as before ...
(The "block-too-big" error can occur if the file is written on
system X, where size_t is a 32-bit unsigned value and the block
is, say, 1048576 bytes -- 1 megabyte -- long, but is to be read
back on system Y, where size_t is a 16-bit unsigned value and can
only count to 65535. This test can be omitted if you are certain
that no blocks will be "too big" this way.)
A file that has '\0' bytes appended will cause l to be 0, because
b[0] through b[3] will all be '\0', which is 0. So instead of
fread()ing "l" bytes, we can check for this. Of course, we must
also handle the condition where nread < 4 (or even is 0) if
this is the last block, so the above code has to read more like
this:
size_t i, nread;
unsigned long l;
nread = fread(b, 1, 4, input_file);
for (i = 0; i < nread && b == '\0'; i++)
continue;
if (i == nread)
... handle input EOF ...
if (nread < 4)
... handle input error ...
/* build and handle l as before */
Any number of other binary file formats can be designed. To make
it possible to use such files on any hosted C system, however, the
design must account for the possibility of extra '\0' bytes added
to these files. If anyone ever decides to port your code to an
old mainframe, they will be glad you considered this.
I
Irrwahn Grausewitz
As binary streams have implementation-defined properties, any program
that produces output dependent on binary streams is not strictly
conforming.
ISO/IEC 9899:TC1 7.19.9.2#3:
[...] A binary stream need not meaningfully support fseek calls with a
whence value of SEEK_END.
ITYM 7.19.2#2; there is no section 1.19.2 in either C89 or C99.
<snip>
Regards
I
Irrwahn Grausewitz
Irrwahn Grausewitz said:
Should read: ... on these properties of binary streams ...
A
Anupam
Yup that was a typo, sorry about that.Irrwahn Grausewitz said:
D
Dan Pop
In said:
Nope. Divide the file between a fixed length header part and a variable
length data part. If the header contains the length of the data part,
you know exactly what is the last byte of the file that was actually
written by your program and you can safely ignore any bytes that might
be following it.
OTOH, it doesn't make much sense to talk about strictly conforming
programs in the context of I/O: any I/O request may fail for unspecified
reasons and, if your program's output is affected by such failures, your
program is no longer strictly conforming. OTOH, what's the point of
reading data from files if the program's output is not going to be
affected by it? ;-)
fseek is useless if you want to go at the end of a binary stream.
However, if you use the above described approach (or if you know how
many bytes have been written to the binary file by the program creating
it, in advance), you can easily fseek to the last byte of data written
to the file by the program that created it.
Dan
J
Joe Wright
pete said:
S
Simon Biber
Joe Wright said:
Not just that -- it's also to ensure that the stream is
terminated correctly and therefore is valid as a whole.
Remember that streams like stdout need not be interactive,
and in the worst case the user may lose the whole contents
of the stream if the last line is not terminated correctly!
Input streams are beyond the control of the C program so
the validity issue doesn't really come up -- I would just
assume that the implementation will give you a stream that
it considers valid.