Programming in standard c

[Gordon Burditt]

I agree, all systems I know where such a function could be useful have fstat

so its already possible to write portable code for your application domain.
No real need to standardize but it would be nice. I just found the
resistance to adding such a function in this thread strange

The size value returned by fstat() is useless for allocating a
buffer of appropriate size to read in the file. (Yes, file size
can fail on unitasking systems also.) Trying to use it that way
encourages virus-friendly coding. No amount of arguing over the
appropriate definition of the size of a file will change this.

Jacob seems to want it more judging by his use of capitals in his posts

Does Jacob have a financial interest in botnets?

 

[Richard]

[snips]

I am not surprised you say this either. Till now, there hasn't been
ANY system where there wasn't an operation to get the file size.
VAX/VMS included. And if there was one, it could ALWAYS do
1) Open the file
2) Read until EOF

to get the file size. Granted, it wouldn't be efficient in some weird
systems but so what? It would be possible.

But it would not necessarily be *meaningful* which you persist in
overlooking.

If you do this and calculate you need to allocate 17,403 bytes to read the
file, do you know that in the time it takes you to allocate the space,
rewind and commence reading, someone else has not changed the size of the
file?

How ridiculous. The data he READS at the time was that size. That is ALL
he can account for. Clearly writing back needs other checks or some sort
of locking out of others (transaction management) IF the data
manipulations he performs on his buffer do indeed need to be written
back.

In a single-user, single-tasking system, not a problem, presumably. In a
multi-user system where the file is shared between many users, it's
entirely possible your calculations aren't valid by the time you use them.

So, explain to us how this file size you've calculated means anything
relevant to the C code issues discussed here - notably, trying to read the
entire file into a buffer.

Well, lets see.

malloc enough memory for the size of the file at the time of
interest. it really doesn't get much simpler.

 

[Gordon Burditt]

So, explain to us how this file size you've calculated means anything

Yes, this is 100% clear logic. I will apply it then.

1) You can't open a file with
fopen("name","a+")
since somebody else could grow the file after the file is positioned at
EOF, so you would overwrite his data.

In some implementations, the file is opened with the POSIX O_APPEND
flag when you fopen with mode "a" or "a+", so *EVEN IF* someone
else grows the file, you will still add on to the end of the file.

2) ftell/fseek/ and in general all file primitives should be dropped
from the standard. If you write something somebody else could have
written at the same position. You should not write in a file.

If you are working with databases, isn't the whole point that you should
see writes made by other programs?

I have answered this thousand times but you still come back with the
same nonsense!

Use of filesize() to allocate a buffer, and then assuming the file size
has not changed encourages virus-friendly programming. You seem to be
advocating buffer overflows. The other problems you mention seem to be
limited to loss of data.

 

[Gordon Burditt]

So, explain to us how this file size you've calculated means anything

Well, lets see.

malloc enough memory for the size of the file at the time of
interest. it really doesn't get much simpler.

No, you malloc() enough memory for the size of the file *AFTER* the
time of interest. Every operation in C takes at least two moments,
and on a multi-tasking system, someone else can make changes the
moment before.

Do you get paid by the buffer overflow?

 

[Serve Lau]

What is a "normal" file?

There are file systems supporting sparse files, which in most cases appear
to be "normal" files, except that while they're reported to be of one
size, they're actually another size entirely - they claim to be 2GB, but
only occupy 200K on disk, for example.

Other systems allow for compressed or partially compressed files, where
the actual size of the data bears little, if any, relation to the size of
the file on disk.

And what do you expect that fread will put into the buffer on such a
filesystem? The compressed or decompressed data?

 

[Army1987]

Portable code which checks a file size can be written then but a programmer
cant just expect to do something strange like check the size of "LPT1" and
expect the standard C function to work correctly. In fact writing
fopen("LPT1:", "r"); is already not portable anymore in the sense that your
code wont work on multiple systems. Same goes for most of the other system
dependant files mentioned.

Yeah, reading from a printer is veeeeery non-portable...

 

[Serve Lau]

Yeah, reading from a printer is veeeeery non-portable...

Not sure what you mean with this, does fopen("LPT1", "r") return a valid
FILE * on Linux for instance?

 

[jacob navia]

Use of filesize() to allocate a buffer, and then assuming the file size
has not changed encourages virus-friendly programming. You seem to be
advocating buffer overflows. The other problems you mention seem to be
limited to loss of data.

I allocate the buffer given by filesize, then I read exactly that with
fread.

If the file has grown I will ignore what goes beyond, and if the file
shrinks, I will have a short read and a too large buffer.

There is NO WAY I COULD HAVE A BUFFER OVERFLOW!

For the nth time:

You are telling just NONSENSE!

 

[Serve Lau]

No, you malloc() enough memory for the size of the file *AFTER* the
time of interest. Every operation in C takes at least two moments,
and on a multi-tasking system, someone else can make changes the
moment before.

So? Like Jacob has indeed said 1000 times already, by that same logic you
can't use fread or fgets anymore either.
Somebody else could've made the file smaller when you started fgets on the
last line.

 

[Serve Lau]

"user923005" <[email protected]> schreef in bericht

I think we can improve this a great deal, with the result being
a function that is written entirely in Standard C and works in
every case in which it is possible for it to work, and -- by
calling a system-dependent function that the user is to supply,
but which may be replaced with a #define that simply returns 0
if desired -- is "reasonably efficient" as well.

[snip]

I work for a database company, and most of our customers are large
customers. (E.g. huge US company, large university, government of
country x, etc.)

It is not at all unusual for a single file to be 20-100 GB. Needless
to say, you would not want to put this file into memory even if you
could do it.
The files I work with are also being modified constantly (though there
are occasionally windows of inactivity for some of them).
I am quite sure that the goal of reliably reading these sorts of files
into memory and doing something useful with them is literally
infeasible (not impossible, but the cost would make it so stupid that
nobody would want to do it).



What is your point? This DBMS doesnt come with an API to update the
database? You have to use fread to get records? Surely not. The reasons for
not including a function like filesize in the standard are getting funnier
by the minute.

There are BTW more uses for a filesize function than just get the size to
malloc memory with. How about for your DBMS and you want to create an
administration tool where you can constantly monitor how big the files are.
Just an extra thingie you would be able to do in standard C.

This is really getting silly, a function like filesize could be put into the
standard and it would be useful. But as pointed out elsewhere in this thread
this discussion wont help to get the function into it.

 

[Walter Roberson]

*waiting for the regulars to point out the obvious flaw in this code*

<stdio.h> not included?

 

[Harald van Dijk]

Not sure what you mean with this, does fopen("LPT1", "r") return a valid
FILE * on Linux for instance?

If a file named LPT1 exists, sure. It's just another name.

But on those systems where LPT1 is a printer device, you should probably
be writing to it, not reading from it.

 

[Serve Lau]

<stdio.h> not included?

We have a winner! You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the toaster
XML configurable over the internet and the developers could not get the size
of the xml files and no network connection with standard C so they opted for
embedded java. You can burn patterns in your bread with it!

 

[jacob navia]

No, you malloc() enough memory for the size of the file *AFTER* the
time of interest. Every operation in C takes at least two moments,
and on a multi-tasking system, someone else can make changes the
moment before.

Do you get paid by the buffer overflow?

There is no buffer overflow as I told you in another message.

I repeat it now since you went silent instead of acknowledging it.

1) I call filesize
2) I allocate a buffer with the result+1.
3) I read with fread into that buffer.

If the file is bigger, the fread argument will ensure I read ONLY
what I allocated. I ignore the rest. If the file is smaller I get
a short read and the buffer is too big.

THERE IS NO POSSIBILITY OF A BUFFER OVERFLOW.

I suppose you will stay silent now, instead of acknowledging your
error.

 

[Serve Lau]

If a file named LPT1 exists, sure. It's just another name.

But on those systems where LPT1 is a printer device, you should probably
be writing to it, not reading from it.

usenet is funny lol, I'm sure you are smarter irl as you make it out be
right now

But ok, let me rephrase!

If you want to write a program that opens the printer for reading (yes
reading because you just want to get the hypothetical filesize) and you use
fopen("LPT1") will you expect the program to actually open the printer on
Linux?

Is it clear now what I meant? rofl

 

[Keith Thompson]

[The above was written by Serve Lau; Gordon, as always, snipped the
attribution line.]

The size value returned by fstat() is useless for allocating a
buffer of appropriate size to read in the file. (Yes, file size
can fail on unitasking systems also.) Trying to use it that way
encourages virus-friendly coding. No amount of arguing over the
appropriate definition of the size of a file will change this.

No, it's not useless.

Determine the *current* size of the file by some system-specific
method; let's say it's exactly 1 megabyte. Allocate buffer of exactly
1 megabyte, and attempt to read exactly 1 megabyte of data from the
file into the buffer. If the file has shrunk, you won't get the full
megabyte. If the file has grown, you won't get the entire current
contents of the file; you can then either realloc() the buffer, or
just use the data you've got. In neither case do you get a buffer
overrun.

Obviously if you blindly read the entire file into your 1-megabyte
buffer, you risk an overrun. So don't do that.

The above was written by Keith Thompson <[email protected]>. Gordon, if
you post a followup, please quote this paragraph. I do not give
permission to quote my words without attribution.

 

[Harald van Dijk]

usenet is funny lol, I'm sure you are smarter irl as you make it out be
right now

If you want to write a program that opens the printer for reading (yes
reading because you just want to get the hypothetical filesize)

That simply doesn't make sense, which is what I believe Army1987's point
was, and which I restated.

and you
use fopen("LPT1") will you expect the program to actually open the
printer on Linux?

I answered that already. LPT1 is just another file name. You might get
the printer if someone has set up a compatibility symlink or special
device for your application. You might get an ordinary file if you
previously wrote to it. Or it might simply not exist.

 

[jacob navia]

[The above was written by Serve Lau; Gordon, as always, snipped the
attribution line.]

The size value returned by fstat() is useless for allocating a
buffer of appropriate size to read in the file. (Yes, file size
can fail on unitasking systems also.) Trying to use it that way
encourages virus-friendly coding. No amount of arguing over the
appropriate definition of the size of a file will change this.

No, it's not useless.

Determine the *current* size of the file by some system-specific
method; let's say it's exactly 1 megabyte. Allocate buffer of exactly
1 megabyte, and attempt to read exactly 1 megabyte of data from the
file into the buffer. If the file has shrunk, you won't get the full
megabyte. If the file has grown, you won't get the entire current
contents of the file; you can then either realloc() the buffer, or
just use the data you've got. In neither case do you get a buffer
overrun.

Obviously if you blindly read the entire file into your 1-megabyte
buffer, you risk an overrun. So don't do that.

The above was written by Keith Thompson <[email protected]>. Gordon, if
you post a followup, please quote this paragraph. I do not give
permission to quote my words without attribution.

Why do you ignore what I am saying?

I posted an identical message in this thread proving I could not
possibly have a buffer overflow.

You say the same thing without quoting me.

 

[Serve Lau]

I answered that already. LPT1 is just another file name. You might get
the printer if someone has set up a compatibility symlink or special
device for your application. You might get an ordinary file if you
previously wrote to it. Or it might simply not exist.

yeah play more games, move along please nothing to see here

 

[Ben Pfaff]

Why do you ignore what I am saying?

The usual reason in Usenet would be that you are in his killfile.
(I don't know whether that is actually the case.)