[snips]
No. Not ignore them. Discard them as the kind of pedantic oneupsmanship
that has place in the real world where real, practical solutions are
required.
So what's the real, practical size of /var/log/apache/access.log?
Calculate the file size _now_, you get, say, 100K. Wait half a second,
it's 102K. Wait another second while logrotate reaps it, it's now 0K. So
which size is it - the 100K you read, the 102K it was almost immediately
after, or the 0K it is now?
Presumably the Java functions will do the same thing the equivalent C
functions would do - report size at time of inspection. They, however,
still face the same problem, that the size reported has absolutely bugger
all to do with the size _now_.
You call this - real-world problems faced by real-world programs -
"pedantic oneupsmanship"; we look at it as simply one more problem which
has not been dealt with.
One way to deal with it is to do what Java apparently did: accept that you
can only get a value for file size at time of querying, which is a
reasonable position to take, but doing so also ignores the fact that the
size may change between querying the size and making use of the size.
Some applications might need to track the changes; others won't. That
some don't does not mean no application needs to, nor does it mean that
the issue is simply one of pedanticism; it is, rather, a question of
determining what the actual problem to be solved is.
Jacob's exemplar of reading a file into memory is a good example. He uses
whatever function to calculate the size, so far so good. He allocates a
buffer, still all good. So his size recorded is, say, 100K and he
allocated 100K, all's well.
When he actually reads the file, though, and discovers zero bytes to read
- because the file has been rotated and emptied - is this an error, or is
this as expected?
The reality is that it is as expected... but it is also reality that
unless his code is particularly smart - *and* knows the details of the
system upon which it is being executed - it is very likely to see this as
an error condition and fail in some manner.
Is this an error? Not really. Is his code going to be smart enough to
recognise this as being something other than an error? Possibly, if he
knows about this exact file, but probably not in the general case,
particularly if his code is used on multiple disparate systems. You want
to write off such issues as simple pedanticism, yet they're not - they're
real-world issues which come up in real-world systems, which need to be
dealt with by real-world code. Simply hand-waving them away doesn't work.
So rather than hand-waving them away, how about explaining how, exactly,
you're going to deal with them? How do you plan to determine the "size"
of a file, when that size keeps changing? How do you plan to determine
the "size" of a compressed, or partially compressed, or sparse file? Or
worse, sparse, partially compressed files? What actual "size" matters,
and how do you determine that this is the size that matters?
Determining the size - if you plan to - can be critical. For example,
take a sparse file. One size value says the file is 10GB, another says
it's 200K. One reports "theoretical" size, the other reports actual
usage. Here's the kicker: if you want to copy this file to another file
system, you _may_ be able to do it onto a file system with 200K free, if
it supports sparse files, or you may require 10GB, if it doesn't. So
which size is the proper one - the 200K actually allocated on disk, or the
10GB which is the "data size" required to store it on another file system
without risking losing data?
This doesn't even address the issue of translation - is that 100K size
that you read still going to be 100K if you want to read the file as a
text file? Or is it only applicable if you read it in binary mode?
Probably the latter, but if it is, in fact, a text file and you wish to
process it as such, how much space do you need to allocate to handle it?
You can't tell this from the size on disk, now can you?
The whole concept of "file size" is one where the pat answer sounds good
and even is good in many cases, but it is also one where the pat answer
falls on its face in far too many cases to naively rely upon it, something
the more experienced folks keep trying to point out, yet some folks, for
some reason, seem to want to ignore this.
Perhaps the simplest way to approach the issue is to ponder those
real-world cases where the pat answer doesn't work well and consider what,
if any, solution you would propose, a solution which actually covers those
cases.
I suggested one such, but it involves reporting not a single answer, but
several: size on disk, size reserved (eg how big a sparse file "actually
is"), size of contained compressed data, size of contained data after
decompression. That's four; perhaps we need to add another four,
basically the same values but after translation to text mode - though I
suspect this is going to make file management awfully inefficient. Yet
that's 8 different size values to describe *one* file, and still doesn't
deal with the fact the file size - any of those sizes - can change at a
moment's notice.
So which of the eight is "the size of the file"? Size on disk has little
bearing on what you'd need to allocate to store the contents in memory.
Size after decompression would be closer, but doesn't tell you how much
overhead is involved due to translation. Neither of these tells you how
much space will be required if you want to copy the file to a file system
which doesn't support sparse files.
Oh, and then there are "forks". Can't forget those; several file systems
include them. Those make things even more interesting. With forks, the
reported size on disk may well represent the total data in all forks, but
if you allocate a buffer that size and try to read the file, you're liable
to get somewhat less than you'd expect, since you're liable to only be
reading one fork, not all of them. So is that an error? Or is that an
expected situation? Does your code know the difference between reported
size on disk and expected size when reading?
There are so many issues to consider, yet some folks want to treat it as
if the only consideration in the world is getting a singular value
representing "the size of the file", without ever stopping to consider -
or worse, writing off as "pedanticism" - the fact that "file size" is a
meaningless concept in all but a few cases.
So explain to us, you - or the others who think this issue is so trivial -
which of the 8 values I mentioned, the ones which don't even deal with
forked files, is "the size of a file".
Chances are, the best answer you'll come up with is "size on disk", but
this value is useless for virtually all purposes, other than determining -
in *some* cases - whether you have enough room on another disk to store a
copy of the file, and can fail miserably even there.
C doesn't have a standardized filelength that does anything useful? Okay,
great, it doesn't. What would *you* put in its place, though? Which of
the umpteen possible size values would you report for a given file? How
would you have it determine - and report - whether that size meant size on
disk, size required to store another copy, size before or after
decompression, etc, etc, etc?
Java designers apparently decided to pick one particular value and report
that. Fine, great, wonderful and all, but it doesn't deal with all the
other case. It is one possible answer, and presumably a perfectly
acceptable one - for the cases to which it applies. It is going to be
considerably less useful for other cases.
