Programming in standard c

[Serve Lau]

In which case you get an end-of-file indication, or an error, not a buffer
overflow which can let someone execute arbitrary code.

Explain to me how this code will cause a buffer overflow when during the
fread the file gets smaller or bigger.

file = fopen("multi_accessed_file", "rb");
size = filesize(file);
buffer = malloc(size+1);
numread = fread(buffer, 1, size, file);

And dont start about not checking for errors, in real code the appropriate
functions would not be called when a function would fail and the user would
get reported. Now please tell us where's the buffer overflow in those 2
situations?

 

[Serve Lau]

Serve Lau said:


Fine. All this means is that the Java standard library designers have
taken
some arbitrary decisions and enshrined them in a standard library. Because
they are not fools, we can presume that these arbitrary decisions were
taken as intelligently as possible, but nevertheless the decisions they
have made will necessarily ignore some of the issues discussed in this
thread.

That is probably true. Making arbitrary decisions as intelligently as
possible is sometimes called "software engineering". Of course this class
cant handle all files in all situations ever possible. They apparently chose
to include a class that handles many common cases and its up to the
programmer to understand that when he now has "sparse files" that
File.length() might not fit his needs. I found a thread where a programmer
wanted to use File.length() on Windows to get the real bytes occupied in
clusters and sectors on disk and was surprised it didnt work. The answer was
to write a Windows dependant JNI function if you wanted to do something like
that. Like jacob has said, you dont have to use it, it is optional. When the
function doesnt fit your needs there's probably a function that does and use
that.

 

[jacob navia]

In which case you get an end-of-file indication, or an error, not a buffer
overflow which can let someone execute arbitrary code.

you are deliberately ignoring what we say. There is NO
buffer overflow you are just lying.
1) allocate a buffer returned by filesize()
2) read with fread that quantity of data into the buffer

nowhere there is a buffer overflow. If the file is bigger the rest will
be ignored.

 

[Serve Lau]

They might be funny to someone that doesn't realize that C runs on a
lot of platforms other than those they he/she is familiar with. I'm
not laughing, however.

C runs on more platforms than I'm familiar with? Is there more than windows
then? The point I'm trying to make that a function like filesize could be
put into the standard easily regardless if there are situations where this
function wont work well. The standard already has many functions that wont
work on many systems.

So you could make sure that the normal I/O to the files happens slower
than it would otherwise. Yes, I could see how some DMBS admins might
think that's actually a good thing. I am laughing now, btw.

Could be interesting to see a graph over time when the files grow or shrink
most. But yeah funny haha, who needs management and marketing information
anyways. What DBMS is it that you work on?

Or, you could write one to do what you desire, in far less time than
you have spent so far on this thread, which does whatever it is you
think is appropriate for your platform.

Dont worry about my time I'm in "the usenet zone" at the moment. I would've
used fstat long ago if a situation called for getting a file's size and
where I expected this function to work. I would've kept it secret to clc of
course because now I have written unportable code.

 

[Richard Bos]

Remember, CISC vs RISC !


Thanks for pointing out that one can pick up a ludicrously over spec'ed
machine which struggles to run a bloated OS, is a virus and spam magnet
and goes tits-up regularly for only marginally less than you can pick up
rock-solid extremely reliable bit of kit thats still rocking nearly two
decades after it was first shipped....

Marginally less? Either you think VMS is less stable than M$ Windows, or
you are not /au fait/ with the current behaviour of the dollar.
Actually, whichever you think is more stable, I wouldn't call 475
_marginally_ less than 600.

Richard

 

[Chris Torek]

I am mostly trying to stay out of this thread at this point, but
I will add a note or two here.

[The Java library designers] apparently chose to include a class
that handles many common cases and its up to the programmer to
understand that when he now has "sparse files" that File.length()
might not fit his needs. I found a thread where a programmer wanted
to use File.length() on Windows to get the real bytes occupied in
clusters and sectors on disk and was surprised it didnt work. The
answer was to write a Windows dependant JNI function if you wanted
to do something like that. Like jacob has said, you dont have to
use it, it is optional. When the function doesnt fit your needs
there's probably a function that does and use that.

It is worth considering these things here, too:

- The Java "base system" is larger (*much* larger) than that for C.
I suspect it handily exceeds even that for C++.

- Java is not really expected to work on the tiniest embedded
systems (although it *is* expected to work on what I would
consider "surprisingly small" systems). For instance, one
would probably not use Java to run the CPU inside a disk drive.
C is in fact used for such things (although C already splits
these off into "freestanding" systems, so increasing the size
of the "hosted" library, as C99 did, is not as fatal as it
might seem at first blush).

 

[jacob navia]

That is probably true. Making arbitrary decisions as intelligently as
possible is sometimes called "software engineering".

Gosh do not use that kind of words with the regulars!

They can't understand anything complicated like that, anyway that
doesn't appear in the C89 standard...

Of course this
class cant handle all files in all situations ever possible. They
apparently chose to include a class that handles many common cases and
its up to the programmer to understand that when he now has "sparse
files" that File.length() might not fit his needs.

The people designing Java were actually interested in providing a
good library, not just making everybody go to C++.

 

[Randy Howard]

C runs on more platforms than I'm familiar with? Is there more than windows
then? The point I'm trying to make that a function like filesize could be
put into the standard easily regardless if there are situations where this
function wont work well. The standard already has many functions that wont
work on many systems.

Leaving freestanding implementations aside for the moment which get to
"cheat" after a fashion as to what they have to support, share some of
them with us. Name 3 of these functions, and give examples of where
they don't work on a hosted implementation. If you mean
"implementation defined behavior by chance, that is != "wont [sic]
work".

Could be interesting to see a graph over time when the files grow or shrink
most. But yeah funny haha, who needs management and marketing information
anyways.

If a database developer felt that such information was quite important,
I suspect that he would implement an efficient low-level solution
particular to the supported platform(s), abstracted as need be for
multi-platform support in its internal implementation and provide that,
rather than rely on some generic "standard C library" implementation to
hopefully give him what he needs, even if it existed. Performance
tuning being as critical as it is in db work, where things like
bypassing the typical file systems completely on the platform and
writing directly to the raw storage media directly instead is not
uncommon, I'd find it surprising for anyone other than a naive
implementer to not at least explore a more specific option, unless he
had performance measurements taken on all supported platforms that
demonstrated there was no need for it. Not to mention a standard
filesize() function to fall back on, of course

What DBMS is it that you work on?

None currently. Feel free to draw on your vast current experience to
point out my mistakes, I'd enjoy learning more about the subject.

Dont worry about my time I'm in "the usenet zone" at the moment. I would've
used fstat long ago if a situation called for getting a file's size and
where I expected this function to work.

If you would rely on the ability to call fstat(), then we can make some
assumptions about those platform that you don't care about. There is
no problem with that, as long as everybody that cares about the code
finds it a credible design decision.

I would've kept it secret to clc of course because now I have written
unportable code.

You can write portable code, while using non-standard functions, it
just takes more care, in some cases, a lot more, and you have to narrow
your definition of "portable" to "portable to those platforms that we
currently or expect to care about in the lifetime of the product".
That is a legitimate, real world design decision that is made all quite
often. Even more common is that the issues are not even considered,
and it is a problem for the "next guy" long after the current one has
left the building.

Deciding to purse this option of separating out the non-standard
support such that it can be ported to platform N+1 with a minimal
impact on the existing common core code is not wholly trivial, but it's
not extremely difficult either. Even so, it happens far less often
than you might hope.

This is not the same as writing pure standard C code alone, but that's
not really an issue if it is done cleanly, and the areas where porting
to an as yet unimplemented new platform would require attention are
documented appropriately.

 

[Kenny McCormack]

On USENET, it takes at least a week to "go silent". It hasn't
been anywhere near that long. Some people have lives outside
of USENET. And article transmission is not instantaneous.
Messages "cross in the mail" all the time.

That's true in the theoretical, but not in the practical, sense.

Here, if Jacob posts something and heathfield doesn't post an attack
within the hour, it is probably a good idea to send an ambulance over to
heathfield's place.

 

[Eric Sosman]

[...]
If you want to write a program that opens the printer for reading (yes
reading because you just want to get the hypothetical filesize) and you
use fopen("LPT1") will you expect the program to actually open the
printer on Linux?

Is it clear now what I meant? rofl

Until you get some printer that sends back status info and you really can open
it for reading.

<off-topic>

Back in the 1960's I used an IBM computer that one booted
by dialing up the address of the boot device and pressing the
IPL ("Initial Program Load") button. The machine read from the
chosen device and stored what it received in low memory, then
branched to location zero and started executing it. And yes:
If you dialed the printer's address, the machine would boot
from it.

The printer actually responded to the read with some kind
of data (if you zeroed memory first, you could see that it had
been changed afterward), but unfortunately the loaded "program"
didn't do anything useful.

</off-topic>

 

[Syren Baran]

Wait for a certain time is impossible in standard C. (As I said,
you can't do much in standard C). But that could be a portable
action if the interface level of C wouldn't be so low.

One logical conclusion to adding such function to C would be "You shalt
not use C for an 8bit microcontroller without a timer!" Amen.

I´m not sure if youre to focused on PC-style hardware to understand how
daft your standardisation propositions are. The same libraries on
everythig from a small microcontroller with a couple of K of memory to a
large supercomputer?
Even Java, which is highly portable, has different libraries for PC´s
and small devices. And even other ones for smartcards.

Some things just dont belong in the language standard. If its part of
the language standard it has to be implementable on every device that
can be programmed in that language, simple as that.

 

[user923005]

On USENET, it takes at least a week to "go silent".  It hasn't
been anywhere near that long.  Some people have lives outside
of USENET.  And article transmission is not instantaneous.
Messages "cross in the mail" all the time.  


You still haven't said anything up to this point about error checking,
or what size you use for fread().


You still haven't stated what you're using as the argument to fread().



Lack of any error checking is an improvement over the possibility of
a buffer overflow, but not by much.  

As we recall, the original goal of this thread was to read a file into
memory.
Jacob's algorithm fails to do that, if someone has extended the file.

 

[Walter Roberson]

Some things just dont belong in the language standard. If its part of
the language standard it has to be implementable on every device that
can be programmed in that language, simple as that.

I don't see how that follows. For example, Appendix F of C99,
describing behaviours of floating point arithmetic, is part of
the C standard and yet specifically says that the rules there
only apply if a certain macro is defined. It is completely conforming
to have floating point that behaves differently than described in
Appendix F, as long as that macro is not defined. Thus, there are
parts of the C language standard which need not be implementable
on every device that can be programmed in C.

 

[jacob navia]

I don't see how that follows. For example, Appendix F of C99,
describing behaviours of floating point arithmetic, is part of
the C standard and yet specifically says that the rules there
only apply if a certain macro is defined. It is completely conforming
to have floating point that behaves differently than described in
Appendix F, as long as that macro is not defined. Thus, there are
parts of the C language standard which need not be implementable
on every device that can be programmed in C.

And in machines with no file system files do not have any sense.

In a 8 bit micro controller maybe there is no sense in implementing long
longs...

Etc.

filesize() could very well be restricted to a certain subset of all
the machines that implement C.

 

[Keith Thompson]

And in machines with no file system files do not have any sense.

And of course the standard allows for this by permitting freestanding
implementations, which don't have to provide any of the standard
library (apart from a few headers that don't declare any functions).

(jacob, the English idiom is "do not make any sense".)

In a 8 bit micro controller maybe there is no sense in implementing long
longs...

But the standard doesn't permit a conforming implementation, either
hosted or freestanding, to omit support for long long, which must be
at least 64 bits. This leaves an implementer for such a target with
two choices: either emulate 64-bit arithmetic in software (which
probably isn't much harder than emulating 32-bit arithmetic, and
needn't affect programs that don't use it), or produce a
non-conforming implementation.

Etc.

filesize() could very well be restricted to a certain subset of all
the machines that implement C.

*If* filesize() were added to the standard, it should probably be
required for all hosted implementations; it can always return an error
indication if the size of a particular file can't be determined. This
is what fseek() does; many files are not seekable.

There is at least one sticking point. What if the only way to
determine the exact size of a file is to read the entire file and
count the bytes? Should filesize() for, say, a 2-gigabyte file
actually be required to do that, or may it immediately return an error
indication? Probably it should be implementation-defined -- which
means there will be cases where the implementation *could* determine
the exact size of a file, but refuses to do so. The alternative is
for a seemingly innocuous query to tie up the program for a long time.

I wouldn't object to seeing filesize() added to the standard, as long
as all these issues are resolved and the description includes
sufficient warnings. But I don't actually expect it to happen.

Note that, in many implementations, the trick of fseek()ing to the end
of a file and calling ftell() to get the offset actually works, at
least for files whose size doesn't exceed LONG_MAX. It's not
portable, of course, but if you happen to know that your program will
run only on systems where that works, it's probably no worse than
invoking some system-specific function like fstat().

 

[Walter Roberson]

There is at least one sticking point. What if the only way to
determine the exact size of a file is to read the entire file and
count the bytes? Should filesize() for, say, a 2-gigabyte file
actually be required to do that, or may it immediately return an error
indication? Probably it should be implementation-defined -- which
means there will be cases where the implementation *could* determine
the exact size of a file, but refuses to do so. The alternative is
for a seemingly innocuous query to tie up the program for a long time.

The amount of work to put into getting the size could be a parameter.

 

[Mark McIntyre]

The amount of work to put into getting the size could be a parameter.

interesting idea. Some new macros in limits.h, to assist:

#define LOOK_LIKE_YOURE_TRYING 1
#define UNDERGRADUATE_EFFORT_LEVEL 2
.....
#define UTTERLY_ANAL_CLC_MODE -1LL

 

[CBFalconer]

Keith Thompson wrote:
.... snip ...

Why do you ignore what I am saying?

I posted an identical message in this thread proving I could not
possibly have a buffer overflow.

You say the same thing without quoting me.

This is Usenet. There is no guaranteed delivery. That is why
articles should stand by themselves, and is the basic reason for
quotation mechanism. I would have thought you already knew this.

 

[CBFalconer]

"Army1987" <[email protected]> schreef in bericht
.... snip ...


Not sure what you mean with this, does fopen("LPT1", "r") return
a valid FILE * on Linux for instance?

It would normally depend on what the file LPT1 is made, and the
details of the system interface to it. Note this does not preclude
(nor enable) use of LPT1 as a printer device.

 

[CBFalconer]

You do, perhaps, but others in the thread seem to think, and gave
procedures or code demonstrating that, that indicates otherwise.


You seem to have the idea that USENET is instantaneous. Chances
are all the posts you are complaining about were posted before
you posted your first rebuttal.


No, it's nonsense, and just because YOU wouldn't do it doesn't
mean others won't. Some of them have described doing just that,
with pseudo-code or code, in this discussion.

I have no idea who you are quoting. This leaves off any possible
explanations for idiotic comments. Kindly include proper
attribution lines in your posts.