Programming in standard c

R

Richard Heathfield

Serve Lau said:

You win a toaster with the software written in embedded
java! They tried C at first but The Management decided to make the
toaster XML configurable over the internet and the developers could not
get the size of the xml files and no network connection with standard C
so they opted for embedded java.

Java faces the same issues as C with regard to establishing the size of a
file. These are software engineering issues, not language issues.
 
J

jacob navia

Ben said:
The usual reason in Usenet would be that you are in his killfile.
(I don't know whether that is actually the case.)

Of course I am not in his killfile when it comes to
attacking me.

My point is proved. The silence of those people speaks for itself.
 
F

Flash Gordon

Ben Pfaff wrote, On 30/12/07 22:53:
The usual reason in Usenet would be that you are in his killfile.
(I don't know whether that is actually the case.)

Or Jacob's message had not reached Keith, or it had but Keith has not
read it yet, or he just happened to respond to a message on a different
part of the thread.

Jacob, you should not assume people are always out to get you.
 
S

Serve Lau

Richard Heathfield said:
Serve Lau said:



Java faces the same issues as C with regard to establishing the size of a
file. These are software engineering issues, not language issues.

Java already faced them you mean because such an operation is included in
*tadam* the standard library :)
 
R

Randy Howard

There is no buffer overflow as I told you in another message.

I repeat it now since you went silent instead of acknowledging it.

1) I call filesize
2) I allocate a buffer with the result+1.
3) I read with fread into that buffer.

If the file is bigger, the fread argument will ensure I read ONLY
what I allocated. I ignore the rest. If the file is smaller I get
a short read and the buffer is too big.

THERE IS NO POSSIBILITY OF A BUFFER OVERFLOW.

Except you left out some steps. Like checking for and handling errors.
Note my caps lock key isn't suffering from intermittent sticking
problems as yours seems to be.
 
R

Randy Howard

Java already faced them you mean because such an operation is included in
*tadam* the standard library :)

Enlighten as to how this Java function handles real time changes to
file size between the time you make the call and the time you use the
result.
 
R

Randy Howard

I work for a database company, and most of our customers are large
customers. (E.g. huge US company, large university, government of
country x, etc.)

It is not at all unusual for a single file to be 20-100 GB. Needless
to say, you would not want to put this file into memory even if you
could do it.

Especially since odds are the underlying file system buffer cache will
do it for you anyway.

[snip]
What is your point? This DBMS doesnt come with an API to update the
database? You have to use fread to get records? Surely not. The reasons for
not including a function like filesize in the standard are getting funnier
by the minute.

They might be funny to someone that doesn't realize that C runs on a
lot of platforms other than those they he/she is familiar with. I'm
not laughing, however.
There are BTW more uses for a filesize function than just get the size to
malloc memory with. How about for your DBMS and you want to create an
administration tool where you can constantly monitor how big the files are.
Just an extra thingie you would be able to do in standard C.

So you could make sure that the normal I/O to the files happens slower
than it would otherwise. Yes, I could see how some DMBS admins might
think that's actually a good thing. I am laughing now, btw.
This is really getting silly, a function like filesize could be put into the
standard and it would be useful. But as pointed out elsewhere in this thread
this discussion wont help to get the function into it.

Or, you could write one to do what you desire, in far less time than
you have spent so far on this thread, which does whatever it is you
think is appropriate for your platform.
 
R

Randy Howard

Ben Pfaff wrote, On 30/12/07 22:53:

Or Jacob's message had not reached Keith, or it had but Keith has not
read it yet, or he just happened to respond to a message on a different
part of the thread.

Or recognize that NNTP is not real-time sync'd all over the globe, or
that everyone that subscribers to a usenet group reads the contents of
every single posting.
Jacob, you should not assume people are always out to get you.

Even the truly paranoid have real enemies.
 
J

jacob navia

Randy said:
Enlighten as to how this Java function handles real time changes to
file size between the time you make the call and the time you use the
result.

It doesn't. This is up to the programmer.

And when I do

fseek(f,0,SEEK_END);
fread(...);

If the file grows and I am not reading at the end this
is not the language problem but the programmer's problem.

Should we discard fseek/fread too?
 
J

jacob navia

Serve said:
Java already faced them you mean because such an operation is included
in *tadam* the standard library :)

Not only that. There is a utility class to format the result
in human readable form:

public static java.lang.String format(long filesize)

Format the size of a file in human readable form. Anything less
than a kilobyte is presented in kilobytes to one decimal place. Anything
between a kilobyte and a megabyte is presented in kilobytes with to zero
decimal places. Anything greater than one megabyte is presented in
megabytes to two decimal places.

eg.

* format(512) -> 0.5 kb
* format(1024) -> 1.0 kb
* format(2048) -> 2 kb
* format(1024 * 400) -> 400 kb
* format(1024 * 1024) -> 1024 kb
* format(1024 * 1024 * 1.2) -> 1.20 Mb
* format(1024 * 1024 * 20) -> 20.00 Mb

Parameters:
filesize - The size of the file in bytes.
Returns:
The size in human readable form.
 
K

Keith Thompson

jacob navia said:
Keith Thompson wrote: [...]
Obviously if you blindly read the entire file into your 1-megabyte
buffer, you risk an overrun. So don't do that.
[...]

Why do you ignore what I am saying?

I posted an identical message in this thread proving I could not
possibly have a buffer overflow.

You say the same thing without quoting me.

You're right.

In response to some other messages in this thread, no, you're not in
my (nonexistent) killfile, and yes, I think I had read your response.
I was refuting what Gordon had said; I probably should have credited
you for saying much the same thing first. I don't always pay complete
attention to who has said what.

Perhaps you could have saved your "My point is proved" remark until
after I had a chance to respond.
 
R

Randy Howard

It doesn't. This is up to the programmer.

He said that java had face the software engineering issues. Apparently
that assertion was incorrect.
And when I do

fseek(f,0,SEEK_END);
fread(...);

If the file grows and I am not reading at the end this
is not the language problem but the programmer's problem.

Should we discard fseek/fread too?

Is that what passes for logic now?
 
G

Golden California Girls

Serve said:
usenet is funny lol, I'm sure you are smarter irl as you make it out be
right now

But ok, let me rephrase!

If you want to write a program that opens the printer for reading (yes
reading because you just want to get the hypothetical filesize) and you
use fopen("LPT1") will you expect the program to actually open the
printer on Linux?

Is it clear now what I meant? rofl

Until you get some printer that sends back status info and you really can open
it for reading.
 
G

Gordon Burditt

No, you malloc() enough memory for the size of the file *AFTER* the
So? Like Jacob has indeed said 1000 times already, by that same logic you
can't use fread or fgets anymore either.
Somebody else could've made the file smaller when you started fgets on the
last line.

In which case you get an end-of-file indication, or an error, not a buffer
overflow which can let someone execute arbitrary code.
 
G

Gordon Burditt

Use of filesize() to allocate a buffer, and then assuming the file size
I allocate the buffer given by filesize, then I read exactly that with
fread.

You do, perhaps, but others in the thread seem to think, and gave
procedures or code demonstrating that, that indicates otherwise.
If the file has grown I will ignore what goes beyond, and if the file
shrinks, I will have a short read and a too large buffer.

There is NO WAY I COULD HAVE A BUFFER OVERFLOW!

For the nth time:

You seem to have the idea that USENET is instantaneous. Chances are
all the posts you are complaining about were posted before you posted
your first rebuttal.
You are telling just NONSENSE!

No, it's nonsense, and just because YOU wouldn't do it doesn't mean
others won't. Some of them have described doing just that, with
pseudo-code or code, in this discussion.
 
G

Gordon Burditt

malloc enough memory for the size of the file at the time of
There is no buffer overflow as I told you in another message.

I repeat it now since you went silent instead of acknowledging it.

On USENET, it takes at least a week to "go silent". It hasn't
been anywhere near that long. Some people have lives outside
of USENET. And article transmission is not instantaneous.
Messages "cross in the mail" all the time.
1) I call filesize
2) I allocate a buffer with the result+1.
3) I read with fread into that buffer.

You still haven't said anything up to this point about error checking,
or what size you use for fread().
If the file is bigger, the fread argument will ensure I read ONLY
what I allocated. I ignore the rest. If the file is smaller I get
a short read and the buffer is too big.

You still haven't stated what you're using as the argument to fread().
THERE IS NO POSSIBILITY OF A BUFFER OVERFLOW.

I suppose you will stay silent now, instead of acknowledging your
error.

Lack of any error checking is an improvement over the possibility of
a buffer overflow, but not by much.
 
R

Richard Heathfield

jacob navia said:
Of course I am not in his killfile when it comes to
attacking me.

My point is proved. The silence of those people speaks for itself.

If you talk nonsense and we say so, you claim we are attacking you. If you
talk nonsense and we don't say so, you claim we are ignoring you.

Either way, your claim is specious and without foundation. When you talk
nonsense, all it means is that you're talking nonsense. Since you rarely
talk anything else, you should not be surprised when people either say so
(and risk being accused of "attacking" you) or don't say so (and risk
being accused of "ignoring" you).

What are you after? Intelligent discourse about C with your peers? Fine, so
start talking intelligently about C. You may think you are already doing
so but, alas, this is rarely the case.
 
R

Richard Heathfield

Serve Lau said:
Java already faced them you mean because such an operation is included in
*tadam* the standard library :)

Fine. All this means is that the Java standard library designers have taken
some arbitrary decisions and enshrined them in a standard library. Because
they are not fools, we can presume that these arbitrary decisions were
taken as intelligently as possible, but nevertheless the decisions they
have made will necessarily ignore some of the issues discussed in this
thread.
 
P

Paul Hsieh

In my "Happy Christmas" message, I proposed a function to read
a file into a RAM buffer and return that buffer or NULL if
the file doesn't exist or some other error is found.

It is interesting to see that the answers to that message prove that
programming exclusively in standard C is completely impossible even
for a small and ridiculously simple program like the one I proposed.

1 I read the file contents in binary mode, what should allow me
to use ftell/fseek to determine the file size.

No objections to this were raised, except of course the obvious
one, if the "file" was some file associated with stdin, for
instance under some unix machine /dev/tty01 or similar...

I did not test for this since it is impossible in standard C:
isatty() is not in the standard.

I would expect fseek() to return with an error if the file is not
"seekable", such as stdin.
2) There is NO portable way to determine which characters should be
ignored when transforming a binary file into a text file. One
reader (CB Falconer) proposed to open the file in binary mode
and then in text mode and compare the two buffers to see which
characters were missing... Well, that would be too expensive.

Oh, well, if you want to do this stuff in text mode, you need to
iterate character by character. This distinction of "text mode"
versus "binary mode" is nothing but performance problems no matter
what. Personally, log files are the only thing I ever open in text
mode any more.
3) I used different values for errno defined by POSIX, but not by
the C standard, that defines only a few. Again, error handling
is not something important to be standardized, according to
the committee. errno is there but its usage is absolutely
not portable at all and goes immediately beyond what standard C
offers.

Yeah, its just one of the many ways the C language standard encourages
vendors to be non-portable. In any event, its not friendly to re-
entrancy anyways, as the error value needs to be copied out to make
way for other errors to be reported. If you are going to hang onto
error values anyways, you might as well just return them, and make up
your own anyways. That's what I do. (OTOH, I don't consider mutable
static context, such as errno, to be a valid interface for anything.)
We hear again and again that this group is about standard C *"ONLY"*.
Could someone here then, tell me how this simple program could be
written in standard C?

Personally, I think trying to do something like this according to the
standard is a complete waste of time. I have files on disk today that
are in excess of ULONG_MAX in size. intmax_t (which is int64_t on my
system) is plenty large enough to hold the size, so its not like it
couldn't or shouldn't be representable. And of course, my system also
has some nice 64 bit extensions to fseek and ftell, so we know its
possible.

If we ignore file sizes and, say, allow INT_MAX-1 to be good enough
for the size of files, then I would recommend downloading "The Better
String Library" and just calling bread() (or breada()). It
iteratively allocates more and more memory to fit the data that is
read -- you can then call ballocmin() on the result if you want to
keep the memory usage tight. Its just an extra O(n) anyways. Meh.
This confirms my arguments about the need to improve the quality
of the standard library!

Well for fseek and ftell, the case is very clear and obvious.
"unsigned long" is not an appropriate type for file offsets. This was
"corrected" with fgetpos() and fsetpos() except that they took all
arithmetic away from you for some reason, and there is no way to
simply seek to the end of the file with them (without simply reading
it all.) Clearly we need to add something like:

intmax_t fgetfilesize (FILE * fp);
int faddpos (fpos_t * pos, intmax_t offset);
You can't do *anything* in just standard C.

Well that might be an exaggeration. But what can definitively be
said, is that as technology and ideas improve, what the C standard
specifies becomes applicable to a smaller and smaller percentage of
that space. And certainly portable C is diminishing to zero.

What the standard specifies does not scale, and does not take
improving technology into account, except in that it allows platforms
to extend away and do their own thing with respect to implementation
specific behavior. (Which makes the idea of a "standard"
meaningless.)

A fairly straight forward application like Bittorrent cannot be
written in anything resembling portable C code, not even just the file
system back end. It just an embarrassment that a tool that should be
seen as a system level tool is best written in Python on Java.
 
S

Serve Lau

Gordon Burditt said:
On USENET, it takes at least a week to "go silent". It hasn't
been anywhere near that long. Some people have lives outside
of USENET. And article transmission is not instantaneous.
Messages "cross in the mail" all the time.


You still haven't said anything up to this point about error checking,
or what size you use for fread().

Let me answer for you. He uses the value returned by filesize as the size
argument. Is it that hard to understand to see what he's trying to say or am
I just smart? Let me explain some more. He says "1 I call filesize". Then he
says "2) allocate the result + 1". In my unending wisdom I could deduce that
"the result" means The Result from filesize! Applause for me.

And uh, lets just assume for the sake of argument that all functions are
checked for errors, or we could just go on arguing about that it was
omitted.
I thought people here were mature?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,770
Messages
2,569,583
Members
45,075
Latest member
MakersCBDBloodSupport

Latest Threads

Top