New book - 'C of Peril'

[Keith Thompson]

But there is a valid point there. Certain forms of "creativity" are
inappropriate. (I'd expound further if I had time.)

What I was thinking of was number 8 from Henry Spencer's "The Ten
Commandments for C Programmers":

Thou shalt make thy program's purpose and structure clear to thy
fellow man by using the One True Brace Style, even if thou likest
it not, for thy creativity is better used in solving problems than
in creating beautiful new impediments to understanding.

(The "One True Brace Style" refers to the style used in K&R; even if
you prefer a different style, the point is a good one.)

<http://www.lysator.liu.se/c/ten-commandments.html>

(Certainly the idea that programming is not the place to express
creativity is ludicrous. Programming isn't just an art, but neither
is it a science; it's a craft.)

 

[Keith Thompson]

You are aware this will utterly f*** up things on many platforms ?
I wouldn't advocate this knowing that it may break if I were you ...

[edited for content]

Many platforms? Name three.

 

[dandelion]

same for the eternal free/realloc/malloc problems. I have been
arguing here for a GC based solution, and there is a lot of other
solutions too.

Introducing Garbage Collection in C wil render the language useless to any
application-developer in need of (hard) Real Time. That *would* be a pity.

 

[jacob navia]

Introducing Garbage Collection in C wil render the language useless to any
application-developer in need of (hard) Real Time. That *would* be a pity.

There is no "introduction of garbage collection" in the language itself!

The standard library isn't changed either.

The only thing that changes is

#define malloc(a) GC_malloc(a)
#define free(a)

There are no language modifications whatsoever.

jacob

 

[dandelion]

There is no "introduction of garbage collection" in the language itself!

The standard library isn't changed either.

The only thing that changes is

#define malloc(a) GC_malloc(a)
#define free(a)

There are no language modifications whatsoever.

And the garbage collection routine? At some point some GC will need to
be done.

 

[jacob navia]

And the garbage collection routine? At some point some GC will need to
be done.

It is done at each allocation.
There are several levels: mini-gcs, full gc, etc.

 

[CBFalconer]

There is no "introduction of garbage collection" in the language itself!

The standard library isn't changed either.

The only thing that changes is

#define malloc(a) GC_malloc(a)
#define free(a)

There are no language modifications whatsoever.

Those redefinitions are specifically forbidden by the standard, and
for good reasons, which in turn should be obvious to any
implementor.

 

[jacob navia]

Those redefinitions are specifically forbidden by the standard, and
for good reasons, which in turn should be obvious to any
implementor.

That is in the case you want to replace all your calls to
malloc with the gc (a sensible solution)

You are not forced to do that, since you can always explicitely
call GC_malloc. You should never call free however, because if
you call free with a GC_malloced object chaos will ensue.

Those errors can be very difficult to catch, hence the proposed defines.
jacob

 

[Keith Thompson]

There is no "introduction of garbage collection" in the language itself!

The standard library isn't changed either.

The only thing that changes is

#define malloc(a) GC_malloc(a)
#define free(a)

There are no language modifications whatsoever.

Apart from the legality questions of redefining malloc(), wouldn't you
want a GC_free() routine as well? After free(a), the variable "a"
will still contain a copy of the (now indeterminate) pointer.
Presumably that would prevent the garbage collector from collecting
memory indirectly pointed to by "a". (The program could set a to
NULL, but that's an extra requirement.)

It's not true that there are no language modifications whatsoever.
Values, including pointers, are made of bytes, which are made of bits.
A strictly conforming program can convert a pointer to an array of
unsigned char, destroy the original pointer value, complement the
unsigned char values, go off and do some other stuff, then reconsitute
the pointer from the scrambled bytes and dereference it. If GC is
being used, once the pointer value doesn't exist in its original form
the system can deallocate whatever it points to.

That's not to say that it would be unreasonable to provide garbage
collection in a C implementation, but it would be a change in the
semantics of the language.

 

[jacob navia]

Apart from the legality questions of redefining malloc(), wouldn't you
want a GC_free() routine as well? After free(a), the variable "a"
will still contain a copy of the (now indeterminate) pointer.
Presumably that would prevent the garbage collector from collecting
memory indirectly pointed to by "a". (The program could set a to
NULL, but that's an extra requirement.)

There *is* a GC_free routine, but its use is not really recommended.
Sometimes it can be useful to use it when you are 100% sure that a
memory block will never be used again.

If we would have to use GC_free, the utility of an automatic GC would
be greatly reduced...

It's not true that there are no language modifications whatsoever.
Values, including pointers, are made of bytes, which are made of bits.
A strictly conforming program can convert a pointer to an array of
unsigned char, destroy the original pointer value, complement the
unsigned char values, go off and do some other stuff, then reconsitute
the pointer from the scrambled bytes and dereference it. If GC is
being used, once the pointer value doesn't exist in its original form
the system can deallocate whatever it points to.

If you use the GC you have to leave the bits in the pointers as they
are. If you hide them, the block could be collected if there are no
other references to it.

This is a "change" in the way you use pointers (maybe, the case you
mention is highly hypotetical), since you can't store them on disk,
or in the "windows extra bytes" (where the collector doesn't look),
or Xor them and expect the pointer to still be valid afterwards.

This is a limitation of the collector/compiler.

A clever compiler would see what you are doing and store a hidden copy
of the pointer somewhere

That's not to say that it would be unreasonable to provide garbage
collection in a C implementation, but it would be a change in the
semantics of the language.

The fact that you shouldn't store the pointers to live objects in
the disk or change their value is a "semantic change to the language" ???

Those kind of operations with pointers are quite rare, and I have
never seen them. But yes, if you insist, there *are* some limitations
and constraints imposed to you, as with any other software that you use.

For 99.9999 % of the applications, where pointers are stored normally
in local variables or in data structures, nothing changes.

Please bear in mind that I am not telling "GC is the best thing since
sliced bread". I am not a salesman. The GC can be useful in many
applications and relieves you from the terrible problems of making
errors in malloc/free, what is a big advantage.

But it can produce errors if you keep unneeded pointers around, errors
that are very difficult to track! I would not recommend it in real
time systems either, but those systems do not use even the "malloc"
system call anyway.

In many PC applications however, it gives the C language a huge
advantage, simplifying the applications, simplifying the programming.

jacob

 

[CBFalconer]

That is in the case you want to replace all your calls to
malloc with the gc (a sensible solution)

You are not forced to do that, since you can always explicitely
call GC_malloc. You should never call free however, because if
you call free with a GC_malloced object chaos will ensue.

Those errors can be very difficult to catch, hence the proposed
defines.

As I said, the proposed defines themselves are illegal. You cannot
redefine malloc, nor free, nor realloc. Any programmer doing so
should be summarily shot at dawn. This avoids generating such
"difficult to catch" errors. From N869:

7.1.3 Reserved identifiers

[#1] Each header declares or defines all identifiers listed
in its associated subclause, and optionally declares or
defines identifiers listed in its associated future library
directions subclause and identifiers which are always
reserved either for any use or for use as file scope
identifiers.

-- All identifiers that begin with an underscore and
either an uppercase letter or another underscore are
always reserved for any use.

-- All identifiers that begin with an underscore are
always reserved for use as identifiers with file scope
in both the ordinary and tag name spaces.

-- Each macro name in any of the following subclauses
(including the future library directions) is reserved
for use as specified if any of its associated headers
is included; unless explicitly stated otherwise (see
7.1.4).

-- All identifiers with external linkage in any of the
following subclauses (including the future library
directions) are always reserved for use as identifiers
with external linkage.143)

-- Each identifier with file scope listed in any of the
following subclauses (including the future library
directions) is reserved for use as macro and as an
identifier with file scope in the same name space if
any of its associated headers is included.

 

[Keith Thompson]

Apart from the legality questions of redefining malloc(), wouldn't you
want a GC_free() routine as well? After free(a), the variable "a"
will still contain a copy of the (now indeterminate) pointer.
Presumably that would prevent the garbage collector from collecting
memory indirectly pointed to by "a". (The program could set a to
NULL, but that's an extra requirement.)

I didn't read quite carefully enough. Your free() macro means that
any calls to free() will do nothing. But that still leaves the
problem of letting the garbage collector know that a pointer is no
longer being used.

A program that assumes standard malloc/free behavior might do
something like this:

struct node *root = malloc(sizeof *root);
/*
* Build a huge tree structure with root pointing to the
* root of the tree.
*/
/*
* Work with the tree.
*/
/*
* Recursively free() all the nodes of the tree.
*/

With your GC system, the free() calls do nothing, root still points to
the root of the tree when you're done, and the garbage collector has
no way of knowing that you're not still using the allocated nodes.

Also, you need to worry about interaction between GC and the standard
malloc/free, which is still likely to be used by other pieces of your
program. Just for starters, you'll need to redefine calloc() and
realloc(). But you also have to worry about non-standard calls like
strdup(), which implicitly calls malloc().

 

[Eric Sosman]

[...]
The fact that you shouldn't store the pointers to live objects in
the disk or change their value is a "semantic change to the language" ???

Yes, of course.

Those kind of operations with pointers are quite rare, and I have
never seen them. But yes, if you insist, there *are* some limitations
and constraints imposed to you, as with any other software that you use.

I haven't seen live pointers stored in files, but I've
certainly seen them mangled. One program I worked with (I
didn't write this part, but I struggled mightily when porting
it to another machine) used pointer-mangling for obfuscation,
to make it harder for someone with a debugger to defeat the
license enforcement. Oh, but the authors were paranoid: the
routine that checksummed the license data periodically was
not called by name, but via a pointer mangled at compilation
and de-mangled on the fly in a register; when interference
was detected it poked another mangled pointer elsewhere to
cause another function to exit the program about a half-
minute later; and just in case the cracker figured that one
out, a pointer to fopen() somewhere in the "save file" routine
was XOR'ed with `(unsigned long)fopen ^ (unsigned long)abort' --
yes, highly non-portable and dirty beyond filthiness, but it's
a counter-example indicating that real-world programs really
do use pointer-mangling.

For 99.9999 % of the applications, where pointers are stored normally
in local variables or in data structures, nothing changes.

I bet the horror stories about the 0.0001% would be
entertaining.

Please bear in mind that I am not telling "GC is the best thing since
sliced bread". I am not a salesman. The GC can be useful in many
applications and relieves you from the terrible problems of making
errors in malloc/free, what is a big advantage.

The only error it avoids directly is the memory leak.
There's no protection against allocating less memory than you
(try to) use, or against freeing something that's not allocated.
Of course, if you really decide to rely on GC you avoid the
latter kind of error by not calling free() at all, so some
improvement beyond the direct influence is present. But I'm
not convinced that "making errors in malloc/free" is as much a
problem as you claim. Misuse of pointers, buffer overruns --
these, I think, are probably far more common, and are not
addressed by GC.

But it can produce errors if you keep unneeded pointers around, errors
that are very difficult to track! I would not recommend it in real
time systems either, but those systems do not use even the "malloc"
system call anyway.

Some do, some don't.

In many PC applications however, it gives the C language a huge
advantage, simplifying the applications, simplifying the programming.

If you want Lisp, you know where to find it. Just follow
the trail of discarded parentheses ;-)))

 

[Eric Sosman]

As I said, the proposed defines themselves are illegal. You cannot
redefine malloc, nor free, nor realloc. Any programmer doing so
should be summarily shot at dawn. [...]

I plead extenuating circumstances, and throw myself
on the mercy of the Court. I knew it was wrong when I
did it, but I was desperate, Your Honor. I could see
that my program was leaking memory, and I'd been over
and over it without finding the leak, and all my efforts
got nowhere, and the pills weren't helping any more ...

Well, I >sob< did it. I confess, I confess, I went
and did it. Here is the awful extent of my shame, for
all to see and shudder at:

void *md_malloc(size_t);
void *md_calloc(size_t,size_t);
void *md_realloc(void*,size_t);
void md_free(void*);
#define malloc md_malloc
#define calloc md_calloc
#define realloc md_realloc
#define free md_free

Oh, I know I deserve your anger! But hear the plea
of a programmer in desperate straits! Once I took that
first awful step, I was hooked: it was an addiction, Your
Honor, the drug of non-conformance had polluted my soul
and I was no longer in control of my actions. Out of my
mind with the delirium of having gotten away with it, I
compounded my felony:

void *md_xmalloc(size_t);
void *md_xcalloc(size_t,size_t);
void *md_xrealloc(void*,size_t);
#define xmalloc md_xmalloc
#define xcalloc md_xcalloc
#define xrealloc md_xrealloc

thus violating not only the Standard, but the non-Standard.

Oh, Your Honor, have mercy. I was out of my head.

But I found the leak (it was in the implementation's
non-Standard stat() function), and I found a work-around.
And although I'm sorry -- truly, abjectly sorry, Your
Honor -- in a strange way I'm really not sorry at all.

Come to think of it, I'm not only "not sorry" but GLAD!
Yes! Glad! Ha-hah, do your worst, you shrivelled tyrant!
I laugh in your pimply face, I scorn your >sneer< conformity!
And I cry out, in one last act of proud defiance

"Shoot if you must this old gray head,
But spare your coder's hack," he said.

Besides, I don't fear you and your pipsqueak firing
squad. Standard-shackled all, they couldn't hit the br-

(TRANSCRIPT ENDS. FURTHER DEPONENT SAITH NOT.)

 

[CBFalconer]

As I said, the proposed defines themselves are illegal. You cannot
redefine malloc, nor free, nor realloc. Any programmer doing so
should be summarily shot at dawn. [...]

I plead extenuating circumstances, and throw myself
on the mercy of the Court. I knew it was wrong when I
did it, but I was desperate, Your Honor. I could see
that my program was leaking memory, and I'd been over
and over it without finding the leak, and all my efforts
got nowhere, and the pills weren't helping any more ...

Oh very well. Just this once, provided you discard the infringing
code after the specialized operation of finding the bug. This is
not quite the same as using such redefines for portable code.

 

[Paul L Daniels]

Alex,

I entirely agree with your comments regarding spelling/grammar.

Incidently, thanks for giving me the degree of doubt regarding it not
being my primary language, however, it is :-|

That said, as mentioned previously, the book will be passed over to a
suitable qualified person for editing.

Paul.

 

[Paul L Daniels]

Jacob,

I'm not seeking to provide anything 'new' or revolutionary. Rather, the
primary point of the book is to provide a guide into what can go wrong
(apart from my spelling/grammar) and perhaps some suggestions to side-step
or avoid the issues.

Paul.

 

[Paul L Daniels]

It seems like the comments about creativity are a hot-topic....

But there is a valid point there. Certain forms of "creativity" are
inappropriate. (I'd expound further if I had time.)

I'll take the time to expand the comment in later revisions. However,
Keith is on the point that I was trying to express. There is a lot of
_undue_ creativity in many C sources. One could consider it similar to
the first rule of optimisation "Don't optimise".

I think the first thing I should do before expanding anything else is list
Arthur as a "Contributor" and credit him ;-)

 

[Bart]

Alex,

I entirely agree with your comments regarding spelling/grammar.

Incidently, thanks for giving me the degree of doubt regarding it not
being my primary language, however, it is :-|

I've looked briefly at this (at present rather slim) book and can't
remember seeing any obvious spelling or grammatical errors.

It's fine.

Bart.

 

[Paul L Daniels]

Bart,

I've looked briefly at this (at present rather slim) book and can't
remember seeing any obvious spelling or grammatical errors.

Hope you're wearing your flame-suit. They're going to descend on you
like... like... oh I have no idea.

There are a few glitches in the text to the eye that is looking for
them, therein lies the differences in opinions.

One of the first tricks to dismembering someone over their work is to
disect their spelling/grammar, it's easy to do and a lot of people like to
do it.

What struck me hardest though was that the text was actually run through a
spell checker (via LyX). Obviously it was insufficient.

Paul.