General method for dynamically allocating memory for a string

[Richard Heathfield]

Philip Potter said:

Richard, while you have earned a lot of my respect and admiration through
your posting, I do feel that you are on a little bit of a crusade here.

Perhaps, but you may be mistaken as to the issues at stake.

You seem to be arguing that because garbage collection can't protect
against all memory leaks without care from the user, it is worthless.

No, I'm sure it has a place in other languages. I just don't think it sits
very well with C, for at least two reasons:

1) As we have already seen upthread, automatic garbage collection (AGC)
encounters serious difficulties when faced with C's heavy use of pointers
for keeping track not only of memory blocks but of particular positions
within them.

2) C is commonly used where not only performance but even timing (in
real-time stuff) is an important criterion for acceptance of the program,
and AGC implementations are notorious for introducing arbitrary delays at
various times through a program's run.

As I have already agreed, AGC is undoubtedly of value to some people in some
situations. I just don't think AGC is a good fit for C programmers, in the
general case. That's not to say that it might not be useful on occasion,
but I don't believe such occasions are sufficiently common to make the
subject a worthwhile diversion away from the topic of this newsgroup, which
is C programming, *not* "C programming + Jacob Navia's AGC extensions".

Were this discussion taking place in Jacob Navia's own newsgroup,
comp.compilers.lcc, then I would not have raised any objections at all.

 

[Philip Potter]

Perhaps, but you may be mistaken as to the issues at stake.

Perhaps. I haven't said very much yet, so it's a bit early to be drawing
conclusions.

No, I'm sure it has a place in other languages. I just don't think it sits
very well with C, for at least two reasons:

1) As we have already seen upthread, automatic garbage collection (AGC)
encounters serious difficulties when faced with C's heavy use of pointers
for keeping track not only of memory blocks but of particular positions
within them.

2) C is commonly used where not only performance but even timing (in
real-time stuff) is an important criterion for acceptance of the program,
and AGC implementations are notorious for introducing arbitrary delays at
various times through a program's run.

You're right that AGC is not always suited to C. However, you said above:

I just don't think it sits
very well with C, for at least two reasons:

suggesting you think it is /never/ suited to C. You have yet to convince me
of this. In non-performance-critical, non-realtime situations, I see nothing
wrong with plugging in a garbage collector. And even in realtime situations,
there are ways to ensure the garbage collector will not activate in critical
sections of code.

As I have already agreed, AGC is undoubtedly of value to some people in some
situations. I just don't think AGC is a good fit for C programmers, in the
general case. That's not to say that it might not be useful on occasion,
but I don't believe such occasions are sufficiently common to make the
subject a worthwhile diversion away from the topic of this newsgroup, which
is C programming, *not* "C programming + Jacob Navia's AGC extensions".

Were this discussion taking place in Jacob Navia's own newsgroup,
comp.compilers.lcc, then I would not have raised any objections at all.

If all you were complaining about was topicality, then you should have said
so. But it looks to me more like you were complaining about garbage
collection itself, and you yourself were part of the "diversion" you are now
complaining about.

Anyhow, I agree with you that this is offtopic, so perhaps we should leave
this here.

Philip

 

[Richard Heathfield]

Philip Potter said:

Perhaps. I haven't said very much yet, so it's a bit early to be drawing
conclusions.

But surely "you may be mistaken" is not a conclusion?

You're right that AGC is not always suited to C. However, you said above:


suggesting you think it is /never/ suited to C.

I don't think I suggested that at all. I think I suggested that it doesn't
sit very well with C, which is not quite the same thing!

But I'm very much in favour of people being allowed to do what they want to
do as long as it doesn't hurt other people or frighten the horses. So if
people want to use AGC in some extended version of C, great, more power to
them - but I do very much hope that ISO continue to mandate the provision
of support for manual garbage collection via free(), so that those of us
who prefer to maintain manual control over our own programs' garbage
collection can continue to have that control.

<snip>

 

[Tak-Shing Chan]

jacob navia said:


<shrug> Finding ways to fool it is a moderately trivial exercise:

p = malloc(n * sizeof *p);
pos = ftell(fp);
fwrite(&p, sizeof p, 1, fp);
p = NULL;
fseek(fp, pos, SEEK_SET);
fread(&p, sizeof p, 1, fp);

Your rebuttal is worse than the disease, because:

(1) If fwrite or fread fails, any further use of p will
invoke undefined behaviour.

(2) Even with error checking inserted, you would still be
leaking memory when fread fails.

(3) In general, writing pointers to files is always
nonportable. So, I am not sure why you are posting this in a
group that values portability.

Tak-Shing

 

[Ben Pfaff]

(3) In general, writing pointers to files is always
nonportable. So, I am not sure why you are posting this in a
group that values portability.

Writing a pointer to a file is *not* always non-portable. It is
portable to write a pointer to a file, read it back within the
same run of the program, and then use the pointer (as long as the
lifetime of the associated memory has not been reached). This is
what Richard did.

Now, whether it's advisable to do so is another question. I
haven't encountered many (any?) situations where it is.

 

[Walter Roberson]

Writing a pointer to a file is *not* always non-portable. It is
portable to write a pointer to a file, read it back within the
same run of the program, and then use the pointer (as long as the
lifetime of the associated memory has not been reached).

Reference, please?

My recollection is that the relevant section says only that
when something is written out to a binary file, that the same
binary value will be read back in. When, though, it comes to a pointer,
that doesn't promise that the reconstituted pointer points to anything.

There is a section of the standard that talks about writing out
pointers and reading them back in, but that has to do with using
the %p printf() format, which Richard did not do.

 

[Tak-Shing Chan]

Writing a pointer to a file is *not* always non-portable. It is
portable to write a pointer to a file, read it back within the
same run of the program, and then use the pointer (as long as the
lifetime of the associated memory has not been reached).

An intervening realloc could mess it up.

This is
what Richard did.

Now, whether it's advisable to do so is another question. I
haven't encountered many (any?) situations where it is.

Tak-Shing

 

[Ben Pfaff]

An intervening realloc could mess it up.

realloc terminates the memory's lifetime.

 

[Ben Pfaff]

Reference, please?

My recollection is that the relevant section says only that
when something is written out to a binary file, that the same
binary value will be read back in. When, though, it comes to a pointer,
that doesn't promise that the reconstituted pointer points to anything.

Do you believe that you can portably copy pointers with memcpy?
Then why wouldn't you believe that you can portably read and
write pointers in a file, under the stated conditions? You need
not even use a binary file: you could encode the bits in the
pointers as plain text in hexadecimal or another notation, then
decode them when you read them back in.

 

[Richard Heathfield]

Tak-Shing Chan said:

On Fri, 1 Sep 2006, Richard Heathfield wrote:


Your rebuttal is worse than the disease, because:

No, it isn't. I agree that it's not a terribly bright idea, but it is legal
C. Yes, I omitted error-checking for brevity. This is not a student
discussion. It's a discussion amongst people who all know the importance of
error-checking, and we're all big enough and ugly enough to know where the
error-checking would go. This is just short-hand, that's all. If you want
full error-checking (as indeed I would, in a real program), you are
perfectly capable of adding it yourself.

(1) If fwrite or fread fails, any further use of p will
invoke undefined behaviour.

Sure, but error-checking would cope with that in a real program.

(2) Even with error checking inserted, you would still be
leaking memory when fread fails.

Yes, that's perfectly true. But when it succeeds, you would not be leaking
memory.

(3) In general, writing pointers to files is always
nonportable.

Why? If you mean that freestanding implementations don't need to have
filesystems, you're trivially but IMHO irrelevantly correct.

 

[Tak-Shing Chan]

realloc terminates the memory's lifetime.

I don't think so. The new object might overlap with the old
one. Perhaps you meant ``object'' rather than ``memory''?

Tak-Shing

 

[Richard Heathfield]

Walter Roberson said:

Reference, please?

My recollection is that the relevant section says only that
when something is written out to a binary file, that the same
binary value will be read back in. When, though, it comes to a pointer,
that doesn't promise that the reconstituted pointer points to anything.

fread and fwrite work "as if" they are successive calls to fgetc and fputc
respectively. These functions may ostensibly deal in ints, but the Standard
requires that they actually read in and write out unsigned chars. So fread
and fwrite effectively read and write objects as if they were a collection
of unsigned chars. In other words, what is being read/written is the object
representation.

Is it your contention that two objects of the same type with the same object
representation can have different values?

There is a section of the standard that talks about writing out
pointers and reading them back in, but that has to do with using
the %p printf() format, which Richard did not do.

Only because I couldn't face the sheer yukkiness of writing a scanf call.

 

[Richard Heathfield]

Tak-Shing Chan said:

An intervening realloc could mess it up.

There was no intervening realloc in the code which you said was
non-portable. If you had claimed "in general, writing pointers to files is
always non-portable if you realloc before you read them back in", that
would be different (and utterly irrelevant to the discussion). But you
didn't.

 

[Ben Pfaff]

I don't think so. The new object might overlap with the old
one. Perhaps you meant ``object'' rather than ``memory''?

Yes, that's more precise.

 

[Tak-Shing Chan]

Tak-Shing Chan said:


There was no intervening realloc in the code which you said was
non-portable. If you had claimed "in general, writing pointers to files is
always non-portable if you realloc before you read them back in", that
would be different (and utterly irrelevant to the discussion). But you
didn't.

In any case, it would hide serious errors (such as having an
intervening realloc or free) where conforming implementations are
not required to issue a diagnostic.

Tak-Shing

 

[jacob navia]

I don't think so. The new object might overlap with the old
one. Perhaps you meant ``object'' rather than ``memory''?

Tak-Shing

You raise a very importaznt point here, that furthers the arguments for
an automatic garbage collector.

When you do a

q = realloc(p,2*n);

after a successful realloc the p pointer and ALL THE ALIASES you have
done for that object, "including but not limited to":

o structures that contain that pointer in some field
o functions, that hold aliases to that object address in the stack
in their parameter list
o other local variables that use pointers based on that pointer,
for instance p+5 or &p[78]

ARE ALL INVALID and must be ALL invalidated!!!

Manually.

Using the GC you do

q = GC_malloc(2*n);
if (q) {
memcpy(q,p,n); // Copy the old object
p = NULL;
}
else
//No more memory Handle error

Now q contains the reallocated object but the old
storage will be freed only (and only then) when there are no
pointers to it!

Obviously, if you are 100% sure that there are no aliases to
the object, you can use GC_realloc();

But this is not without problems too, since the aliases will
point to old object copies and not to the good one, and
those copies will NOT be updated.

This can be a problem, or may be is harmless, it depends on the
application. For instance if the new object is updated by
adding members to it (a table of structures for instance) it
can be OK to keep the old shorter version, but that could also be dangerous.

In the case of
struct {
int nbOfElements;
T data;
};

after a realloc of "data", the old pointer stored elsewhere
will be utterly wrong after a realloc, and a GC solution could make
the problem even worse.

But this is a matter of discipline of alias usage and discipline
of pointer usage in general.

Another thread of discussion.


Yes, memory management *is* tricky.

 

[Richard Heathfield]

Tak-Shing Chan said:

In any case, it would hide serious errors

What would? You seem to be talking about hypothetical code hiding
hypothetical errors. The actual posted code seems to have passed you by
completely.

 

[Ian Collins]

Philip Potter said:



No, I'm sure it has a place in other languages. I just don't think it sits
very well with C, for at least two reasons:

1) As we have already seen upthread, automatic garbage collection (AGC)
encounters serious difficulties when faced with C's heavy use of pointers
for keeping track not only of memory blocks but of particular positions
within them.

2) C is commonly used where not only performance but even timing (in
real-time stuff) is an important criterion for acceptance of the program,
and AGC implementations are notorious for introducing arbitrary delays at
various times through a program's run.

As I have already agreed, AGC is undoubtedly of value to some people in some
situations. I just don't think AGC is a good fit for C programmers, in the
general case. That's not to say that it might not be useful on occasion,
but I don't believe such occasions are sufficiently common to make the
subject a worthwhile diversion away from the topic of this newsgroup, which
is C programming, *not* "C programming + Jacob Navia's AGC extensions".

One situation where GC can be used to good effect with C is as a
diagnostic tool. I often soak test applications linked with a GC
library that logs reclaimed memory and use the library to highlight any
memory leeks that occur. If the application is correctly written, the
GC shouldn't reclaim any blocks.

I would never countenance using the GC library as a band-aid for poor
programming practice, but as another tool in the box, it has its place.

 

[Richard Heathfield]

Ian Collins said:

One situation where GC can be used to good effect with C is as a
diagnostic tool.

Absolutely. Likewise automatic bounds checking, with which I would never
burden production code but which has its place in a test rig.

I often soak test applications linked with a GC
library that logs reclaimed memory and use the library to highlight any
memory leeks that occur.

Not broccoli, then?

Seriously, I don't actually use a GC lib for detecting memory leaks, but I
do use a wrapper around *alloc and free which I wrote specifically for
performing such detections. But it really slows the code down, and
generates colossal logfiles, so it gets ripped out of production code via a
#define (or rather, the absence of one).

 

[Ian Collins]

Ian Collins said:




Absolutely. Likewise automatic bounds checking, with which I would never
burden production code but which has its place in a test rig.




Not broccoli, then?

Must be out of season....

Seriously, I don't actually use a GC lib for detecting memory leaks, but I
do use a wrapper around *alloc and free which I wrote specifically for
performing such detections. But it really slows the code down, and
generates colossal logfiles, so it gets ripped out of production code via a
#define (or rather, the absence of one).

That's one advantage of the GC library, it imposes minimal overhead and
only logs real or potential leaks.