size_t or int for malloc-type functions?

[Richard Heathfield]

Cesar Rabak said:

It is possible to have a conforming implementation returning some kind
of 'super size_t' pointer (that could address in principle SIZE_MAX

[squared])?

That shouldn't be hard. It would require a pointer twice as wide as a
size_t, and (since you imply C99) therefore at least 32 bits wide (as
opposed to 30 for C90). Such a system could easily exist right now, as far
as I can see.

What am I missing?

 

[jacob navia]

Joe Wright a écrit :

While I agree that it ought to be as you describe, it often isn't. My
own DJGPP implementation defines..

void *
calloc(size_t size, size_t nelem)
{
void *rv = malloc(size*nelem);
if (rv)
memset(rv, 0, size*nelem);
return rv;
}

Does any C implementation that you know of get it right?

See the thread I started "Request"
Message-ID: <[email protected]>
NNTP posting date: 03 Jan 2007 00:30:01 CET

I wanted to make a list but most people just answered with
polemic...

 

[jacob navia]

Joe Wright a écrit :

While I agree that it ought to be as you describe, it often isn't. My
own DJGPP implementation defines..

void *
calloc(size_t size, size_t nelem)
{
void *rv = malloc(size*nelem);
if (rv)
memset(rv, 0, size*nelem);
return rv;
}

Does any C implementation that you know of get it right?

lcc-win32 gets it right, gcc (after version 3 gets it right
the versions before are buggy), MSVC gets it right (after
version 5 I think)

 

[Richard Tobin]

Does any C implementation that you know of get it right?

[/QUOTE]

lcc-win32 gets it right, gcc (after version 3 gets it right
the versions before are buggy), MSVC gets it right (after
version 5 I think)

As was mentioned before, whether gcc gets it right depends on the C
library you use it with. You can't deduce anything from the version
of gcc without more information.

-- Richard

 

[Cesar Rabak]

Richard Heathfield escreveu:

Cesar Rabak said:

It is possible to have a conforming implementation returning some kind
of 'super size_t' pointer (that could address in principle SIZE_MAX

[squared])?

That shouldn't be hard. It would require a pointer twice as wide as a
size_t, and (since you imply C99) therefore at least 32 bits wide (as
opposed to 30 for C90). Such a system could easily exist right now, as far
as I can see.

What am I missing?

I don't follow your reasoning:

pointer should be at least 32 bit wide in a C99 (lets talk about present
Standard only ;-)

the /theoretical/ size a calloc object _could_ need to allocate has
twice this width

So what you see is a system where the arguments to calloc can have half
the width of the returnable pointer width. Is this what you see as
possible? And conforming to the Standard?

 

[Richard Heathfield]

Cesar Rabak said:

pointer should be at least 32 bit wide in a C99 (lets talk about present
Standard only ;-)

C&V please.

the /theoretical/ size a calloc object _could_ need to allocate has
twice this width

I don't follow your reasoning.

So what you see is a system where the arguments to calloc can have half
the width of the returnable pointer width. Is this what you see as
possible? And conforming to the Standard?

I can see no reason why an implementation with 16-bit size_t and 32-bit
pointers cannot, as a consequence of those choices, be conforming.

 

[Keith Thompson]

Keith Thompson escreveu:
I concur that in theory the wording of the Standard allows this
interpretation, OTOH, can you devise a scheme where a conforming
implementation could have this behaviour?
Sure.

It is possible to have a conforming implementation returning some kind
of 'super size_t' pointer (that could address in principle SIZE_MAX²)?

size_t is an integer type, not a pointer type. There's no
requirement, or even implication, that pointers and size_t have to be
the same size. You could have, for example, a 32-bit size_t and
64-bit pointers. This is quite plausible if a single object can be no
bigger than 2**32 bytes, but the full address space is 64 bits (e.g.,
on a segmented architecture).

calloc() merely has to return a pointer to the allocated object; it
doesn't have to be able to represent its size as a size_t. Remember
too that pointer arithmetic and array indexing are not defined in
terms of size_t; the integer operand or index, respectively, can be of
any integer type. You could have an allocated array of char whose
size cannot be represented as a size_t, but you could still index it
using uintmax_t.

As Harald van D?k (sorry, my newsreader doesn't display his name
properly) pointed out when I raised this question in comp.std.c, the
committee response to DR 260 says that sizeof may overflow.
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_266.htm>

Now if size_t can't represent the sizes of all allowed objects, the
obvious solution is to make size_t bigger. But implementations aren't
required to be reasonable.

 

[Keith Thompson]

Joe Wright a écrit : [...]

While I agree that it ought to be as you describe, it often
isn't. My own DJGPP implementation defines..
void *
calloc(size_t size, size_t nelem)
{
void *rv = malloc(size*nelem);
if (rv)
memset(rv, 0, size*nelem);
return rv;
}
Does any C implementation that you know of get it right?

lcc-win32 gets it right, gcc (after version 3 gets it right
the versions before are buggy), MSVC gets it right (after
version 5 I think)

gcc gets it neither right nor wrong. gcc is a compiler, not a full
implementation; it uses whatever runtime library is available on the
system.

I've lost count of how many times I've pointed this out to you.

 

[CBFalconer]

.... snip ...

Where does the standard say that "the largest single dynamic object
shall be SIZE_MAX or less bytes"? Chapter and verse, please. The
standard says that size_t is the type of the result of the sizeof
operator, but you can't apply sizeof to a dynamically allocatd array
object.

It's a conclusion that has to be drawn, because it does say
somewhere that size_t can describe the size of any object.

 

[CBFalconer]

I don't think that the standard library should try to interpret the
intent of the programmer. Rather, the programmer should know what he's
doing, (in this case the modulo 2^n nature of unsigned integers in C).

If hand-holding is needed we have a pleothra of other languages.

I agree with Jacob here. The spec for calloc says it receives two
size_ts. Nothing is said about a product. The fact that a
multiplication has to be performed is a deep conclusion from many
other specifications, and far from obvious to someone non-aware,
including most of us before Jacob brought it up. We may not have
noticed because very few really use it. All other allocation
routines return NULL for failure, so should calloc. However calloc
is a system routine, so it can know the precise consequences.

Did you ever think why it is a system routine, when it is
(apparently) so easily implemented with a combination of alloc and
memset?

void *badcalloc(size_t items, size_t nmemb) {
void *p;
if (p = malloc(items * nmemb) memset(p, items * nmemb, 0);
return p;
}

is out and out wrong.

 

[CBFalconer]

.... snip ...

Not quite. A calloc implementation that returns a non-NULL pointer
without allocating the requested space is faulty. A calloc
implementation that actually allocates the requested space (more than
SIZE_MAX bytes) and returns a pointer to it would be unusual but, as
far as I can tell, conforming.

Consider the implications. We are always yelling at people for not
remembering the size they allocated. If a size_t can't hold that
size, where are they to remember it? After all, size_t is
guaranteed to hold the size of any object.

Joe Innocent, on a 16 bit machine, with size_t(-1) == 65535, wants
to save a few 80 char. wide screens, holding 1000 lines each.
Quite reasonable so far. So he calls calloc(1000, 81), which
succeeds. BOOM arrives later.

 

[Peter Nilsson]

Consider the implications. We are always yelling at people for not
remembering the size they allocated.

Are you? I generally only need the N, not the N * sizeof *p.

If a size_t can't hold that size, where are they to remember it?

Maybe with two size_ts instead of one perhaps? But again, I
generally only need the N since the sizeof can be produced
at any time.

After all, size_t is guaranteed to hold the size of any object.

You have yet to demonstrate that the standard actually says that.
Not everyone is convinced by a waving of hands and you saying it's
there 'somewhere'. ;-)

 

[Richard Heathfield]

CBFalconer said:

Consider the implications. We are always yelling at people for not
remembering the size they allocated. If a size_t can't hold that
size, where are they to remember it?

They had the information when they gave it to calloc. Let Them Not Forget.

The information can comfortably be held in two size_t objects - one for an
element count, and the other for an element size.

 

[Keith Thompson]

It's a conclusion that has to be drawn, because it does say
somewhere that size_t can describe the size of any object.

I don't believe it says that, but if you can cite C&V I'd appreciate
it. See also DR 266; if the committee believed that size_t can
describe the size of any object, the response would have been different.
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_266.htm>

 

[Steve Summit]

With perfect 20/20 hindsight I agree, but...

Right. In fact, it *must* do so

I'd be much more inclined to agree with this if the Standard
explicitly said

If the product nmemb * size cannot be represented as a
size_t, the calloc function returns NULL.

I wish the Standard did say that. This is really a grey area, a
longstanding source of bugs and misunderstanding. (See e.g. the
related c.l.c. FAQ 7.16.)

Without the explicit statement (and as the arguments presented by
several knowledgeable posters in this thread prove), I think an
implementor could be excused for assuming that the Standard's
intent was

If the product nmemb * size cannot be represented as a
size_t, the behavior is undefined.

....although since the Standard doesn't say *that* explicitly,
either, it's sort of doubly or meta undefined!

 

[Richard Tobin]

It's a conclusion that has to be drawn, because it does say
somewhere that size_t can describe the size of any object.

I looked for that, but all I found was that size_t is the type
returned by sizeof, and that cannot be applied to objects allocated by
[mc]alloc().

-- Richard

 

[Richard Tobin]

Did you ever think why it is a system routine, when it is
(apparently) so easily implemented with a combination of alloc and
memset?

calloc() was introduced along with malloc() in seventh edition Unix,
before memset() (or even bzero()). At that time the arguments were of
type "unsigned", and memory was obtained using sbrk(), which took an
int argument. So I don't think it's very likely that calloc() exists
for any reason connected with allocating large amounts of memory.
Most likely it was just intended as a convenience, and it's just luck
that it might happen to allow you to allocate a bigger object than
size_t can measure.

Has anyone found any systems where you can allocate more with calloc()
than size_t can measure?

-- Richard

 

[Keith Thompson]

Has anyone found any systems where you can allocate more with calloc()
than size_t can measure?

Incidentally, I've started a thread about this on comp.std.c, subject
"May calloc() allocate more than SIZE_MAX bytes?".

 

[CBFalconer]

It's a conclusion that has to be drawn, because it does say
somewhere that size_t can describe the size of any object.

I looked for that, but all I found was that size_t is the type
returned by sizeof, and that cannot be applied to objects
allocated by [mc]alloc().

Regardless, I believe we can imply that conclusion. C can only
operate on objects. The size of an object is garnered by sizeof,
and is of type size_t. malloc doesn't return generic objects, it
returns pointers.

 

[Jun Woong]

Jun Woong wrote: [...]

That should be

if (n == 0 || s == 0 || SIZE_MAX / n > s) {

or something like that.

The version I have just put in nmalloc.c (not yet published) is:

/* calloc included here to ensure that it handles the
same range of sizes (s * n) as does malloc. The
multiplication n*s can wrap, yielding a too small
value, so we must ensure calloc rejects this.
*/
void *ncalloc(size_t n, size_t s)
{
void *result;
size_t sz;

result = NULL;
if (!n || ((size_t)-1) / n > s) {
sz = n * s;
if ((result = nmalloc(sz))) memset(result, 0, sz);
}
return result;
} (* ncalloc *)

which makes the output of ncalloc be that of nmalloc(0) whenever
either n or s is 0. I think there is still a possible glitch when
((size_t)-1) / n) == s.

Unless you decide to use (size_t)-1 for a special purpose, there is
no reason to exclude the ((size_t)-1 / n) == s case.

If (max / n) - d >= s, then it is always guaranteed that
max >= n * s, where '/' indicates the mathematical division and
all are positive.

The only thing that needs protection
agains n==0 is the division. s==0 will simply force sz==0.

I put both cases into the test since it is up to the programmer to
decide which one is used as a divisor. In my implementation of
calloc, the 'if' statement looks like

if (n == 0 || ((size_t) -1) / n >= s)


--
Jun, Woong (woong at icu.ac.kr)
Samsung Electronics Co., Ltd.

``All opinions expressed are mine, and do not represent
the official opinions of any organization.''