size_t or int for malloc-type functions?

[Kenny McCormack]

Thats impossible, if the argument is unsigned.

Incorrect. In the C language, if you pass a signed integer (for
example, the number -4) to a function where the formal parameter of that
function is declared unsigned (for example, malloc()), the "default
arithmetic conversions" (something you should read up on, by the way),
will convert that signed number to an unsigned, in order to be
compatible with the formal function declaration. On many systems,
numbers like -4 (negative numbers with small absolute values) will be
converted to very large unsigned numbers.

That's what we are talking about here. Do try to keep up.

Huh? So you sacrifice half the possible address space, to cater for a
stupid programming error. Actually, this sounds about par for the
course, given the recent string thread.


It works fine on all my general purpose computers. That is to say, it
fails if the attempt is for too much memory, and it works otherwise.
What else would we reasonably expect it to do?

Google for "malloc overcommit Linux OOM" and get back to us, OK?

 

[Kenny McCormack]

Thats impossible, if the argument is unsigned.

Why don't you take a leaf out of P J Plauger's book and address the
real question, instead of being a tosser and pretending not to
understand?[/QUOTE]

Because this is clc and that's what they (McIntyre and Heathfield, among
others) do. Get with the program!

The value the user supplies is negative. It's converted to a positive
value as part of the function calling process.

Obviously. But where's the fun in that?

 

[Richard Tobin]

The mistake is that you inadvertently
pass an incorrect value to malloc(). Sometimes such incorrect values
will be negative,

[/QUOTE]

Thats impossible, if the argument is unsigned.

Why don't you take a leaf out of P J Plauger's book and address the
real question, instead of being a tosser and pretending not to
understand?

The value the user supplies is negative. It's converted to a positive
value as part of the function calling process.

Huh? So you sacrifice half the possible address space, to cater for a
stupid programming error. Actually, this sounds about par for the
course, given the recent string thread.

Most current operating systems on general-purpose computers do not
allow the user to use the full theoretical address space. It's
common for 32-bit systems to only allow a single process to address
2^31 bytes.

It works fine on all my general purpose computers. That is
to say, it fails if the attempt is for too much memory, and it works
otherwise.

And the result of converting, say, -1 to size_t is typically "too much
memory". So malloc(-1) will fail.

-- Richard

 

[Mike Wahler]

Mark McIntyre a écrit :
That was of course just an advise.

I said in my original post

As always when I post something, the same group
of people started to try to find possible errors,

When I post something here, I sincerely hope that
others will attempt to find any errors it might
contain. Because if they do, and point them out,
it's to my (and probably others') benefit.

a harmless passtime they seem to enjoy.

Not only harmless, but beneficial.

You belong to that group.

The more members the better.

-Mike

 

[Richard Heathfield]

Richard Tobin said:

On the contrary, using an unsigned type as malloc's argument *eliminates*
the possibility of requesting a negative size.

You seem to be deliberately misunderstanding - surely reading the rest
of the thread makes it clear.[/QUOTE]

I'm failing to understand how such a mistake is considered so likely and so
undetectable that we suddenly have to change everybody's standard library.

The mistake is that you inadvertently
pass an incorrect value to malloc().

Sometimes such incorrect values will be negative,

Um, *what*?

p = malloc(n * sizeof *p);

is a very common idiom and about the most-typed code fragment in this group.
It is a well-known, popular idiom.

n is an object count, so it makes sense for it to be a size_t.
sizeof *p is an object size, and is a size_t.

So we have size_t * size_t - how, precisely, will that produce a negative
number when passed to a function taking size_t as an argument?

<snip>

 

[jacob navia]

Richard Heathfield a écrit :

Richard Tobin said:




I'm failing to understand how such a mistake is considered so likely and so
undetectable that we suddenly have to change everybody's standard library.




Um, *what*?

p = malloc(n * sizeof *p);

is a very common idiom and about the most-typed code fragment in this group.
It is a well-known, popular idiom.

n is an object count, so it makes sense for it to be a size_t.
sizeof *p is an object size, and is a size_t.

So we have size_t * size_t - how, precisely, will that produce a negative
number when passed to a function taking size_t as an argument?

<snip>

How to ignore errors. An example of bad programming practices
-------------------------------------------------------------

Mr heathfield says that:

> p = malloc(n * sizeof *p);
>
> is a very common idiom and about the most-typed code fragment in this group.
> It is a well-known, popular idiom.

Yes, but it is quite dangerous.

Suppose:
struct S {
unsigned n;
double d;
double data[8192];
};

I want to allocate 65520 objects of size 65 552 bytes each.

If you do that in a 32 bit system what you obtain?
i=65520, i*sizeof(*p)=4,294,967,040 as signed number = -256

WE OBTAIN A NEGATIVE NUMBER.

Even WORST.

What happens if you want to allocate just ONE object more?
i=65521, i*sizeof(*p)=65,296 as signed number = 65,296

YOU HAVE AN OVERFLOW and you get a POSITIVE BUT WRONG number!!!

You end allocating space for ONE object and not for
65521!!!!!!

And there is NO WAY a malloc will tell you about any errors
since the request is perfectly normal.

I am aware of this bug because we HAVE DISCUSSED this here and
I realized that the calloc implementation of lcc-win32 furnished
by the windows system had also this BUG. I immediately rewrote
the calloc function that SHOULD ALWAYS TEST FOR OVERFLOW.

What is MACHO programming?
--------------------------

It is an attitude of wrong security and lack of critical distance
to oneself. This attitude is specially pervasive in bright and
experienced programmers. You get into the frame of mind:

"I know what I am doing, I do not need barriers or security
considerations".

This is similar to:

"I am driving since 15 years now, never had an accident.
Why should I use the seat belts???"

 

[Richard Heathfield]

jacob navia said:

How to ignore errors. An example of bad programming practices
-------------------------------------------------------------

Mr heathfield says that:

p = malloc(n * sizeof *p);

is a very common idiom and about the most-typed code fragment in this group.
It is a well-known, popular idiom.

Yes, but it is quite dangerous.

Suppose:
struct S {
unsigned n;
double d;
double data[8192];
};

I want to allocate 65520 objects of size 65 552 bytes each.

If you do that in a 32 bit system what you obtain?

If by "32 bit system" you mean that size_t is a 32-bit type, then its
maximum value is 65535. If you don't mean that, what do you mean?

i=65520, i*sizeof(*p)=4,294,967,040 as signed number = -256

WE OBTAIN A NEGATIVE NUMBER.

No, you don't. size_t is an unsigned type.

Even WORST.

What happens if you want to allocate just ONE object more?
i=65521, i*sizeof(*p)=65,296 as signed number = 65,296

YOU HAVE AN OVERFLOW and you get a POSITIVE BUT WRONG number!!!

No, you don't get overflow with unsigned types.

I realized that the calloc implementation of lcc-win32 furnished
by the windows system had also this BUG.

It doesn't surprise me that you found a bug in lcc-win32. That is presumably
your job. But I am surprised that you found *this* bug in lcc-win32,
because it's so easy to get this right. If you have a quantity that cannot
possibly be negative (a count, a size, whatever), use an unsigned type.

I immediately rewrote the calloc function that SHOULD ALWAYS TEST FOR
OVERFLOW.

Unsigned types don't overflow.

<nonsense snipped>

 

[Cesar Rabak]

Kenny McCormack escreveu:

Incorrect. In the C language, if you pass a signed integer (for
example, the number -4) to a function where the formal parameter of that
function is declared unsigned (for example, malloc()), the "default
arithmetic conversions" (something you should read up on, by the way),
will convert that signed number to an unsigned, in order to be
compatible with the formal function declaration. On many systems,
numbers like -4 (negative numbers with small absolute values) will be
converted to very large unsigned numbers.

That's what we are talking about here. Do try to keep up.


Google for "malloc overcommit Linux OOM" and get back to us, OK?

Man, if you have a comprehensive thread on this here in c.l.c I would
like to find it, because I spend a lot of time explaining this to
newcomers in my work...

 

[Richard Tobin]

The mistake is that you inadvertently
pass an incorrect value to malloc().
Sometimes such incorrect values will be negative,

[/QUOTE]

Um, *what*?

p = malloc(n * sizeof *p);

is a very common idiom and about the most-typed code fragment in this group.
It is a well-known, popular idiom.

n is an object count, so it makes sense for it to be a size_t.
sizeof *p is an object size, and is a size_t.

So we have size_t * size_t - how, precisely, will that produce a negative
number when passed to a function taking size_t as an argument?

Well obviously it won't. But the idea was to protect against
programmers who make mistakes, and as we all know not all programmers
use that idiom.

As I said, I don't think the proposal would do much good, but surely
you can see what the idea was?

-- Richard

 

[Richard Heathfield]

Richard Tobin said:

Well obviously it won't.

Quite so.

But the idea was to protect against
programmers who make mistakes, and as we all know not all programmers
use that idiom.

The fix, then, is obvious.

As I said, I don't think the proposal would do much good, but surely
you can see what the idea was?

It appeared to be an attempt to introduce the additional risk of overflow to
the existing risk of specifying an allocation amount other than the amount
actually required. No, I don't see why this would be of benefit.

 

[Clark S. Cox III]

jacob navia said:


If by "32 bit system" you mean that size_t is a 32-bit type, then its
maximum value is 65535. If you don't mean that, what do you mean?

Umm, a 32-bit size_t (with no padding bits, etc.) can hold up to about
4-billion.

 

[jacob navia]

Clark S. Cox III a écrit :

Umm, a 32-bit size_t (with no padding bits, etc.) can hold up to about
4-billion.

Yes, but that is only logic.

This doesn't count as argument as it seems.

heathfield will answer that unsigned ints
wrap around as specified in the standard, so
that is not an overflow (or a similar nonsense.)

 

[CBFalconer]

.... snip ...

Suppose:
struct S {
unsigned n;
double d;
double data[8192];
};

I want to allocate 65520 objects of size 65 552 bytes each.

If you do that in a 32 bit system what you obtain?
i=65520, i*sizeof(*p)=4,294,967,040 as signed number = -256

WE OBTAIN A NEGATIVE NUMBER.

Very simple. We detect the 'unsigned overflow' by:

if (((n = a * b) < a || (n < b)) overflow()
else alliswell(n);

 

[Clark S. Cox III]

Clark S. Cox III a écrit :

Yes, but that is only logic.

This doesn't count as argument as it seems.

heathfield will answer that unsigned ints
wrap around as specified in the standard, so
that is not an overflow (or a similar nonsense.)

Oh, don't take my post the wrong way; just because I corrected a single
factual error in his post, I still agree with him; and disagree with you.
There is no way to obtain a negative size_t.
Unsigned types don't overflow.
A signed argument to malloc makes absolutely no sense.

 

[Richard Heathfield]

Clark S. Cox III said:

Umm, a 32-bit size_t (with no padding bits, etc.) can hold up to about
4-billion.

Oh, so it can, provided you use those silly little American billions.

In which case, what's the big deal? 65520 * 65552 is 4294967040 which is a
little smaller than 4294967295. Probably the allocation will fail (because
it's just so colossal) and NULL will be returned, but if the RAM is there,
it may even succeed. I don't see a problem here.

 

[Richard Heathfield]

jacob navia said:

Clark S. Cox III a écrit :

Yes, but that is only logic.

Well, I wouldn't have called it that. I'd have called it a mistake on my
part. They happen.

This doesn't count as argument as it seems.

heathfield will answer that unsigned ints
wrap around as specified in the standard, so
that is not an overflow

Arithmetic on unsigned integer types is reduced modulo 2^(MAX+1) where MAX
is the maximum value that can be stored in an object of that type.
Therefore, overflow doesn't happen.

(or a similar nonsense.)

If you want to think of the truth as being nonsense, that's up to you, but
it doesn't stop it from being the truth.

 

[jacob navia]

Clark S. Cox III a écrit :

Oh, don't take my post the wrong way; just because I corrected a single
factual error in his post, I still agree with him; and disagree with you.

His majesty is always right, no matter how much nonsense
he says.

There is no way to obtain a negative size_t.

Of course not. It is an unsigned number. The whole point is that
when you have

void fn(unsigned arg);

and you write:

fn(-3);

an implicit cast from signed to unsigned is done,
what in all implementations I know leads to no
machine code, but just a change in the way the bits
of the argument are interpreted.

Obviously this is a simplification of a real situation
when the values are not explicitely gibve like in the
example above.

Unsigned types don't overflow.

This is nonsense. Why I obtain

65521 * 65552 --> 65296 ????

You (and heathfield) are just playing with words. That
overflow is defined for unsigned numbers within C doesn't
mean that the result is mathematically VALID, or that is
the expected result.

The whole point of my argumentation is that the "idiom"

result = malloc(n * sizeof *p);

is a dangerous construct. Use

result = calloc(n, sizeof(*p))

and ensure your implementation of calloc doesn't have the bug!

A signed argument to malloc makes absolutely no sense.

It would catch the overflow in some cases above, as I explained in
my post.

You ropinion may be different, but it would be helpful if you
tried to advance an argument, just saying
"makes no sense"

makes no sense to anyone but you.

 

[Richard Heathfield]

jacob navia said:

Clark S. Cox III a écrit :


Of course not. It is an unsigned number. The whole point is that
when you have

void fn(unsigned arg);

and you write:

fn(-3);

an implicit cast from signed to unsigned is done,

There is no such thing as an implicit cast. Casts are explicit conversions.

This is nonsense.

No, it isn't.

Why I obtain

65521 * 65552 --> 65296 ????

Unsigned integer arithmetic is done by reducing the unsigned number to the
range 0 to MAX, where MAX is the maximum value for the type.

You (and heathfield) are just playing with words. That
overflow is defined for unsigned numbers within C doesn't
mean that the result is mathematically VALID, or that is
the expected result.

Overflow is not defined for unsigned numbers within C, and that's because
overflow is not *possible* for unsigned numbers within C.

The Standard makes this very clear: "A computation involving unsigned
operands can never overflow, because a result that cannot be represented by
the resulting unsigned integer type is reduced modulo the number that is
one greater than the largest value that can be represented by the resulting
unsigned integer type."

And the result is indeed mathematically valid, since unsigned integer types
with N value bits represent a ring with 2^N elements.

The whole point of my argumentation is that the "idiom"

result = malloc(n * sizeof *p);

is a dangerous construct.

You have not demonstrated this to be the case.

Use

result = calloc(n, sizeof(*p))

and ensure your implementation of calloc doesn't have the bug!

Better still, use malloc. If your implementation is buggy, get it fixed or
get it changed.

It would catch the overflow in some cases above, as I explained in
my post.

You can't overflow an unsigned type, as has been explained ad nauseam.

You ropinion may be different, but it would be helpful if you
tried to advance an argument, just saying
"makes no sense" makes no sense to anyone but you.

This isn't a matter of opinion. See ISO/IEC 9899.

 

[P.J. Plauger]

Richard Tobin said:


I'm failing to understand how such a mistake is considered so likely and
so
undetectable that we suddenly have to change everybody's standard library.


Um, *what*?

p = malloc(n * sizeof *p);

is a very common idiom and about the most-typed code fragment in this
group.
It is a well-known, popular idiom.

n is an object count, so it makes sense for it to be a size_t.
sizeof *p is an object size, and is a size_t.

So we have size_t * size_t - how, precisely, will that produce a negative
number when passed to a function taking size_t as an argument?

It won't, of course, but it can still be erroneous if the arithmetic
wraps around. This is because malloc is using an unsigned type to
pass a *non-negative* argument value. Wraparound may be valid for
the host type but not for the intented purpose of the conveyed
argument value. An error is an error, even if the compiled code
isn't obliged to catch it.

A robust program would have code something like:

if ((size_t)(-1) / sizeof (*p) < n)
<non-negative overflow will occur>

but it's rare to see such code in real life. Navia is correct
that calloc(n, sizeof *p) robustly passes the problem off to
calloc, which may itself be robust enough to know nonsense
when it sees it. Or not. I've seen C libraries go both ways.

Note that replacing size_t with its equivalent signed type offers
little protection for oversize allocations:

-- A valid allocation of over half of full memory yields a false
positive, since the signed version of the byte count is negative.

-- An invalid allocation of more than full memory has a 50/50
chance of yielding a negative value, or a bogus positive value.

The best of all worlds is probably a calloc that checks as it
should for wraparound, and further checks a non-wrapped byte
count against RSIZE_MAX, a la TR 24731. And this doesn't really
require any change to existing calloc calls -- just better
runtime checking.

But none of this blather about the virtues of signed arithmetic,
or the imperviousness to overflow of unsigned arithmetic,
addresses the true problem of allocating storage sanely and
reporting insane requests properly.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[jacob navia]

Richard Heathfield a écrit :

Unsigned integer arithmetic is done by reducing the unsigned number to the
range 0 to MAX, where MAX is the maximum value for the type.

This "arithmetic" has nothing to do with arithmetic
and overflow is just declared normal, leading
to unexpected results.

But it is useless to discuss with you

HINT:

I did not answer to you but to somebody else

Overflow doesn't exist?

65521 *65552 --> 65296 ???

OK.

Then if I want to allocate 65521 objects of
65552 bytes each I obtain a block 65296 bytes long

WONDERFUL!

Then, I can write more than 4GB into 65296 bytes

You should work for RAM producers heathfield.
The would be very inetersted in your
method for writing 4GB into 65296 bytes!!!

You have not demonstrated this to be the case.

I have demonstrated that the multiplication
65521 * 65552 gives 65296, what is clearly not
enough space to store 65521 objects of size 65552
each. You are free to call that "multiplication that
doesn't overflow".

You can babble as much as you like but that is a fact.