Request

[jacob navia]

As a consequence of a heavy discussion in another thread,
I wrote this program:
#include <stdlib.h>
int main(void)
{
char *p=calloc(65521,65552);
if (p)
printf("BUG!!!!\n");
}

The multiplication will give an erroneus result
in 32 bit size_t systems. I have tested this
in several systems with several compilers.

Maybe people with other compilers/implementations
would compile this program and send the results?

The results I have so far are:

LINUX
GCC 2.95
HAS THE BUG

GCC-4.02
NO BUGS

GCC-3.x
NO BUGS

WINDOWS
cygwin gcc 3.4.4
NO bugs
pelles c:
HAS THE BUG
Microsoft
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.42
for 80x86
NO BUGS (If the multiplication overflows it will trap. Very good
behavior)

AIX
IBM-cc (compiling in 32 bit mode)
NO BUGS
gcc 4.0.0 (compiling in 32 bit mode)
NO BUGS


Thanks in advance for your cooperation

 

[Ben Pfaff]

LINUX
GCC 2.95
HAS THE BUG

GCC is not a complete C implementation. In particular, it lacks
a C library. Thus, it doesn't make sense to say that GCC 2.95
has a bug in its calloc function. In fact, when I run your test
program here, using GCC 2.95.4 on top of a Linux kernel, I don't
see the bug you claim to be present. This probably means that we
are using different C libraries.

 

[Walter Roberson]

As a consequence of a heavy discussion in another thread,
I wrote this program:
#include <stdlib.h>
int main(void)
{
char *p=calloc(65521,65552);
if (p)
printf("BUG!!!!\n");
}

You appear to be missing a prototype for printf(), and you
appear to be relying on C99 behaviour upon running off the
end of main().

The multiplication will give an erroneus result
in 32 bit size_t systems.

That may or may not be the case, but that isn't what you
are testing.

Your program would claim as a BUG any system
on which size_t was able to accomedate 65521 * 65552 and was
able to allocate that much memory.

The ability to allocate that much memory may depend upon the memory and
swap space installed in the system, and may depend upon any resource
usage limitations that have been put in place. Some of those limits
may be "soft" limits that can be removed with a simple command or two;
some of the limits might be "hard" limits imposed by the systems
administrator on systems that would have otherwise have sufficient
resources.

I have tested this
in several systems with several compilers.

Maybe people with other compilers/implementations
would compile this program and send the results?

SGI IRIX MipsPro 7.3.1.2:
With -o32 or -n32, size_t is 32 and
the implementation does not allocate the memory. With -64,
size_t is 64 and the particular system I'm running on
has enough resources that the system is able to allocate the
full extent of the memory.

According to your program, that means that the above compiler both
has and does not have the bug (it's the same compiler in all
the cases.)

If I were me, I would conclude that your methodology is flawed.

 

[jacob navia]

Walter Roberson a écrit :

You appear to be missing a prototype for printf(), and you
appear to be relying on C99 behaviour upon running off the
end of main().

Yes, add them at will.

That may or may not be the case, but that isn't what you
are testing.

Your program would claim as a BUG any system
on which size_t was able to accomedate 65521 * 65552 and was
able to allocate that much memory.

With a 32 bit size_t as I said, it would be interesting that
the system allocates a size_t of 33 bits...

I may be missing something... BUT!!!

The ability to allocate that much memory may depend upon the memory and
swap space installed in the system, and may depend upon any resource
usage limitations that have been put in place. Some of those limits
may be "soft" limits that can be removed with a simple command or two;
some of the limits might be "hard" limits imposed by the systems
administrator on systems that would have otherwise have sufficient
resources.

No, this is impossible in 32 bit size_t systems!!!

SGI IRIX MipsPro 7.3.1.2:
With -o32 or -n32, size_t is 32 and
the implementation does not allocate the memory.

Fine We can add that SGI IRIX has NOT the bug.

Thanks

With -64,
size_t is 64 and the particular system I'm running on
has enough resources that the system is able to allocate the
full extent of the memory.

Excuse me but you haven't a 32 bit size_t but a 64 bit
size_t when compiling in 64 bit mode. If you care to see my table
I compiled explicitely in 32 bit mode in AIX systems!!!

According to your program, that means that the above compiler both
has and does not have the bug (it's the same compiler in all
the cases.)

It is not the same size_t!!!

I said:
32 bit size_t!!!

If I were me, I would conclude that your methodology is flawed.

I am glad that you are you and I am me



What would happen If i were you? I would try to read
what I am writi,ng and not trying to find inexistent
bugs

Thanks for your input in any case Walter.

 

[Walter Roberson]

Walter Roberson a écrit :

With a 32 bit size_t as I said, it would be interesting that
the system allocates a size_t of 33 bits...

I may be missing something... BUT!!!

You *are* missing something. What you wrote was that,

"The multiplication will give an erroneus result in 32 bit size_t systems."

You did NOT, however, restrict the testing to systems with size_t
of 32 bits: you merely made an assertion about what would happen
on systems on which size_t is 32 bits. The proof is in the code,
and your code declares as a bug any system and option and
environment combination which is able to allocate the required amount
of memory.

If you care to see my table
I compiled explicitely in 32 bit mode in AIX systems!!!

Irrelevant, since you did not restrict the scope of the test
to 32 bit mode.

 

[user923005]

No, this is impossible in 32 bit size_t systems!!!

Why? The calloc() call has two parameters. There is no reason that
the total allocation cannot exceed the value of a size_t.

BTW, there are definitely 32 bit systems that will allocate memory in
excess of 32 bits. For example:
http://support.microsoft.com/kb/274750

From this, I do not see any restriction that the allocation must fail

if nmemb*size > SIZE_MAX:
"7.20.3.1 The calloc function
Synopsis
1 #include <stdlib.h>
void *calloc(size_t nmemb, size_t size);
Description
2 The calloc function allocates space for an array of nmemb objects,
each of whose size
is size. The space is initialized to all bits zero.252)
Returns
3 The calloc function returns either a null pointer or a pointer to the
allocated space."

 

[Spiros Bousbouras]

As a consequence of a heavy discussion in another thread,
I wrote this program:
#include <stdlib.h>
int main(void)
{
char *p=calloc(65521,65552);
if (p)
printf("BUG!!!!\n");
}

What bug are you claiming exists ? If you're
saying that only 65296 bytes have been allocated
how do you know ?

The multiplication will give an erroneus result
in 32 bit size_t systems. I have tested this
in several systems with several compilers.

How do you know what the calloc implementation does
upon receiving the arguments 65521 and 65552 ? It is
reasonable to assume that it multiplies them but how
do you know that the result of the multiplication is limited
to 32 bits ? I don't see what relevance the width of size_t
has.

Maybe people with other compilers/implementations
would compile this program and send the results?

I can't be bothered to be honest because I don't see the
point. As far as I can see , even if calloc returns a non NULL
pointer you can't know how much memory has been allocated
unless you have seen the code of the calloc implementation.
Note that even if you know for sure that the system cannot
allocate 65521*65552 bytes and calloc returns non NULL you
still cannot be sure that it is calloc which has the bug. It may
be that calloc uses some platform specific system call to get
the memory and it is the system call which has the bug.

 

[jacob navia]

Walter Roberson a écrit :

You *are* missing something. What you wrote was that,

"The multiplication will give an erroneus result in 32 bit size_t systems."

You did NOT, however, restrict the testing to systems with size_t
of 32 bits

I thought that that restriction would be evident from the restriction
of size_t to 32 bits.

OK This is a misunderstanding then. The test is intended only
for 32 bit systems

Updated version:

#include <stdlib.h>
#include <limits.h>
#include <stdio.h> // for printf
int main(void)
{
int s = sizeof(size_t)*CHAR_BIT;
if (s == 32 && NULL != calloc(65521,65552))
printf("BUG!!!!\n");
return 0; // For C89 compilers
}

 

[Jens Thoms Toerring]

As a consequence of a heavy discussion in another thread,
I wrote this program:
#include <stdlib.h>
int main(void)
{
char *p=calloc(65521,65552);
if (p)
printf("BUG!!!!\n");
}

The multiplication will give an erroneus result
in 32 bit size_t systems.

I think that assuming that a 32 bit multiplication is the sole
reason is a bit premature. At least I don't see convincing evi-
dence of that.

I have tested this
in several systems with several compilers.

Maybe people with other compilers/implementations
would compile this program and send the results?

The results I have so far are:

LINUX
GCC 2.95
HAS THE BUG

GCC-4.02
NO BUGS

GCC-3.x
NO BUGS

I have 3.3.1 and get the opposite result (with no overcommitment
enabled). Things get even more interesting if I try to print out
the pointers value (of course after including <stdio.h>;-): While
the test of p tells that it's not zero ("BUG!!!!" isn't printed)
a line like

printf( "%p\n", ( void * ) p );

gives me "(nil)", the same result as if I had a NULL pointer. And,
if, on top of it, I try to assign a value to the memory (even the
very first byte of it) then a segmentation fault is the result.

I guess that indicates that a bit more than a simple unsigned 32
bit multiplication that overflows and isn't detected in time is
going on behind the scenes...
Regards, Jens

 

[Spiros Bousbouras]

I think that assuming that a 32 bit multiplication is the sole
reason is a bit premature. At least I don't see convincing evi-
dence of that.







I have 3.3.1 and get the opposite result (with no overcommitment
enabled). Things get even more interesting if I try to print out
the pointers value (of course after including <stdio.h>;-): While
the test of p tells that it's not zero ("BUG!!!!" isn't printed)

You got it backwards , if "BUG" isn't printed then it is
a NULL pointer.

a line like

printf( "%p\n", ( void * ) p );

gives me "(nil)", the same result as if I had a NULL pointer. And,
if, on top of it, I try to assign a value to the memory (even the
very first byte of it) then a segmentation fault is the result.

Not surprising since you try dereferencing a NULL pointer.

 

[jacob navia]

user923005 a écrit :

Why? The calloc() call has two parameters. There is no reason that
the total allocation cannot exceed the value of a size_t.

BTW, there are definitely 32 bit systems that will allocate memory in
excess of 32 bits. For example:
http://support.microsoft.com/kb/274750

This is irrelevant. Those links refer as to how
to use certain flags in the boot.ini to allow
*3GB* if user space. (!!!!!!!)

And 3GB is well below a 32 bit size_t.

With a 32 bit size_t and pointer size how can you
address more than 4GB?

But this is a well known tactic.

Since I present the code, I do the effort of making all those
measurements, it is so easy to just sit by and say

"A system with 32 bit size_t could allocate more than 4GB
memory. A system like that COULD exist."

So WHAT?

Can you please tell me where that system is????

If not, just do not participate. Thanks for your
correction. A system where this test gives a false
positive COULD exist. It is conceivable.

 

[Spiros Bousbouras]

Walter Roberson a écrit :

I thought that that restriction would be evident from the restriction
of size_t to 32 bits.

OK This is a misunderstanding then. The test is intended only
for 32 bit systems

What do you mean by 32 bit system ? One that cannot
address more than 32 bits of physical memory ? What
if the system is connected to a hard disk which has more
than 2**32 bytes of space. Surely then it would be possible
to somehow address the whole of it even if it meant using
special libraries to do arithmetic on more than 32 bits.

 

[jacob navia]

Spiros Bousbouras a écrit :

What do you mean by 32 bit system ? One that cannot
address more than 32 bits of physical memory ?

where sizeof(void *) == sizeof(size_t) == 4 and CHAR_BIT is 8

What
if the system is connected to a hard disk which has more
than 2**32 bytes of space. Surely then it would be possible
to somehow address the whole of it even if it meant using
special libraries to do arithmetic on more than 32 bits.

File pointers use 64 bit integers this days when it is
impossible to buy a hard disk of 32 GB only

This makes the system not 64 bits!

 

[jacob navia]

Jens Thoms Toerring a écrit :
[snip]

Please see the answer of Mr Spiros Bousburas in this same thread
and try to understand what I want before jumping to conclusions.

Thanks

P.S. if you do not want to participate it is OK but please do not
send noise.

 

[Jens Thoms Toerring]

You got it backwards , if "BUG" isn't printed then it is
a NULL pointer.

Grrr;-) Right you are - I played around with it too much (with
disruptions by my cat), loosing track of what I had been trying
out. Time to go to bed...
Regards, Jens

 

[jacob navia]

Spiros Bousbouras a écrit :

What bug are you claiming exists ? If you're
saying that only 65296 bytes have been allocated
how do you know ?

if size_t is 32 bits, it would be highly surprising that
an allocator returns an object of size 33 bits at least.

You are just saying here that such a system *could* exist.
This doesn't make it any more real.

OK. In those hypothetical systems my test would fail.

How do you know what the calloc implementation does
upon receiving the arguments 65521 and 65552 ? It is
reasonable to assume that it multiplies them but how
do you know that the result of the multiplication is limited
to 32 bits ? I don't see what relevance the width of size_t
has.

because if an object of size > 4GB could exist, sizeof
(that returns a size_t) would not work.

I can't be bothered to be honest because I don't see the
point.

You can't be bothered to be honest?

I hope you mean

To be honest, I can't be bothered...



BUG!!!!

> As far as I can see , even if calloc returns a non NULL
pointer you can't know how much memory has been allocated
unless you have seen the code of the calloc implementation.
Note that even if you know for sure that the system cannot
allocate 65521*65552 bytes and calloc returns non NULL you
still cannot be sure that it is calloc which has the bug. It may
be that calloc uses some platform specific system call to get
the memory and it is the system call which has the bug.

Excuse me but what calloc does is not measured. I just
test the result. If it is the system's fault or whatever
this is not relevant.

 

[Spiros Bousbouras]

user923005 a écrit :

This is irrelevant. Those links refer as to how
to use certain flags in the boot.ini to allow
*3GB* if user space. (!!!!!!!)

And 3GB is well below a 32 bit size_t.

With a 32 bit size_t and pointer size how can you
address more than 4GB?

But this is a well known tactic.

A tactic to achieve what ?

Since I present the code, I do the effort of making all those
measurements, it is so easy to just sit by and say

"A system with 32 bit size_t could allocate more than 4GB
memory. A system like that COULD exist."

So WHAT?

I don't know "so WHAT". You see the problem here is
that you have not explained what your point is. In
addition to making the effort (actually not that much
effort it seems to me) of making the measurements, it
would have been very nice if you had explained what point
the measurements are meant to illustrate/investigate.

I'll take a guess: your point is that calloc should check
whether the multiplication (if that's what it does) yields
a value so big that the system cannot possibly have that
much memory. Perhaps it should , or perhaps the programmer
should , it seems like a matter of taste to me and I won't
enter into an endless argument about it. But the main problem
as I pointed out in a previous post is that your experiments
even done on a 32 bit system (whatever that means) don't
tell us for sure what checks the calloc implementations on
those systems do. As I said you would need to check the code
of calloc to verify that.

Can you please tell me where that system is????

If not, just do not participate. Thanks for your
correction. A system where this test gives a false
positive COULD exist. It is conceivable.

You cannot set arbitrary rules about when people
cannot participate in the threads you start. If your
point in the above paragraph (see ? I have to *guess*
again what your point is because you don't spell it out)
is that people should only participate if they're going
to address your main point in the thread, then I can kind
of see this as a reasonable requirement but you need to
explain first what your point is and you haven't done that.

 

[user923005]

user923005 a écrit :

This is irrelevant. Those links refer as to how
to use certain flags in the boot.ini to allow
*3GB* if user space. (!!!!!!!)

That is either 8 GB user space or 32 GB, not 3 GB. I suggest you read
again -- this time with comprehension enabled... Here is a quote from
that page that you may find helpful:
"SQL Server 2000 Enterprise Edition introduces support for the use of
Microsoft Windows 2000 Address Windowing Extensions (AWE) to address
approximately 8 GB of memory for instances that run on Microsoft
Windows 2000 Advanced Server, and approximately 32 GB for instances
that run on Microsoft Windows 2000 Datacenter."

P.S.
These operating systems are 32 bits.

And 3GB is well below a 32 bit size_t.

You seem to have a bit of a comprehension problem.

With a 32 bit size_t and pointer size how can you
address more than 4GB?

There are lots of ways to do it.

But this is a well known tactic.

Since I present the code, I do the effort of making all those
measurements, it is so easy to just sit by and say

"A system with 32 bit size_t could allocate more than 4GB
memory. A system like that COULD exist."

So WHAT?

So that is all that is needed to show that your test is flawed.

Can you please tell me where that system is????

I mentioned one above that uses 32 bit addressing lines and yet can
address 32 GB or RAM. The way it does it is with memory models at the
low level. Just like back in the bad old days when MS-DOS programmers
had to model with large, medium or small memory models because of 16
bit addressing, in a similar way you can use 32 bit segments and
offsets to address (potentially) 64 GB of RAM or any other scheme you
like.

If not, just do not participate. Thanks for your
correction. A system where this test gives a false
positive COULD exist. It is conceivable.

Your test has no merit that I can imagine.
Systems that will allocate 2^64 pages of ram could return a failure
result if the user space is less than 4GB. So what have we learned
from your code? Nothing.

 

[jacob navia]

Spiros Bousbouras a écrit :

A tactic to achieve what ?




I don't know "so WHAT". You see the problem here is
that you have not explained what your point is. In
addition to making the effort (actually not that much
effort it seems to me) of making the measurements, it
would have been very nice if you had explained what point
the measurements are meant to illustrate/investigate.

I'll take a guess: your point is that calloc should check
whether the multiplication (if that's what it does) yields
a value so big that the system cannot possibly have that
much memory. Perhaps it should , or perhaps the programmer
should , it seems like a matter of taste to me and I won't
enter into an endless argument about it. But the main problem
as I pointed out in a previous post is that your experiments
even done on a 32 bit system (whatever that means) don't
tell us for sure what checks the calloc implementations on
those systems do. As I said you would need to check the code
of calloc to verify that.




You cannot set arbitrary rules about when people
cannot participate in the threads you start. If your
point in the above paragraph (see ? I have to *guess*
again what your point is because you don't spell it out)
is that people should only participate if they're going
to address your main point in the thread, then I can kind
of see this as a reasonable requirement but you need to
explain first what your point is and you haven't done that.

My objective is to

1) make a list of all the buggy systems that
is here in clc and everybody can see.

2) Make people aware of this bug in their implementations.
(if any)

 

[jacob navia]

P.S.
I forgot to spell out clearly the objective of this tests.

My objective is to

1) make a list of all the buggy systems that
is here in clc and everybody can see.

2) Make people aware of this bug in their implementations.
(if any)