bad_alloc after 4GB on 64bit

[cman]

Hi guys, why does this fail raising bad_alloc
int *v = new int [6000000000];

if this succeeds
int *v = (int *) malloc((unsigned)6000000000)

both on the same machine, same compiler g++, 64bit linux red hat
enterprise 4, no ulimits on the user, enough virtual memory ecc...

in both cases compiled with
g++ source.cpp -o executable
the file is always a c++ .cpp file, that line above is the only one that
differs.

Thanks

 

[Stuart Redmann]

Hi guys, why does this fail raising bad_alloc
int *v = new int [6000000000];

AFAIK, any integer literal will be interpreted as 32 bit signed integer, even
though the compiler is a 64bit compiler. I think that this is meant to be so for
backward compatibility reasons to 32bit compilers. If you want an int literal to
be a 64bit int literal, you'll have to add LL at the back: 6000000000LL should
do the trick.

if this succeeds
int *v = (int *) malloc((unsigned)6000000000)

both on the same machine, same compiler g++, 64bit linux red hat
enterprise 4, no ulimits on the user, enough virtual memory ecc...

in both cases compiled with
g++ source.cpp -o executable
the file is always a c++ .cpp file, that line above is the only one that
differs.

Regards,
Stuart

 

[=?iso-8859-1?Q?Leo_Havm=F8ller?=]

int *v = new int [6000000000];

Request for 24000000000 bytes.

int *v = (int *) malloc((unsigned)6000000000)

Request for 6000000000 bytes.

Leo Havmøller.

 

[Ioannis Gyftos]

int *v = new int [6000000000];

Request for 24000000000 bytes.

So an int is 4 bytes?

 

[Marco Manfredini]

int *v = new int [6000000000];

Request for 24000000000 bytes.

So an int is 4 bytes?

OP said RHEL4 64 bit, usual compiler. where indeed sizeof(int)=4.

 

[cman]

int *v = new int [6000000000];

Request for 24000000000 bytes.

int *v = (int *) malloc((unsigned)6000000000)

Request for 6000000000 bytes.

Ok this was my mistake, however this is not the problem.
Here is a new test source code that shows the problem:

#include <iostream>
#include <cstdlib>
int main() {
//char *v = new char[48000000000]; //FAILS
char * v = (char*)malloc((unsigned)48000000000); //SUCCEEDES
std::cout << (void*)v << std::endl;
return 0;
}

the commented line fails with bad_alloc while the other one succeedes,
that is, nonzero pointer.

 

[Tadeusz B. Kopec]

int *v = new int [6000000000];

Request for 24000000000 bytes.

int *v = (int *) malloc((unsigned)6000000000)

Request for 6000000000 bytes.

Ok this was my mistake, however this is not the problem. Here is a new
test source code that shows the problem:

#include <iostream>
#include <cstdlib>
int main() {
//char *v = new char[48000000000]; //FAILS char *
v = (char*)malloc((unsigned)48000000000); //SUCCEEDES std::cout
<< (void*)v << std::endl;
return 0;
}

the commented line fails with bad_alloc while the other one succeedes,
that is, nonzero pointer.

I guess it's a feature of OS, not language or compiler. It's based on an
article by Herb Sutter (on of his 'Exceptional C++...' books, article
title like 'Why not bother with catching bad_alloc').
I mean that OS might wait with allocating memory until you try to access
it. operator new tries to initialise allocated memory while malloc
doesn't. If I am right, calloc should fail.

 

[karpov2007]

#include <cstdlib>
int main() {
//char *v = new char[48000000000]; //FAILS
char * v = (char*)malloc((unsigned)48000000000); //SUCCEEDES

I wish to specify. The size of int is equal 4 bytes?
If yes then:
char *v = new char[48000000000]; //FAILS
Allocate: 48 000 000 000 bytes.

char *v = (char*)malloc((unsigned)48000000000); //SUCCEEDES
Allocate: 755 359 744 bytes!!!

755359744 < 48000000000

Programming under 64-bit systems in general is fraught with errors.
Welcome:
http://www.viva64.com/articles/20_issues_of_porting_C++_code_on_the_64-bit_platform.html
http://www.viva64.com/articles/Forgotten_problems.html
http://www.viva64.com/articles/Viva64_-_what_is_and_for.html
http://www.viva64.com

 

[James Kanze]

#include <iostream>
#include <cstdlib>
int main() {
//char *v = new char[48000000000]; //FAILS
char * v = (char*)malloc((unsigned)48000000000); //SUCCEEDES

I wish to specify. The size of int is equal 4 bytes?

Probably. I think he mentioned Linux, which is generally
I32LP64 in 64 bit mode.

If yes then:
char *v = new char[48000000000]; //FAILS
Allocate: 48 000 000 000 bytes.

char *v = (char*)malloc((unsigned)48000000000); //SUCCEEDES
Allocate: 755 359 744 bytes!!!

755359744 < 48000000000

Programming under 64-bit systems in general is fraught with errors.

No more so than programming under anything else. Porting
non-portable code to a different system is fraught with errors.
By definition. The problem is that a lot of code is
non-portable (e.g. assumes 32 bit long) for no real reason.

In this case, one has to wonder why the cast to unsigned, which
serves no possible cause. Malloc takes a size_t (which is 8
bytes in an I32LP64 system), just like operator new.

 

[Ron Natalie]

Hi guys, why does this fail raising bad_alloc
int *v = new int [6000000000];

AFAIK, any integer literal will be interpreted as 32 bit signed integer,

Well not exactly. An integer literal (without a suffix) will be
represented as either int or long int. Unfortunately, long int
on this implementation is still 32 bits.

 

[Ron Natalie]

int *v = new int [6000000000];
Request for 24000000000 bytes.

So an int is 4 bytes?

OP said RHEL4 64 bit, usual compiler. where indeed sizeof(int)=4.

And more importanlty sizeof (long int) is still 4.

 

[karpov2007]

And more importanlty sizeof (long int) is still 4.

Are you sure? RHEL4 64 bit used LP64 data model.


Red Hat Enterprise Linux 3.0 ES x86_64 on an Intel Xeon EM64T:

int main(int argc, char** argv) {
printf("pointer: %d\n", sizeof(char*));
printf("int: %d\n", sizeof(int));
printf("long: %d\n", sizeof(long));
}

pointer: 8
int: 4
long: 8

 

[Alex Buell]

Red Hat Enterprise Linux 3.0 ES x86_64 on an Intel Xeon EM64T:

int main(int argc, char** argv) {
printf("pointer: %d\n", sizeof(char*));
printf("int: %d\n", sizeof(int));
printf("long: %d\n", sizeof(long));
}

pointer: 8
int: 4
long: 8

What happens if you use 'long long'? On my 32bit system it's 64 bits.

 

[karpov2007]

What happens if you use 'long long'? On my 32bit system it's 64 bits.

Must be 8 byte. But I have not understood, to what this question. I
have simply objected, that in RHEL4 64 sizeof (long int) = 4.

 

[James Kanze]

cman wrote:

Hi guys, why does this fail raising bad_alloc
int *v = new int [6000000000];

AFAIK, any integer literal will be interpreted as 32 bit signed integer,

Well not exactly. An integer literal (without a suffix) will
be represented as either int or long int. Unfortunately, long
int on this implementation is still 32 bits.

Which implementation? The OP mentionned a 64 bit Red Hat
Linux---on my system (using Mandriva, instead of Red Hat, but I
don't think that matters), long int is 64 bits.

 

[cman]

#include <iostream>
#include <cstdlib>
int main() {
//char *v = new char[48000000000]; //FAILS
char * v = (char*)malloc((unsigned)48000000000); //SUCCEEDES

I wish to specify. The size of int is equal 4 bytes?
If yes then:
char *v = new char[48000000000]; //FAILS
Allocate: 48 000 000 000 bytes.

char *v = (char*)malloc((unsigned)48000000000); //SUCCEEDES
Allocate: 755 359 744 bytes!!!

755359744 < 48000000000

I think you are right. The (unsigned) must have wrapped the value to 32
bits. I don't remember why we did put that unsigned, it was because of
some our previous tests. We thought it would turn the value to unsigned
long but actually it turned it to unsigned int.
We will fix and repeat that test when the 64bit machine is free, now
there is a BIG process taking all the memory, running there...

BTW: yes int is 32 bits in there
strange thing: shouldn't int be the native and fastest datatype on a
machine? Why isn't it 64 bits then?

Thanks

 

[Victor Bazarov]

[..talking about 64bit machine..]
BTW: yes int is 32 bits in there
strange thing: shouldn't int be the native and fastest datatype on a
machine? Why isn't it 64 bits then?

Who says that the 64-bit int is the fastest on a 64-bit machine? It
is quite possible that the 32-bin int is faster (and just as native).

V

 

[cman]

Who says that the 64-bit int is the fastest on a 64-bit machine? It
is quite possible that the 32-bin int is faster (and just as native).

The 16 bit type is certainly faster, then!

 

[Victor Bazarov]

The 16 bit type is certainly faster, then!

Nothing is certain when hardware is concerned.

V

 

[James Kanze]

(e-mail address removed) wrote:

[...]

BTW: yes int is 32 bits in there
strange thing: shouldn't int be the native and fastest datatype on a
machine? Why isn't it 64 bits then?

Histerical reasons, no doubt. I think you'll find that in
practice, 1, 2, 4 and 8 byte integers are pretty much the same
speed; depending on the application, the smaller ones may even
run faster, because of better locality. Up until 32 bits, the
size of an int automatically grew, without question, because the
bounds were too small otherwise to be really useful. Beyond 32
bits, that's not so clearly the case, and most 64 bit
implementations which derive from earlier 32 bit ones use 32 bit
ints.