Changing intobject to use int rather than long

[Clarence]

Does anyone think (or know) that it might cause any internal problems
if the ival member of the struct defining an intobject were to be
changed from its current "long int" to just "int"?

When you move your application to a 64-bit system in order to get a
bigger address space to store your millions/billions of integers in
RAM, but they get twice as big, you don't gain very much.

Thanks.

 

[Chris Mellon]

Does anyone think (or know) that it might cause any internal problems
if the ival member of the struct defining an intobject were to be
changed from its current "long int" to just "int"?

When you move your application to a 64-bit system in order to get a
bigger address space to store your millions/billions of integers in
RAM, but they get twice as big, you don't gain very much.

Your int objects get twice as large, but you get 4294967296 times more
address space.

(They don't always get twice as large, and you don't actually get that
much address space, and there's lots of other things wrong with this
answer, but the core truth - that your available address space grows
at a greater rate than the size of your ints - is correct).

 

[Clarence]

Your int objects get twice as large, but you get 4294967296 times more
address space.

(They don't always get twice as large, and you don't actually get that
much address space, and there's lots of other things wrong with this
answer, but the core truth - that your available address space grows
at a greater rate than the size of your ints - is correct).

Technically true, but in my real world, I deal with physical RAM in
the machine. I went from a 2GB machine that was using half its memory
to an 8GB machine that is using one quarter its memory. Yes, it's
better,
but I still want that factor of two!

 

[Hrvoje Niksic]

When you move your application to a 64-bit system in order to get a
bigger address space to store your millions/billions of integers in
RAM, but they get twice as big, you don't gain very much.

I don't think changing the underlying type will help at all. The
layout of the int object is as follows:

typedef struct {
// PyObject_HEAD
Py_ssize_t ob_refcnt;
struct _typeobject *ob_type;
// the actual value
long ob_ival;
} PyIntObject;

On a 64-bit machine, that's 16 bytes for PyObject_HEAD and 8 more
bytes for the value, 24 bytes total. Changing long to int won't
decrease the struct size to 20 because the compiler will pad it to 24,
the nearest multiple of 8. (Forcing the compiler to pack the struct
won't help because malloc will still pad it for you.)

If you need to store millions of integers compactly, maybe
array.array('i') can help.

 

[Clarence]

I don't think changing the underlying type will help at all. The

On a 64-bit machine, that's 16 bytes for PyObject_HEAD and 8 more
bytes for the value, 24 bytes total. Changing long to int won't
decrease the struct size to 20 because the compiler will pad it to 24,
the nearest multiple of 8. (Forcing the compiler to pack the struct
won't help because malloc will still pad it for you.)

That's an excellent point. And true, too. Thanks, that will lay the
issue
to rest.

 

[Terry Reedy]

| That's an excellent point. And true, too. Thanks, that will lay the
| issue to rest.

Good idea. I think people who moved to 64 bits to get 64 bits would be
upset if they did not ;-).

 

[Martin v. Löwis]

On a 64-bit machine, that's 16 bytes for PyObject_HEAD and 8 more

That's an excellent point. And true, too. Thanks, that will lay the
issue to rest.

As a side observation, notice how the increase in memory consumption
does not primarily originate from the increase in the size of a long.
Instead, all pointers double their sizes. In an OO language (like
Python), you have lots of pointers, so you will often find that the
application uses more memory when run in 64-bit mode.

If that is a concern to you, let me rephrase Terry's observation:
just try running a 32-bit Python interpreter on your 64-bit system,
and you may find that it actually runs better because it uses less
memory.

Regards,
Martin

 

[Christian Heimes]

Good idea. I think people who moved to 64 bits to get 64 bits would be
upset if they did not ;-).

Windows X64 users still get 32bit ints. The long datatype is 32bit even
on the 64bit version of Windows.

Christian

 

[Terry Reedy]

| Terry Reedy wrote:
| > Good idea. I think people who moved to 64 bits to get 64 bits would be
| > upset if they did not ;-).
|
| Windows X64 users still get 32bit ints. The long datatype is 32bit even
| on the 64bit version of Windows.

Yes, anyone expecting to get 64 bit ints from Windows would be upset. And
those who are getting 64 bit ints from real 64 bit OSes would be upset if
the OPs suggestion were implemented.

 

[Hrvoje Niksic]

| Terry Reedy wrote:
| > Good idea. I think people who moved to 64 bits to get 64 bits would be
| > upset if they did not ;-).
|
| Windows X64 users still get 32bit ints. The long datatype is 32bit even
| on the 64bit version of Windows.

Yes, anyone expecting to get 64 bit ints from Windows would be
upset.

Why? Using only 32 bits doesn't make the structure any smaller
because of pointer alignment.