Changing intobject to use int rather than long

Discussion in 'Python' started by Clarence, Dec 18, 2007.

  1. Clarence

    Clarence Guest

    Does anyone think (or know) that it might cause any internal problems
    if the ival member of the struct defining an intobject were to be
    changed from its current "long int" to just "int"?

    When you move your application to a 64-bit system in order to get a
    bigger address space to store your millions/billions of integers in
    RAM, but they get twice as big, you don't gain very much.

    Thanks.
     
    Clarence, Dec 18, 2007
    #1
    1. Advertising

  2. Clarence

    Chris Mellon Guest

    On Dec 18, 2007 11:59 AM, Clarence <> wrote:
    > Does anyone think (or know) that it might cause any internal problems
    > if the ival member of the struct defining an intobject were to be
    > changed from its current "long int" to just "int"?
    >
    > When you move your application to a 64-bit system in order to get a
    > bigger address space to store your millions/billions of integers in
    > RAM, but they get twice as big, you don't gain very much.
    >


    Your int objects get twice as large, but you get 4294967296 times more
    address space.

    (They don't always get twice as large, and you don't actually get that
    much address space, and there's lots of other things wrong with this
    answer, but the core truth - that your available address space grows
    at a greater rate than the size of your ints - is correct).

    > Thanks.
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Chris Mellon, Dec 18, 2007
    #2
    1. Advertising

  3. Clarence

    Clarence Guest

    On Dec 18, 6:24 pm, "Chris Mellon" <> wrote:
    >
    > Your int objects get twice as large, but you get 4294967296 times more
    > address space.
    >
    > (They don't always get twice as large, and you don't actually get that
    > much address space, and there's lots of other things wrong with this
    > answer, but the core truth - that your available address space grows
    > at a greater rate than the size of your ints - is correct).


    Technically true, but in my real world, I deal with physical RAM in
    the machine. I went from a 2GB machine that was using half its memory
    to an 8GB machine that is using one quarter its memory. Yes, it's
    better,
    but I still want that factor of two!
     
    Clarence, Dec 18, 2007
    #3
  4. Clarence <> writes:

    > When you move your application to a 64-bit system in order to get a
    > bigger address space to store your millions/billions of integers in
    > RAM, but they get twice as big, you don't gain very much.


    I don't think changing the underlying type will help at all. The
    layout of the int object is as follows:

    typedef struct {
    // PyObject_HEAD
    Py_ssize_t ob_refcnt;
    struct _typeobject *ob_type;
    // the actual value
    long ob_ival;
    } PyIntObject;

    On a 64-bit machine, that's 16 bytes for PyObject_HEAD and 8 more
    bytes for the value, 24 bytes total. Changing long to int won't
    decrease the struct size to 20 because the compiler will pad it to 24,
    the nearest multiple of 8. (Forcing the compiler to pack the struct
    won't help because malloc will still pad it for you.)

    If you need to store millions of integers compactly, maybe
    array.array('i') can help.
     
    Hrvoje Niksic, Dec 18, 2007
    #4
  5. Clarence

    Clarence Guest

    On Dec 18, 6:58 pm, Hrvoje Niksic <> wrote:

    > I don't think changing the underlying type will help at all. The


    >
    > On a 64-bit machine, that's 16 bytes for PyObject_HEAD and 8 more
    > bytes for the value, 24 bytes total. Changing long to int won't
    > decrease the struct size to 20 because the compiler will pad it to 24,
    > the nearest multiple of 8. (Forcing the compiler to pack the struct
    > won't help because malloc will still pad it for you.)


    That's an excellent point. And true, too. Thanks, that will lay the
    issue
    to rest.

    > If you need to store millions of integers compactly, maybe
    > array.array('i') can help.
     
    Clarence, Dec 18, 2007
    #5
  6. Clarence

    Terry Reedy Guest

    "Clarence" <> wrote in message
    news:...
    | That's an excellent point. And true, too. Thanks, that will lay the
    | issue to rest.

    Good idea. I think people who moved to 64 bits to get 64 bits would be
    upset if they did not ;-).
     
    Terry Reedy, Dec 18, 2007
    #6
  7. >> On a 64-bit machine, that's 16 bytes for PyObject_HEAD and 8 more
    >> bytes for the value, 24 bytes total. Changing long to int won't
    >> decrease the struct size to 20 because the compiler will pad it to
    >> 24, the nearest multiple of 8. (Forcing the compiler to pack the
    >> struct won't help because malloc will still pad it for you.)

    >
    > That's an excellent point. And true, too. Thanks, that will lay the
    > issue to rest.


    As a side observation, notice how the increase in memory consumption
    does not primarily originate from the increase in the size of a long.
    Instead, all pointers double their sizes. In an OO language (like
    Python), you have lots of pointers, so you will often find that the
    application uses more memory when run in 64-bit mode.

    If that is a concern to you, let me rephrase Terry's observation:
    just try running a 32-bit Python interpreter on your 64-bit system,
    and you may find that it actually runs better because it uses less
    memory.

    Regards,
    Martin
     
    Martin v. Löwis, Dec 18, 2007
    #7
  8. Terry Reedy wrote:
    > Good idea. I think people who moved to 64 bits to get 64 bits would be
    > upset if they did not ;-).


    Windows X64 users still get 32bit ints. The long datatype is 32bit even
    on the 64bit version of Windows.

    Christian
     
    Christian Heimes, Dec 19, 2007
    #8
  9. Clarence

    Terry Reedy Guest

    "Christian Heimes" <> wrote in message
    news:fkainq$61f$...
    | Terry Reedy wrote:
    | > Good idea. I think people who moved to 64 bits to get 64 bits would be
    | > upset if they did not ;-).
    |
    | Windows X64 users still get 32bit ints. The long datatype is 32bit even
    | on the 64bit version of Windows.

    Yes, anyone expecting to get 64 bit ints from Windows would be upset. And
    those who are getting 64 bit ints from real 64 bit OSes would be upset if
    the OPs suggestion were implemented.
     
    Terry Reedy, Dec 19, 2007
    #9
  10. "Terry Reedy" <> writes:

    > "Christian Heimes" <> wrote in message
    > news:fkainq$61f$...
    > | Terry Reedy wrote:
    > | > Good idea. I think people who moved to 64 bits to get 64 bits would be
    > | > upset if they did not ;-).
    > |
    > | Windows X64 users still get 32bit ints. The long datatype is 32bit even
    > | on the 64bit version of Windows.
    >
    > Yes, anyone expecting to get 64 bit ints from Windows would be
    > upset.


    Why? Using only 32 bits doesn't make the structure any smaller
    because of pointer alignment.
     
    Hrvoje Niksic, Dec 22, 2007
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Schnoffos
    Replies:
    2
    Views:
    1,236
    Martien Verbruggen
    Jun 27, 2003
  2. Daniel Rudy

    unsigned long long int to long double

    Daniel Rudy, Sep 19, 2005, in forum: C Programming
    Replies:
    5
    Views:
    1,221
    Peter Shaggy Haywood
    Sep 20, 2005
  3. pereges

    Promoting unsigned long int to long int

    pereges, Jun 30, 2008, in forum: C Programming
    Replies:
    112
    Views:
    2,118
    David Thompson
    Jul 28, 2008
  4. veryhotsausage
    Replies:
    1
    Views:
    1,844
    veryhotsausage
    Jul 4, 2008
  5. Oliver Graeser
    Replies:
    10
    Views:
    595
    Oliver Graeser
    Sep 26, 2008
Loading...

Share This Page