Is a 32-bit build faster than a 64-bit build

Discussion in 'Python' started by Raymond Hettinger, Nov 12, 2010.

  1. Has anyone here benchmarked a 32-bit Python versus a 64-bit Python for
    Django or some other webserver?

    My hypotheses is that for apps not needing the 64-bit address space,
    the 32-bit version has better memory utilization and hence better
    cache performance. If so, then switching python versions may enable a
    single server to handle a greater traffic load. Has anyone here tried
    that?


    Raymond
     
    Raymond Hettinger, Nov 12, 2010
    #1
    1. Advertising

  2. Am 12.11.2010 22:24, schrieb Raymond Hettinger:
    > My hypotheses is that for apps not needing the 64-bit address space,
    > the 32-bit version has better memory utilization and hence better
    > cache performance. If so, then switching python versions may enable a
    > single server to handle a greater traffic load. Has anyone here tried
    > that?

    In most cases (all?) I have seen did the performance not differ when
    compared
    between running python scripts in 32 or 64 bit env.
    But that is not true for compiled code.
    Just take your script and do some benchmarking.

    In germany we say: probieren geht ├╝ber studieren.
    Don't study - try it.

    An ex-post explanation will show at hand, I promise :)
     
    Stefan Sonnenberg-Carstens, Nov 12, 2010
    #2
    1. Advertising

  3. Raymond Hettinger, 12.11.2010 22:24:
    > Has anyone here benchmarked a 32-bit Python versus a 64-bit Python for
    > Django or some other webserver?
    >
    > My hypotheses is that for apps not needing the 64-bit address space,
    > the 32-bit version has better memory utilization and hence better
    > cache performance.


    OTOH, x86_64 has more registers and allows faster compiler flags for
    default installations (e.g. there is no x86_64 processor without MMX and
    SSE). So, if you don't compile your software yourself (including OS kernel
    and libraries, e.g. OpenSSL), it will likely run faster on 64bits simply
    due to the better compiler optimisations.

    There are good reasons for both being able to run faster depending on the
    specific code, so benchmarking is basically all you can do.

    Stefan
     
    Stefan Behnel, Nov 12, 2010
    #3
  4. On Fri, 12 Nov 2010 13:24:09 -0800 (PST)
    Raymond Hettinger <> wrote:
    > Has anyone here benchmarked a 32-bit Python versus a 64-bit Python for
    > Django or some other webserver?
    >
    > My hypotheses is that for apps not needing the 64-bit address space,
    > the 32-bit version has better memory utilization and hence better
    > cache performance. If so, then switching python versions may enable a
    > single server to handle a greater traffic load. Has anyone here tried
    > that?


    On micro-benchmarks, x86-64 is always faster by about 10-30% compared
    to x86, simply because of the extended register set and other
    additional niceties.

    On a benchmark stressing the memory system a little more such as
    dcbench-py3k.py in http://bugs.python.org/issue9520, the 64-bit build
    is still faster until the tested data structure (a large dict)
    overwhelms the 2MB last-level cache in my CPU, after which the 32-bit
    build becomes 10% faster for the same numbers of elements:

    To be clear, here are the figures in 64-bit mode:

    10000 words ( 9092 keys), 2893621 inserts/s, 13426069 lookups/s, 86 bytes/key (0.8MB)
    20000 words ( 17699 keys), 3206654 inserts/s, 12338002 lookups/s, 44 bytes/key (0.8MB)
    40000 words ( 34490 keys), 2613517 inserts/s, 7643726 lookups/s, 91 bytes/key (3.0MB)
    80000 words ( 67148 keys), 2579562 inserts/s, 4872069 lookups/s, 46 bytes/key (3.0MB)
    160000 words ( 130897 keys), 2377487 inserts/s, 5765316 lookups/s, 48 bytes/key (6.0MB)
    320000 words ( 254233 keys), 2119978 inserts/s, 5003979 lookups/s, 49 bytes/key (12.0MB)
    640000 words ( 493191 keys), 1965413 inserts/s, 4640743 lookups/s, 51 bytes/key (24.0MB)
    1280000 words ( 956820 keys), 1854546 inserts/s, 4338543 lookups/s, 52 bytes/key (48.0MB)

    And here are the figures in 32-bit mode:

    10000 words ( 9092 keys), 2250163 inserts/s, 9487229 lookups/s, 43 bytes/key (0.4MB)
    20000 words ( 17699 keys), 2543235 inserts/s, 7653839 lookups/s, 22 bytes/key (0.4MB)
    40000 words ( 34490 keys), 2360162 inserts/s, 8851543 lookups/s, 45 bytes/key (1.5MB)
    80000 words ( 67148 keys), 2415169 inserts/s, 8581037 lookups/s, 23 bytes/key (1.5MB)
    160000 words ( 130897 keys), 2203071 inserts/s, 6914732 lookups/s, 24 bytes/key (3.0MB)
    320000 words ( 254233 keys), 2005980 inserts/s, 5670133 lookups/s, 24 bytes/key (6.0MB)
    640000 words ( 493191 keys), 1856385 inserts/s, 4929790 lookups/s, 25 bytes/key (12.0MB)
    1280000 words ( 956820 keys), 1746364 inserts/s, 4530747 lookups/s, 26 bytes/key (24.0MB)


    However, it's not obvious to me that a program like "Django or some
    other webserver" would have really bad cache locality. Even if the
    total working set is larger than the CPU cache, there can still be
    quite a good cache efficiency if a large fraction of CPU time is spent
    on small datasets.

    By the way, I've been experimenting with denser dicts and with
    linear probing (in the hope that it will improve cache efficiency and
    spatial locality in real applications), and there doesn't seem to be
    adverse consequences on micro-benchmarks. Do you think I should upload
    a patch?

    Regards

    Antoine.
     
    Antoine Pitrou, Nov 13, 2010
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ken Tucker

    Anything faster than stat() ?

    Ken Tucker, Jul 7, 2003, in forum: Perl
    Replies:
    1
    Views:
    1,639
    Steve Grazzini
    Jul 8, 2003
  2. =?Utf-8?B?Sm9obiBQYXVsLiBBIChNQ1AgSUQjIDMwMTUxNzYp

    Comment : ASP.NET Performs 10 Times Faster than J2EE

    =?Utf-8?B?Sm9obiBQYXVsLiBBIChNQ1AgSUQjIDMwMTUxNzYp, Jul 1, 2004, in forum: ASP .Net
    Replies:
    6
    Views:
    745
    Kevin Spencer
    Jul 7, 2004
  3. NOBODY
    Replies:
    2
    Views:
    403
    NOBODY
    Oct 30, 2004
  4. Timo Nentwig
    Replies:
    6
    Views:
    8,893
    el goog
    Feb 26, 2005
  5. Replies:
    1
    Views:
    352
    Tom Hawtin
    Jan 24, 2007
Loading...

Share This Page