Performance (pystone) of python 2.4 lower then python 2.3 ???

Discussion in 'Python' started by Lucas Hofman, Dec 13, 2004.

  1. Lucas Hofman

    Lucas Hofman Guest

    Hi,

    Just installed Python 2.4 on a machine (RH8.0 Linux) that also has python 2.3
    and python 2.2 installed. The latter came with the linux distribution, the other
    are compiled from source tarballs.

    Comparing them gives the following unexpected result:

    [lucas@oslwb03 test]$ /usr/bin/python pystone.py
    Pystone(1.1) time for 50000 passes = 1.86
    This machine benchmarks at 26881.7 pystones/second
    [lucas@oslwb03 test]$ /usr/local/bin/python2.3 pystone.py
    Pystone(1.1) time for 50000 passes = 1.22
    This machine benchmarks at 40983.6 pystones/second

    This is ok, a 52% speed increase, but:

    lucas@oslwb03 test]$ /usr/local/bin/python2.4 pystone.py
    Pystone(1.1) time for 50000 passes = 1.31
    This machine benchmarks at 38167.9 pystones/second

    A 7% speed DECREASE??? According to the documentation it should be a 5% increase?

    The machine is a 3.0 GHz Xeon box.

    Both python 2.3 and 2.4 where configure without any options.

    Anyone who understands what is going on?

    Regards, Lucas
     
    Lucas Hofman, Dec 13, 2004
    #1
    1. Advertising

  2. Lucas Hofman

    Peter Hansen Guest

    Lucas Hofman wrote:
    > lucas@oslwb03 test]$ /usr/local/bin/python2.4 pystone.py
    > Pystone(1.1) time for 50000 passes = 1.31
    > This machine benchmarks at 38167.9 pystones/second
    >
    > A 7% speed DECREASE??? According to the documentation it should be a 5% increase?
    >
    > The machine is a 3.0 GHz Xeon box.
    >
    > Both python 2.3 and 2.4 where configure without any options.


    For comparison, I do get a decent speedup. Machine is an
    AMD Athlon XP 2500+ (1.82GHz) running Win XP Pro SP2.

    Python 2.3.4: 36393 pystones.
    Python 2.4: 39400 pystones.

    ....about an 8% speedup.
     
    Peter Hansen, Dec 13, 2004
    #2
    1. Advertising

  3. Lucas Hofman wrote:

    > Anyone who understands what is going on?


    It is difficult to measure a speedup that might be
    well within your measurement error.

    Run the same pystone benchmark repeatedly and
    see what variation you get.

    Istvan.
     
    Istvan Albert, Dec 13, 2004
    #3
  4. Peter Hansen <> wrote:
    > For comparison, I do get a decent speedup. Machine is an
    > AMD Athlon XP 2500+ (1.82GHz) running Win XP Pro SP2.
    >
    > Python 2.3.4: 36393 pystones.
    > Python 2.4: 39400 pystones.
    >
    > ...about an 8% speedup.


    On my 2.6 GHz P4 running debian testing I got the following results :-

    $ for p in 2.1 2.2 2.3 2.4; do echo $p; python$p pystone.py 1000000 ; done

    2.1
    Pystone(1.1) time for 1000000 passes = 40.67
    This machine benchmarks at 24588.1 pystones/second
    2.2
    Pystone(1.1) time for 1000000 passes = 39.64
    This machine benchmarks at 25227 pystones/second
    2.3
    Pystone(1.1) time for 1000000 passes = 32.49
    This machine benchmarks at 30778.7 pystones/second
    2.4
    Pystone(1.1) time for 1000000 passes = 29.88
    This machine benchmarks at 33467.2 pystones/second

    Showing that 2.4 is the fastest so far! (And is also a good advert
    for AMD ;-)

    --
    Nick Craig-Wood <> -- http://www.craig-wood.com/nick
     
    Nick Craig-Wood, Dec 14, 2004
    #4
  5. Lucas Hofman

    Lucas Hofman Guest

    Istvan Albert <ialbert <at> mailblocks.com> writes:

    >
    > Lucas Hofman wrote:
    >
    > > Anyone who understands what is going on?

    >
    > It is difficult to measure a speedup that might be
    > well within your measurement error.
    >
    > Run the same pystone benchmark repeatedly and
    > see what variation you get.
    >
    > Istvan.


    Very little variation actually. The system measured is only lightly loaded (and
    it is a 2 processor box). I ran the benchmark > 4 times and got 3 results that
    are within 1% of each other.

    Lucas
     
    Lucas Hofman, Dec 14, 2004
    #5
  6. Lucas Hofman

    Lucas Hofman Guest

    Nick Craig-Wood <nick <at> craig-wood.com> writes:

    >
    > Peter Hansen <peter <at> engcorp.com> wrote:
    > > For comparison, I do get a decent speedup. Machine is an
    > > AMD Athlon XP 2500+ (1.82GHz) running Win XP Pro SP2.
    > >
    > > Python 2.3.4: 36393 pystones.
    > > Python 2.4: 39400 pystones.
    > >
    > > ...about an 8% speedup.

    >
    > On my 2.6 GHz P4 running debian testing I got the following results :-
    >
    > $ for p in 2.1 2.2 2.3 2.4; do echo $p; python$p pystone.py 1000000 ; done
    >
    > 2.1
    > Pystone(1.1) time for 1000000 passes = 40.67
    > This machine benchmarks at 24588.1 pystones/second
    > 2.2
    > Pystone(1.1) time for 1000000 passes = 39.64
    > This machine benchmarks at 25227 pystones/second
    > 2.3
    > Pystone(1.1) time for 1000000 passes = 32.49
    > This machine benchmarks at 30778.7 pystones/second
    > 2.4
    > Pystone(1.1) time for 1000000 passes = 29.88
    > This machine benchmarks at 33467.2 pystones/second
    >
    > Showing that 2.4 is the fastest so far! (And is also a good advert
    > for AMD
    >



    I got this list on a single processor P4 1.6 Ghz:
    2.1
    Pystone(1.1) time for 100000 passes = 6.74
    This machine benchmarks at 14836.8 pystones/second
    2.2
    Pystone(1.1) time for 100000 passes = 6.36
    This machine benchmarks at 15723.3 pystones/second
    2.3
    Pystone(1.1) time for 100000 passes = 4.92
    This machine benchmarks at 20325.2 pystones/second
    2.4
    Pystone(1.1) time for 100000 passes = 4.51
    This machine benchmarks at 22172.9 pystones/second

    Which shows the expected speedup.

    On a dual Xeon 3.0 Ghz:
    2.2
    Pystone(1.1) time for 1000000 passes = 37.45
    This machine benchmarks at 26702.3 pystones/second
    2.3
    Pystone(1.1) time for 1000000 passes = 25.28
    This machine benchmarks at 39557 pystones/second
    2.4
    Pystone(1.1) time for 1000000 passes = 25.94
    This machine benchmarks at 38550.5 pystones/second

    Which shows a decrease in performance. Could this have anything to do with the
    fact that is is a dual processor box?

    Lucas
     
    Lucas Hofman, Dec 14, 2004
    #6
  7. > Which shows a decrease in performance. Could this have anything to do with the
    > fact that is is a dual processor box?


    Doubtful. I'm running dual Pentium 4 2.8 Ghz (Win XP) and get the
    following results:

    C:\>python23\python Python23\lib\test\pystone.py
    Pystone(1.1) time for 50000 passes = 1.43068
    This machine benchmarks at 34948.3 pystones/second

    C:\>python24\python Python24\lib\test\pystone.py
    Pystone(1.1) time for 50000 passes = 1.28359
    This machine benchmarks at 38953.2 pystones/second

    Sw.
     
    Simon Wittber, Dec 14, 2004
    #7
  8. Lucas Hofman

    Mark Asbach Guest

    Hi Lucas,

    > On a dual Xeon 3.0 Ghz:


    [...]

    > Which shows a decrease in performance. Could this have anything to do with the
    > fact that is is a dual processor box?


    Maybe. But my 3Gh P4/HT is also detected as a dual processor machine
    (Kernel 2.6), so it might be a general problem with the Xeon?

    BTW: The discussion got me interested so I compiled Python 2.4 myself
    with different compiler switches and different compilers, just to find
    that generating machine dependant code doesn't speed up the pystone
    benchmark in any way (with gcc 3.3.4) and using intel's icc V8 only
    results in a small speedup (from approx. 40000 with gcc to approx. 41500
    with icc).

    So python performance doesn't seem to be that dependant on machine code
    representation. Maybe you're running into a different problem like cache
    size.

    In addition: running python2.3 with psyco 1.2 proxying all 8 Procs of
    pystone resulted in approx 155000 pystones, while python2.4 with psyco
    1.3 resulted in approx 200000 pystones ...

    Mark
     
    Mark Asbach, Dec 14, 2004
    #8
  9. "Lucas Hofman
    > This machine benchmarks at 38167.9 pystones/second


    Pystone is an abyssmally bad benchmark for comparing the relative speeds of
    different versions of python (it nets out all eval loop improvements and it
    exercises only a microscopic portion of the language).

    I would be interested in seeing other people's comparitive results for pybench,
    test_decimal, and parrotbench.

    To run test_decimal.py, you first have to copy Lib/test/decimal.py into Py2.3's
    lib directory. On an old PentiumIII running WinMe, I get 48.940 sec in Py2.3
    and 44.820 sec in Py2.4, a 8.4% improvement.

    For pybench, I get 7418.90ms in Py2.3 and 6320.07 ms in Py2.4, a 14.8%
    improvement.

    For parrotbench, I get 54.517 seconds in Py2.3 and 45.009 seconds in Py2.4, a
    17.4% improvement.

    It is also interesting to time specific features not covered by the above
    benchmarks. For example, list comprehensions got a nice 60% boost on my
    machine:

    C:\py24\Lib>\python23\python timeit.py -r9 "[i for i in xrange(1000)]"
    100 loops, best of 9: 1.11 msec per loop

    C:\py24\Lib>\python24\python timeit.py -r9 "[i for i in xrange(1000)]"
    1000 loops, best of 9: 417 usec per loop


    Raymond Hettinger
     
    Raymond Hettinger, Dec 14, 2004
    #9
  10. Lucas Hofman

    Lucas Hofman Guest

    Mark Asbach wrote:
    > Hi Lucas,
    >
    >
    >>On a dual Xeon 3.0 Ghz:

    >
    >
    > [...]
    >
    >
    >>Which shows a decrease in performance. Could this have anything to do with the
    >>fact that is is a dual processor box?

    >
    >
    > Maybe. But my 3Gh P4/HT is also detected as a dual processor machine
    > (Kernel 2.6), so it might be a general problem with the Xeon?
    >


    Hi Mark,

    No,the reason that you see 2 times as many processors as really are
    installed is the hyperthreading feature of the Xeon (see
    http://www.infoworld.com/infoworld/article/02/02/25/020225plxeon_1.html)

    I turned it off (in the BIOS). The machine I tested on has 2 (pysical)
    processors installed. Turning on or off does not influence the pystone
    number significantly..

    Regards, Lucas
     
    Lucas Hofman, Dec 14, 2004
    #10
  11. Lucas Hofman

    Dan Guest

    Lucas Hofman wrote:
    > A 7% speed DECREASE??? According to the documentation it should be a 5% increase?


    I also see an 8-10% speed decrease in 2.4 (I built) from 2.3.3 (shipped
    w/Fedora2) in the program I'm writing (best of 3 trials each). Memory
    use seems to be about the same.

    2.4: real 2m44.131s
    2.3.3: real 2m29.865s

    /Dan
    --
    dedded att verizon dott net
     
    Dan, Dec 15, 2004
    #11
  12. Dan> I also see an 8-10% speed decrease in 2.4 (I built) from 2.3.3
    Dan> (shipped w/Fedora2) in the program I'm writing (best of 3 trials
    Dan> each). Memory use seems to be about the same.

    How do you how the compiler flags were the same if you didn't compile both
    versions yourself?

    Skip
     
    Skip Montanaro, Dec 15, 2004
    #12
  13. Lucas Hofman

    Mark Asbach Guest

    Hi Lucas,

    > No,the reason that you see 2 times as many processors as really are
    > installed is the hyperthreading feature of the Xeon (see
    > http://www.infoworld.com/infoworld/article/02/02/25/020225plxeon_1.html)
    >
    > I turned it off (in the BIOS). The machine I tested on has 2 (pysical)
    > processors installed. Turning on or off does not influence the pystone
    > number significantly..


    Sorry for the confusion - that was what I wanted to say: from my
    perspective it is unlikely that the performance decrease seen on your
    dual processor CPU is related to having two processors. Because python
    thinks it has two processors on my machine, too (because of
    hyperthreading) and should exhibit the same decrease then (IF it would
    be coupled to two processors).

    Yours,

    Mark
     
    Mark Asbach, Dec 15, 2004
    #13
  14. Lucas Hofman

    Dan Guest

    Skip Montanaro wrote:
    > Dan> I also see an 8-10% speed decrease in 2.4 (I built) from 2.3.3
    > Dan> (shipped w/Fedora2) in the program I'm writing (best of 3 trials
    > Dan> each). Memory use seems to be about the same.
    >
    > How do you how the compiler flags were the same if you didn't compile both
    > versions yourself?


    I don't, and I also don't know if the Redhat folks modified the sources
    in some way. That's why I mentioned the difference in builds. But I
    suppose that makes this as good as no data.

    /Dan

    --
    dedded att verizon dott net
     
    Dan, Dec 16, 2004
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Zhi Min
    Replies:
    0
    Views:
    405
    Zhi Min
    Jun 11, 2004
  2. Andreas Kostyrka
    Replies:
    0
    Views:
    427
    Andreas Kostyrka
    Dec 17, 2004
  3. Franco Fiorese

    Pystone benchmark: Win vs. Linux (again)

    Franco Fiorese, Jan 28, 2005, in forum: Python
    Replies:
    6
    Views:
    2,782
    Franco Fiorese
    Jan 29, 2005
  4. Replies:
    2
    Views:
    438
    Raymond Hettinger
    Mar 27, 2011
  5. Replies:
    0
    Views:
    296
Loading...

Share This Page