Generic Python Benchmark suite?

Discussion in 'Python' started by Anand Vaidya, Jan 18, 2010.

  1. Anand Vaidya

    Anand Vaidya Guest

    Is there a generic python benchmark suite in active development? I am
    looking forward to comparing some code on various python
    implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
    Psyco).

    I am happy with something that gives me a relative number eg: ULS is
    30% faster than CPy2.x etc

    I found pybench which is probably not maintained actively.

    What do you suggest?

    PS: I think a benchmark should cover file / network, database I/O,
    data structures (dict, list etc), object creation/manipulation,
    numbers, measure looping inefficiencies, effects of caching (memcache
    etc) at the minimum
    Anand Vaidya, Jan 18, 2010
    #1
    1. Advertising

  2. Anand Vaidya, 18.01.2010 10:58:
    > Is there a generic python benchmark suite in active development?
    > [...]
    > PS: I think a benchmark should cover file / network, database I/O,
    > data structures (dict, list etc), object creation/manipulation,
    > numbers, measure looping inefficiencies, effects of caching (memcache
    > etc) at the minimum


    That doesn't sound generic at all. Maybe you should prefer an application
    based benchmark instead.

    Stefan
    Stefan Behnel, Jan 18, 2010
    #2
    1. Advertising

  3. Anand Vaidya

    Dotan Cohen Guest

    Dotan Cohen, Jan 18, 2010
    #3
  4. Le Mon, 18 Jan 2010 11:30:16 +0100, Stefan Behnel a écrit :
    > Anand Vaidya, 18.01.2010 10:58:
    >> Is there a generic python benchmark suite in active development? [...]
    >> PS: I think a benchmark should cover file / network, database I/O,
    >> data structures (dict, list etc), object creation/manipulation,
    >> numbers, measure looping inefficiencies, effects of caching (memcache
    >> etc) at the minimum

    >
    > That doesn't sound generic at all. Maybe you should prefer an
    > application based benchmark instead.


    Actually, it sounds much more generic that most of the benchmarks we
    usually rely on :)
    Antoine Pitrou, Jan 18, 2010
    #4
  5. Le Mon, 18 Jan 2010 01:58:42 -0800, Anand Vaidya a écrit :
    > Is there a generic python benchmark suite in active development? I am
    > looking forward to comparing some code on various python implementations
    > (primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco).
    >
    > I am happy with something that gives me a relative number eg: ULS is 30%
    > faster than CPy2.x etc
    >
    > I found pybench which is probably not maintained actively.


    pybench is not abandoned. However, it only gets an addition now and then.
    More importantly, it's a set of low-level microbenchmarks designed to
    stress the execution cost of certain primitives. It won't give you the
    answer to any high-level questions.

    In the SVN sandbox (*) you'll find a couple of other benchmarks:
    - stringbench, stressing string operations
    - iobench, stressing common file I/O operations
    - ccbench, trying to measure interpreter efficiency in the face of multi-
    threaded workloads

    (*) http://svn.python.org/view/sandbox/trunk/

    Regards

    Antoine.
    Antoine Pitrou, Jan 18, 2010
    #5
  6. Anand Vaidya

    Terry Reedy Guest

    On 1/18/2010 4:58 AM, Anand Vaidya wrote:
    > Is there a generic python benchmark suite in active development? I am
    > looking forward to comparing some code on various python
    > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
    > Psyco).


    You might find this interesting if you have not seen it before:
    http://code.google.com/p/unladen-swallow/wiki/Benchmarks
    Terry Reedy, Jan 18, 2010
    #6
  7. Anand Vaidya

    Anand Vaidya Guest

    On Jan 19, 5:42 am, Terry Reedy <> wrote:
    > On 1/18/2010 4:58 AM, Anand Vaidya wrote:
    >
    > > Is there a generic python benchmark suite in active development? I am
    > > looking forward to comparing some code on various python
    > > implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
    > > Psyco).

    >
    > You might find this interesting if you have not seen it before:http://code.google.com/p/unladen-swallow/wiki/Benchmarks


    @Antoine, Terry,

    Thanks for the suggestions.

    I will investigate those. I just ran the pybench, doesn't run on 3.x,
    2to3 fails. I was hoping to locate something like Phoronix Test Suite
    (that is a suite of microbenchmarks for Linux OS)

    Regards
    Anand
    Anand Vaidya, Jan 19, 2010
    #7
  8. Le Mon, 18 Jan 2010 21:05:26 -0800, Anand Vaidya a écrit :
    > @Antoine, Terry,
    >
    > Thanks for the suggestions.
    >
    > I will investigate those. I just ran the pybench, doesn't run on 3.x,
    > 2to3 fails.


    You just have to use the pybench version that is bundled with 3.x (in the
    Tools directory).
    Antoine Pitrou, Jan 19, 2010
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Murat Tasan
    Replies:
    1
    Views:
    8,042
    Chaitanya
    Feb 3, 2009
  2. Replies:
    2
    Views:
    435
  3. stelios xanthakis

    ANN: Python benchmark suite

    stelios xanthakis, Apr 9, 2005, in forum: Python
    Replies:
    0
    Views:
    345
    stelios xanthakis
    Apr 9, 2005
  4. Matthijs van Waveren

    SPEC is seeking candidate codes for an MPI-based benchmark suite

    Matthijs van Waveren, Dec 9, 2004, in forum: C Programming
    Replies:
    2
    Views:
    350
    Rich Townsend
    Dec 9, 2004
  5. gabriele renzi

    a benchmark suite for the interpreter ?

    gabriele renzi, Apr 16, 2004, in forum: Ruby
    Replies:
    1
    Views:
    102
    Robert Klemme
    Apr 17, 2004
Loading...

Share This Page