Generic Python Benchmark suite?

[Anand Vaidya]

Is there a generic python benchmark suite in active development? I am
looking forward to comparing some code on various python
implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
Psyco).

I am happy with something that gives me a relative number eg: ULS is
30% faster than CPy2.x etc

I found pybench which is probably not maintained actively.

What do you suggest?

PS: I think a benchmark should cover file / network, database I/O,
data structures (dict, list etc), object creation/manipulation,
numbers, measure looping inefficiencies, effects of caching (memcache
etc) at the minimum

 

[Stefan Behnel]

Anand Vaidya, 18.01.2010 10:58:

Is there a generic python benchmark suite in active development?
[...]
PS: I think a benchmark should cover file / network, database I/O,
data structures (dict, list etc), object creation/manipulation,
numbers, measure looping inefficiencies, effects of caching (memcache
etc) at the minimum

That doesn't sound generic at all. Maybe you should prefer an application
based benchmark instead.

Stefan

 

[Dotan Cohen]

What do you suggest?
$ man time

 

[Antoine Pitrou]

Le Mon, 18 Jan 2010 11:30:16 +0100, Stefan Behnel a écrit :

Anand Vaidya, 18.01.2010 10:58:

Is there a generic python benchmark suite in active development? [...]
PS: I think a benchmark should cover file / network, database I/O,
data structures (dict, list etc), object creation/manipulation,
numbers, measure looping inefficiencies, effects of caching (memcache
etc) at the minimum

That doesn't sound generic at all. Maybe you should prefer an
application based benchmark instead.

Actually, it sounds much more generic that most of the benchmarks we
usually rely on

 

[Antoine Pitrou]

Le Mon, 18 Jan 2010 01:58:42 -0800, Anand Vaidya a écrit :

Is there a generic python benchmark suite in active development? I am
looking forward to comparing some code on various python implementations
(primarily CPython 2.x, CPython 3.x, UnladenSwallow, Psyco).

I am happy with something that gives me a relative number eg: ULS is 30%
faster than CPy2.x etc

I found pybench which is probably not maintained actively.

pybench is not abandoned. However, it only gets an addition now and then.
More importantly, it's a set of low-level microbenchmarks designed to
stress the execution cost of certain primitives. It won't give you the
answer to any high-level questions.

In the SVN sandbox (*) you'll find a couple of other benchmarks:
- stringbench, stressing string operations
- iobench, stressing common file I/O operations
- ccbench, trying to measure interpreter efficiency in the face of multi-
threaded workloads

(*) http://svn.python.org/view/sandbox/trunk/

Regards

Antoine.

 

[Terry Reedy]

Is there a generic python benchmark suite in active development? I am
looking forward to comparing some code on various python
implementations (primarily CPython 2.x, CPython 3.x, UnladenSwallow,
Psyco).

You might find this interesting if you have not seen it before:
http://code.google.com/p/unladen-swallow/wiki/Benchmarks

 

[Anand Vaidya]

You might find this interesting if you have not seen it before:http://code.google.com/p/unladen-swallow/wiki/Benchmarks

@Antoine, Terry,

Thanks for the suggestions.

I will investigate those. I just ran the pybench, doesn't run on 3.x,
2to3 fails. I was hoping to locate something like Phoronix Test Suite
(that is a suite of microbenchmarks for Linux OS)

Regards
Anand

 

[Antoine Pitrou]

Le Mon, 18 Jan 2010 21:05:26 -0800, Anand Vaidya a écrit :

@Antoine, Terry,

Thanks for the suggestions.

I will investigate those. I just ran the pybench, doesn't run on 3.x,
2to3 fails.

You just have to use the pybench version that is bundled with 3.x (in the
Tools directory).