IronPython vs CPython: faster in 1.6 times?

[dmitrey]

Hi all,
the url http://torquedev.blogspot.com/2008/02/changes-in-air.html
(blog of a game developers)
says IronPython is faster than CPython in 1.6 times.
Is it really true?
If yes, what are IronPython drawbacks vs CPython?
And is it possible to use IronPython in Linux?

D.

 

[Jeff]

IronPython runs on top of .NET. I would be suspect of any claims that
it is faster than cPython, just as I would of claims that Stackless or
Jython are faster.

 

[Steve Holden]

IronPython runs on top of .NET. I would be suspect of any claims that
it is faster than cPython, just as I would of claims that Stackless or
Jython are faster.

Well don't be. There are benchmarks that clearly show IronPython as
faster for selected tests. Other tests show CPython running more quickly.

As always, a benchmark is only really valuable on a typical workload for
the intended application.

regards
Steve

 

[Mike C. Fletcher]

Hi all,
the url http://torquedev.blogspot.com/2008/02/changes-in-air.html
(blog of a game developers)
says IronPython is faster than CPython in 1.6 times.
Is it really true?

On certain platforms, I believe so, for certain types of operations.
Not sure if Mono also provides a speedup. Most of the speedup is due to
large amounts of (paid) effort being spent creating a high-speed ILM
optimizer. Because IronPython can make use of the work that MS has been
poring into Dynamic language compilation, it can get quite a few
speedups that CPython just doesn't get because they don't have the
people to do the work. Optimising code automatically is a reasonably
complex process that tends to introduce lots of potential errors. The
CPython devs are not AFAIK working on performance much these days, so
likely CPython won't improve any time soon, i.e. 3.0 will likely not be
any faster than 2.5 from anything I've heard.

PyPy is attempting to address this issue via a separate interpreter, but
it's currently just playing catch-up on performance most of the time.
It does have a JIT, and might one day be fast enough to be a usable
replacement for CPython, but it will require a lot of developer-years to
get it there, most likely.

It would be really nice if PyPy could get Python 2.5 running say 5x
faster and then run with that. With that Python would open out into
entire new areas of applicability, becoming reasonable as an embedded
language, or a systems language. Only 2x slower than C would make
Python pretty close to a perfect language...
(far more attractive than a slightly tweaked syntax IMO). That's
probably 5-10 developer years out, though, not counting any distractions
from trying to support Python 3.x.

If yes, what are IronPython drawbacks vs CPython?

Mostly library access from what I understand. Numpy and SciPy, for
instance, are not AFAIK ported to IronPython. Those are the people who
*really* need speed, and without those APIs having "Python" available
faster doesn't really mean much. IronPython has access to the Win32
API, so if you want to use Win32 APIs, rather than the CPython ones,
you're golden, but Numpy/SciPy's interface is *really* elegant for
working with large arrays of data.

If you're trying to write tight numeric loops for gigabyte arrays in raw
Python, 1.6 times performance isn't really usable... even 5x is just
barely usable. Numpy lets you use the optimized (C) libraries for the
heavy lifting and Python friendliness where you interact with humans.
If Python were 10x faster you *might* completely rewrite your Numpy in
Python code, but I'd expect that naive Python code would still be beat
handily by BLAS or what have you under the covers in Numpy.

If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.

are the two lines that tend to preclude CPython ever becoming *really*
fast. Optimizing code is almost always complex and hard to explain.
You need lots and lots of thought to make a compiler smart enough to
wring performance out of naive code, and you need a lot of thought to
reason about what the compiler is going to do under the covers with your
code. IronPython (and Jython, and Parrot) can use the underlying
system's complexity without introducing it into their own project. PyPy
is trying to create the complexity itself (with the advantage of a
smaller problem domain than optimising *every* language).

And is it possible to use IronPython in Linux?

Yes, running on Mono, though again, I don't believe Mono has had the
optimisation effort put in to make it competitive with MS's platforms.

Just my view from out in the boonies,
Mike

 

[bearophileHUGS]

Mike C. Fletcher:

Not sure if Mono also provides a speedup.

There is a set of good benchmarks here, the answer is negative:
http://shootout.alioth.debian.org/sandbox/benchmark.php?test=all&lang=iron

Bye,
bearophile

 

[Stefan Behnel]

Mike C. Fletcher:

There is a set of good benchmarks here, the answer is negative:
http://shootout.alioth.debian.org/sandbox/benchmark.php?test=all&lang=iron

This doesn't look like Mono to me:

IronPython 1.1 (1.1) on .NET 2.0.50727.42

Stefan

 

[Istvan Albert]

Hi all,
the urlhttp://torquedev.blogspot.com/2008/02/changes-in-air.html
(blog of a game developers)
says IronPython is faster than CPython in 1.6 times.
Is it really true?

This is a second time around that IronPython piqued my interest
sufficiently to create a toy program to benchmark it and I must say
the results are not encouraging:

$ python bench.py
Elapsed time: 1.10 s

$ ipy bench.py
Elapsed time:65.01 s

and here is the benchmark program:

import time
start = time.time()

def foo(a):
return a * a

data = {}
for i in xrange( 10**6 ):
data = foo(i)

print 'Elapsed time:%5.2f s' % ( time.time() - start)

 

[Christian Heimes]

Hi all,
the url http://torquedev.blogspot.com/2008/02/changes-in-air.html
(blog of a game developers)
says IronPython is faster than CPython in 1.6 times.
Is it really true?
If yes, what are IronPython drawbacks vs CPython?
And is it possible to use IronPython in Linux?

IronPython implements only a limited subset of Python. [1] All extension
that depend on a C code don't work under IronPython.

Christian

[1]
http://www.codeplex.com/IronPython/Wiki/View.aspx?title=Differences&referringTitle=Home

 

[Arnaud Delobelle]

Hi all,
the urlhttp://torquedev.blogspot.com/2008/02/changes-in-air.html
(blog of a game developers)
says IronPython is faster than CPython in 1.6 times.
Is it really true?

This is a second time around that IronPython piqued my interest
sufficiently to create a toy program to benchmark it and I must say
the results are not encouraging:

$ python bench.py
Elapsed time: 1.10 s

$ ipy bench.py
Elapsed time:65.01 s

and here is the benchmark program:

import time
start = time.time()

def foo(a):
    return a * a

data = {}
for i in xrange( 10**6 ):
    data = foo(i)

print 'Elapsed time:%5.2f s' % ( time.time() - start)

Could it be because .NET doesn't have arbitrary length integer types
and your little benchmark will create lots of integers > 2**32 ?
What is the result if you replace foo(a) with

def foo(a): return sqrt(a)

 

[Istvan Albert]

Could it be because .NET doesn't have arbitrary length integer types
and your little benchmark will create lots of integers > 2**32 ?
What is the result if you replace foo(a) with

def foo(a): return sqrt(a)

Good observation, in the case above the run times are about the same.

i.

 

[mensanator]

This is a second time around that IronPython piqued my interest
sufficiently to create a toy program to benchmark it and I must say
the results are not encouraging:

$ python bench.py
Elapsed time: 1.10 s

$ ipy bench.py
Elapsed time:65.01 s

and here is the benchmark program:

import time
start = time.time()

def foo(a):
    return a * a

data = {}
for i in xrange( 10**6 ):
    data = foo(i)

print 'Elapsed time:%5.2f s' % ( time.time() - start)

Could it be because .NET doesn't have arbitrary length integer types

A good reason to not use it.

 

[Luis M. González]

Hi all,
the urlhttp://torquedev.blogspot.com/2008/02/changes-in-air.html
(blog of a game developers)
says IronPython is faster than CPython in 1.6 times.
Is it really true?
If yes, what are IronPython drawbacks vs CPython?
And is it possible to use IronPython in Linux?

D.

I posted a little benchmark some time ago in ironpython's list that
showed a big performance gap between both implementations (being
cpython much faster than ironpython).
Jim Hugunin replied showing that making the script more abstract
(encapsulating code as much as possible into functions) the .NET
framework makes a better job at optimizing everything.

So I took Istvan script and made a little modification as follows:

import time

def foo(a):
return a * a

def do():
start = time.time()
data = {}
for i in xrange( 10**6 ):
data = foo(i)
print 'Elapsed time:%5.2f s' % ( time.time() - start)

do() # pre-run to avoid initialization time
do()

import psyco
psyco.full()

print '\nNow with psyco...\n'

do()
do()

input()

The result is that it runs slighty faster in both, IP and CP, but
cpython is still faster (around 2x) than ironpython.
However, when using psyco simply blows everything out of the water...

These are my results.

Ironpython 2.0 Alpha 8:
Elapsed time: 3.14 s
Elapsed time: 3.36 s

cpyhon 2.5:
Elapsed time: 1.55 s
Elapsed time: 1.58 s

Now with psyco...

Elapsed time: 0.88 s
Elapsed time: 0.80 s

 

[Christian Heimes]

The result is that it runs slighty faster in both, IP and CP, but
cpython is still faster (around 2x) than ironpython.
However, when using psyco simply blows everything out of the water...

CPython is very fast here because it keeps blocks of allocated integer
objects to reduce the memory overhead. Your test shows that Python
highly specialized and optimized memory management for ints is superior
over IronPython's more general memory management. It also shows that
psyco optimized the function call. It's probably inlined.

You could replace foo(i) by i*i and see how much function calls affect
the speed.

Christian

 

[Steve Holden]

CPython is very fast here because it keeps blocks of allocated integer
objects to reduce the memory overhead. Your test shows that Python
highly specialized and optimized memory management for ints is superior
over IronPython's more general memory management. It also shows that
psyco optimized the function call. It's probably inlined.

You could replace foo(i) by i*i and see how much function calls affect
the speed.

At the risk of boring you all, allow me to repeat:

"""
As always, a benchmark is only really valuable on a typical workload for
the intended application.
"""

There is no "better" and "worse" in the general case. Make rational
decisions. Benchmark your applications, don't scheme to make an
arbitrary benchmark run faster.

regards
Steve

 

[bearophileHUGS]

Stefan Behnel:

This doesn't look like Mono to me:
IronPython 1.1 (1.1) on .NET 2.0.50727.42

You are right! I think this shows that IronPython isn't faster than
CPython at all (And it uses more memory).

Bye,
bearophile

 

[Isaac Gouy]

This doesn't look like Mono to me:

IronPython 1.1 (1.1) on .NET 2.0.50727.42

Stefan

Have you actually looked at the version string from IronPython-1.1-
Bin.zip running on Mono?

 

[Isaac Gouy]

Stefan Behnel:


You are right!
No.

I think this shows that IronPython isn't faster than
CPython at all (And it uses more memory).

 

[Fuzzyman]

This doesn't look like Mono to me:

IronPython1.1 (1.1) on .NET 2.0.50727.42

Running on Debian? Fairly unlikely.

Fuzzyman
http://www.manning.com/foord

 

[Fuzzyman]

Well don't be. There are benchmarks that clearly showIronPythonas
faster for selected tests. Other tests show CPython running more quickly.

This has been our experience at Resolver Systems. Code that makes
heavy use of function calls, non-exceptional exception handling or
arithmetic tends to run faster whereas the built-in types tend to be
slower.

It makes profiling code for optimisation all the more important
because all your usual assumptions about Python performance tend to be
wrong...

Michael Foord
http://www.manning.com/foord

 

[Carsten Haese]

Running on Debian? Fairly unlikely.

Well, that *is* what the version string reports for IronPython on Mono
on Linux:

$ uname -sr
Linux 2.6.18-1.2200.fc5smp
$ mono ipy.exe
IronPython 1.1 (1.1) on .NET 2.0.50727.42
Copyright (c) Microsoft Corporation. All rights reserved.