Please enlighten me about PyPy

[Christian Tismer]

Luis M. González wrote:
....


Indeed. Another possibility is to write a PyPy extension module in
RPython, have that translated to C and then use this in your pure python
code. Actually, one of our current rather wild ideas (which might not be
followed) is to be able to even use RPython to write extension modules
for CPython.

Actually, this wild idea is mine, but PyPy may share.

And there is no need for this to be followed, since
I'm going to do this anyway, because I have good personal
reasons:

There is a lot of demand to get Stackless ported more
easily, and the current way of manually fighting the
ever growing number of C modules by hand just sucks.

Waiting for PyPy to get mature enough to replace Stackless
is one way, which takes too long. Waiting for "readyness"
of PyPy to produce something ahead of explorational and
toy interpreters is also no option, although these are very
nice and great for education.

My alternative is to use translation of RPython to produce
extension modules for CPython. Although this is considered
an "implementation detail" by most of the PyPy core people,
companies which are considering to write extensions in C
are just finger-licking for such a detail to use, instead.

I will use it for Stackless Python as a show-case. As an
example, I want to revert itertools and dequeues to almost
their Python equivalent and then translate them into C using
PyPy's translator. While this is of no visible worth for
the normal Python user, it gives me the advantage that
these modules will gain support for the Stackless features
automatically, because the base support is built into PyPy.

This is not trying to split apart from PyPy, or to short-cut its
goals. I'm completely with PyPy's goals, and it will do much
more than RPython translation ever will, this is out of question.

One problem is that we cannot produce a competitive Python
implementation by now. There is a lot more work involved
to gain the necessary speed to be considered. On the
other hand, the produced low-level code for builtin objects
is already almost as efficient as hand-written code in many
cases. As a proof of concept, I have used this to turn
an application program into compiled RPython, which became
over 10 times faster and outperformed its highly optimized
Java counterpart.

I just believe that RPython is a piece of gold, a gem created
aside while trying to build the huge thing, and we should
not leave its potential unused. Sure, it needs some processing
and finishing to make it easier to use and have better support
for interfacing to existing CPython objects.

After three years, the PyPy project can really take the chance
to produce a small, useful tool for the ambitioned developer.
Not making his task trivial, as PyPy will, but considerably
simpler than writing C.

merry christmas -- chris
--
Christian Tismer :^) <mailto:[email protected]>
tismerysoft GmbH : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

 

[Christian Tismer]

This is not trying to split apart from PyPy, or to short-cut its
goals. I'm completely with PyPy's goals, and it will do much
more than RPython translation ever will, this is out of question.

Of course I meant "this is beyond question"

--
Christian Tismer :^) <mailto:[email protected]>
tismerysoft GmbH : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

 

[=?iso-8859-1?q?Luis_M._Gonz=E1lez?=]

Hi Christian,

I'd like to know, in your opinion, how far is the goal of making pypy
complete and fast?
Regarding the current state of the project, are you confident that the
goals will be met, or you still have doubts?
(The same questions go for the RPython translator project as
stand-alone tool)...

Luis

 

[Christian Tismer]

I'd like to know, in your opinion, how far is the goal of making pypy
complete and fast?

Me too

PyPy is doing a great job, that's for sure.

I'm hesitant with making estimates, after I learned what a bad
job I'm doing at extrapolation.

First I thought that we would reach our first self-contained
PyPy much earlier, gaining CPython speed. When I had lost my
faith a little, we suddenly made it. Then we made very much
progress in speeding it up, but still we are 7 to 10 times
slower than CPython, and it gets harder and harder.

Now we are aiming at JIT technology, which is able to accelerate
Python quite much in many cases, even if we should fail to
improve the basic translation reasonably. Of course it would
be nice to reach both aims, and I expect that the things we
will learn from writing the JIT will also improve the static
translation.

Completenes? In some aspects, like CPython compatibility,
we are very complete, maybe more than the original, even.
Concerning the promises we made to the EU, we will have a hard
time to make it all happen on schedule, but we have a chance,
given that the support by external helpers keeps growing.
Concerning all what we ever said about PyPy? This is a never-
ending story and unlimited, as I don't expect PyPy to stop
growing and extending in any near future, like Python doesn't...

Regarding the current state of the project, are you confident that the
goals will be met, or you still have doubts?

I no longer have doubts about success. I never really had, but
my time estimates are less pessimistic as they sometimes were.
I don't really believe that we will outperform CPython with
a translated RPython interpreter by the end of next year.
We will probably, in conjunction with a JIT compiler.
For gaining a maximum of performance, my guess is another
two years would make very much sense.

(The same questions go for the RPython translator project as
stand-alone tool)...

This is a matter of viewpoint. As a developer, I'm able to
create extenson modules on demand without any explicit tools.
Enabling/supporting the most needed features might be doable
in a couple of weeks and months, depending on the expectations.
A simple-to-use, stand-alone tool for making extensions will
maybe not happen at all, unless we get a lot of extra-resources.

I'm expecting something to happen in the first quarter of the year.
It depends on how much we can extend activities without missing
the promised goals which we have to fulfill, and how much
sponsoring we can create. I believe that by providing just enough
support to make some companies productive in using PyPy, we will
create enough funding for the time after 2006 to make PyPy survive
for a long time, and creating tools like this will become a
self-running motor for PyPy. A matter of good balancing

merry christmas -- chris
--
Christian Tismer :^) <mailto:[email protected]>
tismerysoft GmbH : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/