Standardizing RPython - it's time.

Discussion in 'Python' started by John Nagle, Oct 11, 2010.

  1. John Nagle

    John Nagle Guest

    It may be time to standardize "RPython".

    There are at least three implementations of "RPython" variants - PyPy,
    Shed Skin, and RPython for LLVM. The first two are up and running.
    There's a theory paper on the subject:

    http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.1457&rep=rep1&type=pdf

    All three have somewhat different restrictions:

    PyPy's Rpython:
    http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html

    Shed Skin:
    file:///C:/Users/nagle/AppData/Local/Temp/shedskin-tutorial-0.3.html

    Rpython for LLVM:
    http://code.google.com/p/rpython/

    So a language standardization effort, independent of CPython,
    would be useful.

    All attempts to make the dialect defined by CPython significantly
    faster have failed. PyPy did not achieve much of a speed
    improvement over CPython, and is sometimes slower. Unladen Swallow
    has not achieved a significant speedup, and is now a year behind
    on quarterly releases, indicating that the project is in serious
    trouble. PyPy's RPython, Psyco, and Shed Skin, on the other hand,
    all achieved order of magnitude or better speedups over CPython.
    So this is the way to go for higher performance.

    RPython may be the future of Python, a "golden middle way" between
    statically and dynamically typed languages. A basic concept of RPython
    is that it is declaration-free, like the CPython dialect, but limits
    the ability to change the type of an object at run time.
    Both PyPy's RPython and Shed Skin use type inference to determine
    the types of objects.

    PyPy's RPython has the best defined set of restrictions.
    Most of the restrictions are for speed, such as the unique-type
    restriction. A few simply reflect unimplemented features,
    such as generators. (Generators are not inherently inefficient.)

    A significant advantage of having a well defined set of
    restrictions is that it encourages writing Python libraries which will
    run under both RPython and CPython. This makes migration to a faster
    system much easier.

    John Nagle
    John Nagle, Oct 11, 2010
    #1
    1. Advertising

  2. John Nagle

    Jed Smith Guest

    On Mon, Oct 11, 2010 at 4:01 PM, John Nagle <> wrote:

    > file:///C:/Users/nagle/AppData/Local/Temp/shedskin-tutorial-0.3.html


    This gives me a 404. Your Web server is broken! Fix it! ;)

    Temporarily mirrored: http://jedsmith.org/tmp/shedskin-tutorial-0.5.html

    --
    Jed Smith
    Jed Smith, Oct 11, 2010
    #2
    1. Advertising

  3. John Nagle

    John Nagle Guest

    John Nagle, Oct 11, 2010
    #3
  4. John Nagle

    Ryan Kelly Guest

    On Mon, 2010-10-11 at 13:01 -0700, John Nagle wrote:
    > It may be time to standardize "RPython".
    >
    > There are at least three implementations of "RPython" variants - PyPy,
    > Shed Skin, and RPython for LLVM. The first two are up and running.
    > There's a theory paper on the subject:
    >
    > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.1457&rep=rep1&type=pdf
    >
    > All three have somewhat different restrictions:
    >
    > PyPy's Rpython:
    > http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html
    >
    > Shed Skin:
    > file:///C:/Users/nagle/AppData/Local/Temp/shedskin-tutorial-0.3.html
    >
    > Rpython for LLVM:
    > http://code.google.com/p/rpython/
    >
    > So a language standardization effort, independent of CPython,
    > would be useful.


    A similar topic was recently discussed on the pypy-dev mailing list:

    http://codespeak.net/pipermail/pypy-dev/2010q3/006170.html


    My interpretation is that the pypy devs are -0 on such a standardisation
    effort, as it would give them less flexibility in moulding rpython for
    their specific needs. Adding features to rpython that make it better as
    a general-purpose programming language could actually make it *worse* as
    a specialised language for building pypy.

    OTOH, there does seem to be a growing interest in using rpython as a
    stand-alone language. I've used it for some small projects and it
    worked out great.

    But the intersection of (people who want rpython as a general-purpose
    language) and (people who have the ability to make that happen) seems to
    be approximately zero at the moment...


    Ryan

    --
    Ryan Kelly
    http://www.rfk.id.au | This message is digitally signed. Please visit
    | http://www.rfk.id.au/ramblings/gpg/ for details


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.10 (GNU/Linux)

    iEYEABECAAYFAkyzd8YACgkQfI5S64uP50pozwCgiTc2ZqZGRrrLTAcQf7cBoQvl
    mssAn1OcpaKGVIBZVJVGCWgsZsWUpVjk
    =xg5a
    -----END PGP SIGNATURE-----
    Ryan Kelly, Oct 11, 2010
    #4
  5. John Nagle

    John Nagle Guest

    On 10/11/2010 1:47 PM, Ryan Kelly wrote:
    > On Mon, 2010-10-11 at 13:01 -0700, John Nagle wrote:
    >> It may be time to standardize "RPython".
    >>
    >> There are at least three implementations of "RPython" variants - PyPy,
    >> Shed Skin, and RPython for LLVM. The first two are up and running.
    >> There's a theory paper on the subject:
    >>
    >> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.1457&rep=rep1&type=pdf
    >>
    >> All three have somewhat different restrictions:
    >>
    >> PyPy's Rpython:
    >> http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html
    >>
    >> Shed Skin:
    >> file:///C:/Users/nagle/AppData/Local/Temp/shedskin-tutorial-0.3.html
    >>
    >> Rpython for LLVM:
    >> http://code.google.com/p/rpython/
    >>
    >> So a language standardization effort, independent of CPython,
    >> would be useful.

    >
    > A similar topic was recently discussed on the pypy-dev mailing list:
    >
    > http://codespeak.net/pipermail/pypy-dev/2010q3/006170.html
    >
    >
    > My interpretation is that the pypy devs are -0 on such a standardisation
    > effort, as it would give them less flexibility in moulding rpython for
    > their specific needs. Adding features to rpython that make it better as
    > a general-purpose programming language could actually make it *worse* as
    > a specialised language for building pypy.
    >
    > OTOH, there does seem to be a growing interest in using rpython as a
    > stand-alone language. I've used it for some small projects and it
    > worked out great.
    >
    > But the intersection of (people who want rpython as a general-purpose
    > language) and (people who have the ability to make that happen) seems to
    > be approximately zero at the moment...


    With Unladen Swallow looking like a failed IT project, a year
    behind schedule and not delivering anything like the promised
    performance, Google management may pull the plug on funding.
    Since there hasn't been a "quarterly release" in a year, that
    may already have effectively happened.

    All the schemes to speed up Python as defined by CPython seem to hit
    a wall on speed improvement. Some of the numeric benchmarks go faster
    on implementations that don't box all numbers, but 2x seems to be about
    as good as it gets, even with a JIT compiler.

    John Nagle
    John Nagle, Oct 12, 2010
    #5
  6. John Nagle

    alex23 Guest

    John Nagle <> wrote:
    >      All the schemes to speed up Python as defined by CPython seem to hit
    > a wall on speed improvement.  Some of the numeric benchmarks go faster
    > on implementations that don't box all numbers, but 2x seems to be about
    > as good as it gets, even with a JIT compiler.


    That hasn't been the case with PyPy for quite some time: http://goo.gl/dA7v
    (link to specific speed.pypy.org comparison).

    Comparing the latest PyPy+JIT to CPython 2.6.2, there are 6 tests that
    perform worse and 12 that perform better. Of the 12 improvements, 9
    are of an improvement of 3x or better, with one exceeding 12x and one
    15x. The django benchmark is 6x faster under PyPy+JIT than CPython.
    (The PyPy guys have done a fantastic job with http://speed.pypy.org to
    help show the gains that they're getting.)

    Don't be so quick to condemn PyPy, especially as they're _actively_
    working on this project, not just engaging in endless online rhetoric
    about how great life would be if only Python was faster. Achieving
    performance gains _without_ sacrificing the power of the language
    should be the ultimate goal.
    alex23, Oct 12, 2010
    #6
  7. John Nagle, 11.10.2010 22:01:
    > It may be time to standardize "RPython".
    >
    > There are at least three implementations of "RPython" variants - PyPy,
    > Shed Skin, and RPython for LLVM. The first two are up and running.


    The thing is, while RPython can be seen as a general purpose programming
    language, it lacks much of what larger applications would need. It doesn't
    have access to the CPython ecosystem, for one. Not at all in the case of
    Shedskin, and only part of it in the case of PyPy (ironically, excluding
    mostly those parts that were written for high performance). Don't know
    about the LLVM implementation, which seems to be far from mature anyway.

    If you implemented an RPython to CPython extension compiler, I'd expect
    that at least some people would use it. Lacking that, however, I don't see
    a major interest in writing code in RPython that doesn't have access to all
    of the CPython ecosystem. I'm not saying it can't be useful in certain
    cases, especially when you can be sure in advance that you won't need that
    ecosystem. But I don't really see the general purpose applicability of
    RPython to application development.

    Stefan
    Stefan Behnel, Oct 12, 2010
    #7
  8. Stefan Behnel, 12.10.2010 09:18:
    > If you implemented an RPython to CPython extension compiler, [...]


    BTW, if anyone wanted to do that, it might be a good idea to start with
    Cython, adapt its type inference layer and add the few missing Python
    language features (or pay the core developers to do it for you). That would
    likely be the fastest way to get this rolling.

    Stefan
    Stefan Behnel, Oct 12, 2010
    #8
  9. Hi John,

    John Nagle <nagle <at> animats.com> writes:

    > All attempts to make the dialect defined by CPython significantly
    > faster have failed. PyPy did not achieve much of a speed
    > improvement over CPython, and is sometimes slower.


    This is not true. While PyPy is indeed sometimes slower than CPython, there is a
    significant speedup for many synthetic and real-life benchmarks. PyPy with a JIT
    is up to 20 times faster than CPython for synthetic benchmarks like richards,
    but still something like 6 times faster for Django. It also beats Psyco in
    almost all benchmarks. For detailed measurements, please visit our benchmarking
    site: http://speed.pypy.org

    All these numbers are also only the current state, PyPy is still continuously
    improving.

    Cheers,

    Carl Friedrich
    Carl Friedrich Bolz, Oct 13, 2010
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jacek Generowicz

    Pre-standardizing hash_map & friends.

    Jacek Generowicz, Aug 26, 2003, in forum: C++
    Replies:
    0
    Views:
    359
    Jacek Generowicz
    Aug 26, 2003
  2. Simon Burton
    Replies:
    1
    Views:
    604
    Bjoern Schliessmann
    Jan 15, 2007
  3. ZeeGeek

    Standardizing XML

    ZeeGeek, Apr 15, 2007, in forum: Python
    Replies:
    7
    Views:
    244
    ZeeGeek
    Apr 15, 2007
  4. Brendan Miller

    what's the point of rpython?

    Brendan Miller, Jan 17, 2009, in forum: Python
    Replies:
    102
    Views:
    2,218
  5. sarvi

    PyPy and RPython

    sarvi, Sep 1, 2010, in forum: Python
    Replies:
    11
    Views:
    808
    sarvi
    Sep 3, 2010
Loading...

Share This Page