pyvm -- faster python

Discussion in 'Python' started by Stelios Xanthakis, May 8, 2005.

  1. Hi.

    pyvm is a program which can run python 2.4 bytecode (the .pyc files).
    A demo pre-release is available at:
    http://students.ceid.upatras.gr/~sxanth/pyvm/


    Facts about pyvm:
    - It's FAST. According to the "cooked-bench" benchmark suite it finishes
    in 55% of the time python takes;)
    - It's SMALL. Currently the source code is under 15k lines with the
    builtin modules.
    - It's new. Uses no code from CPython.
    - It's incomplete. Not even near the stability and quality of python.
    It needs A LOT of work before it can be compared to CPython.
    Moreover, at the time it lacks many things like closures, long numbers
    new style classes, etc.
    - It's incompatible with CPython. Not all programs run.
    - The C API is incompatible. You can't run C modules (a thin wrapper to
    make pyvm appear as libpython *could* be possible but not a goal
    AFAIC)
    - The demo is an x86/linux binary only. You shouldn't trust binaries,
    run it in a chrooted environment not as root!

    Hope it works!

    Cheers,

    Stelios
     
    Stelios Xanthakis, May 8, 2005
    #1
    1. Advertising

  2. Stelios Xanthakis

    Kay Schluehr Guest

    Stelios Xanthakis wrote:
    > Hi.
    >
    > pyvm is a program which can run python 2.4 bytecode (the .pyc files).
    > A demo pre-release is available at:
    > http://students.ceid.upatras.gr/~sxanth/pyvm/
    >
    >
    > Facts about pyvm:
    > - It's FAST. According to the "cooked-bench" benchmark suite it

    finishes
    > in 55% of the time python takes;)
    > - It's SMALL. Currently the source code is under 15k lines with the
    > builtin modules.
    > - It's new. Uses no code from CPython.
    > - It's incomplete. Not even near the stability and quality of python.
    > It needs A LOT of work before it can be compared to CPython.
    > Moreover, at the time it lacks many things like closures, long

    numbers
    > new style classes, etc.
    > - It's incompatible with CPython. Not all programs run.
    > - The C API is incompatible. You can't run C modules (a thin wrapper

    to
    > make pyvm appear as libpython *could* be possible but not a goal
    > AFAIC)
    > - The demo is an x86/linux binary only. You shouldn't trust binaries,
    > run it in a chrooted environment not as root!
    >
    > Hope it works!
    >
    > Cheers,
    >
    > Stelios


    Hi Stelios,

    could You tell us a bit more about Your motivation to create an
    alternative C-Python interpreter? There is AFAIK no such ambitious
    project that has ever survived. The last one I remember died shortly
    after it was born:

    http://www.python.org/pycon/papers/pymite/

    This is sad because it is still challenging to offer a tiny interpreter
    of a dynamic language for glue code/RAD on tiny hardware. A lot of
    effort was spent to provide Java for microcontrollers especially for
    SmartCards. I think a lot of people would show interest in Your project
    if it gets somehow focussed and does not seem to be redundant.

    Ciao,
    Kay
     
    Kay Schluehr, May 8, 2005
    #2
    1. Advertising

  3. Stelios Xanthakis

    Guest

    I've seen the benchmarks, they look quite interesting.

    This project is probably a LOT of work; maybe people can tell us about
    such efforts *before* doing so much work, so we can discuss it, and
    avoid wasting time.

    Maybe you can explain us why it is so fast, and/or maybe you can work
    with the other developers to improve the speed of the normal CPython,
    this can require equal or less work for you, and it can produce more
    long-lasting results/consequences for your work.

    Bye,
    Bearophile
     
    , May 8, 2005
    #3
  4. Stelios Xanthakis

    Roger Binns Guest

    > could You tell us a bit more about Your motivation to create an
    > alternative C-Python interpreter?


    I'd also be curious to know if the performance gains would remain
    once it gets fleshed out with things like closures, long numbers,
    new style classes and a C library compatibility shim.

    Roger
     
    Roger Binns, May 8, 2005
    #4
  5. Stelios Xanthakis

    Peter Hansen Guest

    Kay Schluehr wrote:
    > Stelios Xanthakis wrote:
    >>pyvm is a program which can run python 2.4 bytecode (the .pyc files).
    >>A demo pre-release is available at:
    >> http://students.ceid.upatras.gr/~sxanth/pyvm/

    >
    > could You tell us a bit more about Your motivation to create an
    > alternative C-Python interpreter? There is AFAIK no such ambitious
    > project that has ever survived. ..


    If you check the URL more closely, you'll notice that this is a
    university site.

    It seem likely this being done at least partly as a school project,
    which is certainly motivation enough for such a thing, *even* if it
    doesn't survive (since it will pay for itself in learning).

    -Peter
     
    Peter Hansen, May 8, 2005
    #5
  6. Stelios Xanthakis

    Paul Rubin Guest

    Stelios Xanthakis <> writes:
    > - The demo is an x86/linux binary only. You shouldn't trust binaries,
    > run it in a chrooted environment not as root!


    Are you going to release the source? If not, it's a lot less interesting.
     
    Paul Rubin, May 8, 2005
    #6
  7. Stelios Xanthakis

    djw Guest

    Paul Rubin wrote:
    > Stelios Xanthakis <> writes:
    >
    >>- The demo is an x86/linux binary only. You shouldn't trust binaries,
    >> run it in a chrooted environment not as root!

    >
    >
    > Are you going to release the source? If not, it's a lot less interesting.


    From the website:

    "...the source will be released when it becomes more complete and stable."
     
    djw, May 9, 2005
    #7
  8. djw <> writes:

    > Paul Rubin wrote:
    >> Stelios Xanthakis <> writes:
    >>
    >>>- The demo is an x86/linux binary only. You shouldn't trust binaries,
    >>> run it in a chrooted environment not as root!

    >>
    >>
    >> Are you going to release the source? If not, it's a lot less interesting.

    >
    > From the website:
    >
    > "...the source will be released when it becomes more complete and stable."


    If he releases the source now, it'll probably become more complete and
    stable sooner...

    Remember, release early, release often!

    Dan
     
    Dan Christensen, May 10, 2005
    #8
  9. Kay Schluehr wrote:
    >
    > could You tell us a bit more about Your motivation to create an
    > alternative C-Python interpreter? There is AFAIK no such ambitious
    > project that has ever survived. The last one I remember died shortly
    > after it was born:
    >


    The motivation is that I just needed some bytecode engine. I deciced
    to go with python bytecode, because well, IMO it's the best!
    Eventually it turned out what it is.

    I'm afraid this may end up dead before unborn too.
    So it depends what people want. If nobody cares, I just learned python
    the hard way and the project does to the abandonware cemetary.


    Stelios
     
    Stelios Xanthakis, May 10, 2005
    #9
  10. wrote:

    > This project is probably a LOT of work; maybe people can tell us about
    > such efforts *before* doing so much work, so we can discuss it, and
    > avoid wasting time.
    >


    It is a lot of work indeed.
    Usually, when people announce "we shall create X", it doesn't happen.
    And you don't know if you can do it at all *before*.


    > Maybe you can explain us why it is so fast, and/or maybe you can work
    > with the other developers to improve the speed of the normal CPython,
    > this can require equal or less work for you, and it can produce more
    > long-lasting results/consequences for your work.
    >


    The reason is that it's built from the scratch.
    Guido would disagree with that, see py-dev thread:

    http://www.mail-archive.com//msg01225.html


    There *are* alternative ways to do some things in the vm and Joel
    is simply wrong:)
    Could this be .... Python 3000!!!? :)


    Stelios
     
    Stelios Xanthakis, May 10, 2005
    #10
  11. Roger Binns wrote:

    >>could You tell us a bit more about Your motivation to create an
    >>alternative C-Python interpreter?

    >
    >
    > I'd also be curious to know if the performance gains would remain
    > once it gets fleshed out with things like closures, long numbers,
    > new style classes and a C library compatibility shim.
    >


    I guess it will. There are several places with /*XXX:speed up*/
    and a couple more ideas to gain exrta speed. On the other hand
    pyvm does not check for Bad Internal Calls and other checks Cpython
    does to be OK in all cases (for example if __cmp__ modifies a list
    while it's being sorted, in pyvm you *can* get a crash).

    There is no plan to implement closures; unless there is a way to
    turn closures to default arguments with bytecode hacking. But yes,
    closures are interesting, especially in generator expressions and
    lambdas.

    I am not very interested on C compatibility. Once the source is
    open people can implement it if they want but I'd rather go with
    an advanced ctypes module which will make loading dynamic libraries
    a matter of python code.


    Stelios
     
    Stelios Xanthakis, May 10, 2005
    #11
  12. [Stelios Xanthakis]

    > I'm afraid this may end up dead before unborn too. So it depends what
    > people want. If nobody cares, [...]


    People might not care so much about what could be done about your
    project, unless you give them proper and complete means for evaluating
    the state of affairs. Your project is very likely to die if you keep
    your sources closed, and yourself loose interest in the project.

    Opening your sources is no guarantee either that the community will
    adopt your project. But this might give your project a better chance.

    --
    Fran├žois Pinard http://pinard.progiciels-bpi.ca
     
    =?iso-8859-1?Q?Fran=E7ois?= Pinard, May 10, 2005
    #12
  13. Stelios Xanthakis

    Terry Reedy Guest

    "Stelios Xanthakis" <> wrote in message
    news:...
    >> Maybe you can explain us why it is so fast, and/or maybe you can work
    >> with the other developers to improve the speed of the normal CPython,
    >> this can require equal or less work for you, and it can produce more
    >> long-lasting results/consequences for your work.
    >>

    >
    > The reason is that it's built from the scratch.
    > Guido would disagree with that, see py-dev thread:


    Guido, like me, believes in being correct before being fast.

    > http://www.mail-archive.com//msg01225.html
    >
    >
    > There *are* alternative ways to do some things in the vm and Joel
    > is simply wrong:)


    The smiley doesn't negate your silliness. Joel explicitly talked about
    complex, mature, debugged, in-use systems, with actual or potential
    competitive replacements, not laboratory development toys and prototypes.
    Three years between releases *is* a long time. And the Python codebase
    *does* embody a lot of hard-won knowledge, hardly available elsewhere, that
    should not be tossed. A particular example is how to get certain things to
    work across numerous platforms in the face of ambuiguity and 'not defined'
    behavior in the C standard and implementations.

    Its possible that some future reference Python will be based on a new,
    built-from-scratch core developed *in parallel* with current Python. But
    that will only happen after it has been tested on a hundred different
    platforms with a hundred different applications and packages (as happens,
    more or less, with each new CPython release).

    Terry J. Reedy
     
    Terry Reedy, May 10, 2005
    #13
  14. Stelios Xanthakis

    Roger Binns Guest

    > I am not very interested on C compatibility.

    That will rule out all the gui frameworks, SSL, cryptography
    and numerous other packages. Have a look at what happened to
    Prothon. What ultimately killed it was the problem of having
    a decent library. You don't have to make the C library
    compatibility totally high performance, but having any form of
    it there will make adoption by others easier.

    Roger
     
    Roger Binns, May 11, 2005
    #14
  15. Stelios Xanthakis

    Paul Rubin Guest

    "Roger Binns" <> writes:
    > That will rule out all the gui frameworks, SSL, cryptography
    > and numerous other packages. Have a look at what happened to
    > Prothon.


    I think it would be enough to retarget SWIG.

    > What ultimately killed it was the problem of having a decent
    > library.


    Nah, I think people just lost interest. Otherwise, why isn't Jython
    dead?
     
    Paul Rubin, May 11, 2005
    #15
  16. Stelios Xanthakis

    Roger Binns Guest

    "Paul Rubin" <http://> wrote in message news:...
    > "Roger Binns" <> writes:
    >> That will rule out all the gui frameworks, SSL, cryptography
    >> and numerous other packages. Have a look at what happened to
    >> Prothon.

    >
    > I think it would be enough to retarget SWIG.


    That is also a lot of work, and a lot of toolkits that use Swig
    for Python bindings are very strongly tied to the Swig-Python
    internals. For example I doubt there are many where you could
    tell Swig to output Java and it would just work. (I know none
    of mine would work, nor would wxPython).

    >> What ultimately killed it was the problem of having a decent
    >> library.

    >
    > Nah, I think people just lost interest. Otherwise, why isn't Jython
    > dead?


    Err, you proved my point! Prothon was fine at the VM level. The
    author couldn't figure out how to get a decent sized "standard"
    library, and ultimately ended up abandoning his VM for .Net since
    that gets you a large standard library. Jython also gets a large
    standard library from the Java ones.

    Roger
     
    Roger Binns, May 11, 2005
    #16
  17. Stelios Xanthakis

    Peter Hansen Guest

    Paul Rubin wrote:
    > "Roger Binns" <> writes:
    >
    >>That will rule out all the gui frameworks, SSL, cryptography
    >>and numerous other packages. Have a look at what happened to
    >>Prothon.

    >
    >
    > I think it would be enough to retarget SWIG.
    >
    >
    >>What ultimately killed it was the problem of having a decent
    >>library.

    >
    >
    > Nah, I think people just lost interest. Otherwise, why isn't Jython
    > dead?


    I thought Jython had full access to the entire library of standard Java,
    plus at least the Python parts of the Python standard library...

    -Peter
     
    Peter Hansen, May 11, 2005
    #17
  18. Stelios Xanthakis

    Paul Rubin Guest

    "Roger Binns" <> writes:
    > Err, you proved my point! Prothon was fine at the VM level. The
    > author couldn't figure out how to get a decent sized "standard"
    > library, and ultimately ended up abandoning his VM for .Net since
    > that gets you a large standard library. Jython also gets a large
    > standard library from the Java ones.


    What kind of stuff is in the existing Python C library that couldn't
    be reimplemented or retargeted pretty easily? Most of it is either
    wrappers for standard C functions (system calls), stuff that shouldn't
    be in C in the first place (the Python compiler and interpreter), etc.

    I hope that PyPy will replace CPython once it's solid enough. Trying
    to stay backwards compatible with the legacy C API doesn't seem to me
    to be that important a goal. Redoing the library may take more work
    than the Prothon guy was willing to do for Prothon, but PyPy has more
    community interest and maybe can attract more resources.
     
    Paul Rubin, May 11, 2005
    #18
  19. Stelios Xanthakis

    Roger Binns Guest

    "Paul Rubin" <http://> wrote in message news:...
    > "Roger Binns" <> writes:
    > What kind of stuff is in the existing Python C library that couldn't
    > be reimplemented or retargeted pretty easily? Most of it is either
    > wrappers for standard C functions (system calls), stuff that shouldn't
    > be in C in the first place (the Python compiler and interpreter), etc.


    Some examples are gui toolkits (eg wxPython), SSL (eg M2Crypto, pyopenssl)
    and database (pysqlite, APSW). These aren't in the shipped with Python
    library but are widely used.

    > I hope that PyPy will replace CPython once it's solid enough. Trying
    > to stay backwards compatible with the legacy C API doesn't seem to me
    > to be that important a goal. Redoing the library may take more work
    > than the Prothon guy was willing to do for Prothon, but PyPy has more
    > community interest and maybe can attract more resources.


    You don't have to stay backwards compatible. It is best to provide
    some sort of way of using the old extensions even if it is suboptimal
    (eg some sort of mapping shim). The extensions can later be improved,
    but if there is no simple way of recompiling and running then it will
    take a long time for all the extensions to get ported.

    I already get burnt out on the matrix of CPython versions and different
    platforms. Adding another interpretter would make life even harder
    for extension authors.

    Roger
     
    Roger Binns, May 11, 2005
    #19
  20. Stelios Xanthakis

    Kay Schluehr Guest

    Paul Rubin wrote:
    > "Roger Binns" <> writes:
    > > Err, you proved my point! Prothon was fine at the VM level. The
    > > author couldn't figure out how to get a decent sized "standard"
    > > library, and ultimately ended up abandoning his VM for .Net since
    > > that gets you a large standard library. Jython also gets a large
    > > standard library from the Java ones.

    >
    > What kind of stuff is in the existing Python C library that couldn't
    > be reimplemented or retargeted pretty easily? Most of it is either
    > wrappers for standard C functions (system calls), stuff that

    shouldn't
    > be in C in the first place (the Python compiler and interpreter),

    etc.

    Delete the "standard" and You still obtain huge librarys for .Net, Java
    and Python. I also regret that Prothon starved in infancy but it might
    be exeggerated to demand that each language designer or one of his
    apostels should manage a huge community that enjoys doing redundant
    stuff like writing Tk-bindings, regexp-engines and all that.

    > I hope that PyPy will replace CPython once it's solid enough. Trying
    > to stay backwards compatible with the legacy C API doesn't seem to me
    > to be that important a goal. Redoing the library may take more work
    > than the Prothon guy was willing to do for Prothon, but PyPy has more
    > community interest and maybe can attract more resources.


    PyPy is a beast on it's own and provides interesting new ideas to
    Python as well as runtime design in general. But I'm not quite shure
    that it will stay in the boundarys of the current Python community
    process and will subordinate to Guidos will. As a Marxist would say:
    who controls the runtime also controls the language. Sooner or later
    Python will go the LISP way of having a standardized "Common-Python" (
    std_objectspace) and a number of dialects and DSLs running in their own
    derived object spaces. Maybe Python 3000 is an illusion and will fade
    away like a Fata Morgana the closer we seem come.

    Kay
     
    Kay Schluehr, May 11, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Armin Rigo

    Python is faster than C

    Armin Rigo, Apr 3, 2004, in forum: Python
    Replies:
    36
    Views:
    890
    Stephen Horne
    Apr 6, 2004
  2. stelios xanthakis

    Looking for Benchmarklets to improve pyvm

    stelios xanthakis, Apr 1, 2005, in forum: Python
    Replies:
    2
    Views:
    306
    stelios xanthakis
    Apr 2, 2005
  3. stelios xanthakis

    pyvm source code

    stelios xanthakis, Dec 30, 2005, in forum: Python
    Replies:
    2
    Views:
    346
    EleSSaR^
    Dec 30, 2005
  4. Python Maniac
    Replies:
    24
    Views:
    803
    Matt McCredie
    Sep 23, 2007
  5. Vladimir G. Ivanovic

    Searching for pyvm

    Vladimir G. Ivanovic, May 25, 2009, in forum: Python
    Replies:
    0
    Views:
    280
    Vladimir G. Ivanovic
    May 25, 2009
Loading...

Share This Page