Python compilers?

Discussion in 'Python' started by Svein Ove Aas, May 17, 2004.

  1. Is anyone working on a python-to-native compiler?
    I'd be interested in taking a look.

    Come to think of it, is anyone working on a sexpr-enabled version of
    Python, or anything similar? I really miss my macros whenever I try to
    use it...
     
    Svein Ove Aas, May 17, 2004
    #1
    1. Advertising

  2. Svein Ove Aas

    Peter Hansen Guest

    Svein Ove Aas wrote:

    > Is anyone working on a python-to-native compiler?
    > I'd be interested in taking a look.
    >
    > Come to think of it, is anyone working on a sexpr-enabled version of
    > Python, or anything similar? I really miss my macros whenever I try to
    > use it...


    Lots of past threads on this, including this one:
    http://groups.google.com/groups?&th=8f7b4867334c3d07

    (Short answers: no, maybe, look at Psyco, PyPy, and others...)

    -Peter
     
    Peter Hansen, May 18, 2004
    #2
    1. Advertising

  3. Svein Ove Aas

    Guest

    Svein Ove Aas wrote:
    >
    > Is anyone working on a python-to-native compiler?
    > I'd be interested in taking a look.
    >
    > Come to think of it, is anyone working on a sexpr-enabled version of
    > Python, or anything similar? I really miss my macros whenever I try to
    > use it...


    I really wish there was a python-to-native compiler, a good one that
    would produce fairly fast execution.

    Mitchell Timin

    --
    "Many are stubborn in pursuit of the path they have chosen, few in
    pursuit of the goal." - Friedrich Nietzsche

    http://annevolve.sourceforge.net is what I'm into nowadays.
    Humans may write to me at this address: zenguy at shaw dot ca
     
    , May 18, 2004
    #3
  4. Svein Ove Aas wrote:
    > Is anyone working on a python-to-native compiler?
    > I'd be interested in taking a look.


    What are you trying to achieve?

    If it's faster code execution, the primary slowdown with a very
    high-level language like Python is caused by high-level data structures
    (introspection, everything being an object, etc.), not the code itself.
    A native compiler would still have to use high-level data structures to
    work with all Python code, so the speed increase wouldn't be very much.

    If it's ease of distribution you're looking for, I think distutils can
    make standalone programs on Windows, and most Linux distros have Python
    installed by default.

    If you just think a compiler would be cool (or would like to see how it
    would be done), check out Psyco, Pyrex, and probably some other
    projects. Pyrex even combats the speed issue by allowing native C types
    to be used in addition to Python high-level types.
     
    Leif K-Brooks, May 18, 2004
    #4
  5. Svein Ove Aas

    Yermat Guest

    Svein Ove Aas wrote:
    > Is anyone working on a python-to-native compiler?
    > I'd be interested in taking a look.
    >
    > Come to think of it, is anyone working on a sexpr-enabled version of
    > Python, or anything similar? I really miss my macros whenever I try to
    > use it...
    >


    look at starkiller http://www.python.org/pycon/dc2004/papers/1/

    The first step is to create a Type Inference system so there is star killer.
    Michael Salib said he is working on a C++ translation for python...

    --
    Yermat
     
    Yermat, May 18, 2004
    #5
  6. Leif K-Brooks wrote:

    > Svein Ove Aas wrote:
    >> Is anyone working on a python-to-native compiler?
    >> I'd be interested in taking a look.

    >
    > What are you trying to achieve?
    >
    > If it's faster code execution, the primary slowdown with a very
    > high-level language like Python is caused by high-level data structures
    > (introspection, everything being an object, etc.), not the code itself.
    > A native compiler would still have to use high-level data structures to
    > work with all Python code, so the speed increase wouldn't be very much.


    I'd like to point out the usual suspects: Lisp compilers.
    *They* somehow manage to get within 2x of C++ speed, so why can't Python?

    > If it's ease of distribution you're looking for, I think distutils can
    > make standalone programs on Windows, and most Linux distros have Python
    > installed by default.


    Nope.

    > If you just think a compiler would be cool (or would like to see how it
    > would be done), check out Psyco, Pyrex, and probably some other
    > projects. Pyrex even combats the speed issue by allowing native C types
    > to be used in addition to Python high-level types.


    Erk.
    Seems to me that you want 'smarter', not 'worse'. I can't take a language
    seriously if it says that 1/3 is 0.33333... .
     
    Svein Ove Aas, May 18, 2004
    #6
  7. Svein Ove Aas wrote:
    > Erk.
    > Seems to me that you want 'smarter', not 'worse'. I can't take a language
    > seriously if it says that 1/3 is 0.33333... .


    What is that? Are you arguing for an integer result, a fixed result or
    a rational result?

    Cheers,
    Brian
     
    Brian Quinlan, May 18, 2004
    #7
  8. Svein Ove Aas <> writes:

    > Leif K-Brooks wrote:
    >
    > > Svein Ove Aas wrote:
    > >> Is anyone working on a python-to-native compiler?
    > >> I'd be interested in taking a look.

    > >
    > > What are you trying to achieve?
    > >
    > > If it's faster code execution, the primary slowdown with a very
    > > high-level language like Python is caused by high-level data structures
    > > (introspection, everything being an object, etc.), not the code itself.
    > > A native compiler would still have to use high-level data structures to
    > > work with all Python code, so the speed increase wouldn't be very much.

    >
    > I'd like to point out the usual suspects: Lisp compilers.
    > *They* somehow manage to get within 2x of C++ speed, so why can't Python?


    Type declarations. Lots of effort by seriously smart cookies.

    We're working on the latter :)

    Cheers,
    mwh

    --
    > say-hi-to-the-flying-pink-elephants-for-me-ly y'rs,

    No way, the flying pink elephants are carrying MACHINE GUNS!
    Aiiee!! Time for a kinder, gentler hallucinogen...
    -- Barry Warsaw & Greg Ward, python-dev
     
    Michael Hudson, May 18, 2004
    #8
  9. Leif K-Brooks <> writes:

    > Svein Ove Aas wrote:
    > > Is anyone working on a python-to-native compiler?
    > > I'd be interested in taking a look.

    >
    > What are you trying to achieve?
    >
    > If it's faster code execution, the primary slowdown with a very
    > high-level language like Python is caused by high-level data
    > structures (introspection, everything being an object, etc.), not the
    > code itself. A native compiler would still have to use high-level data
    > structures to work with all Python code, so the speed increase
    > wouldn't be very much.


    Oh please !

    Native compilers for other languages just as dynamic as Python
    exist. These compilers manage to achieve very significant speed
    increases[*].

    Psyco is a native compiler, of sorts, for Python, and it manages to
    produce dramatic improvements (in the areas where it works).

    While it's true that often speed doesn't matter, and many of the
    criticisms levelled at Python for being too slow are completely
    unfounded in real world situations, this is no reason for Pythonistas
    to

    a) be happy about the fact that Python is slow,

    b) be convinced that Python _must_ be slow.

    The sooner "we" stop believing that Python's flexibility comes at the
    unavoidable cost of piss-poor runtime performance, and the sooner we
    accept that it would be useful to have a Python which maintains its
    flexibility but runs like a bat out of hell (ie, the sooner we stop
    making excuses for Pythons lack of speed), the sooner we will get one.

    Fortunately there is already a bunch of people who understand this,
    and is trying to do something about it.


    [*] A prime example is the Common Lisp implementation CMUCL. Ironically
    enough, CMUCL's compiler is called ... Python.
     
    Jacek Generowicz, May 18, 2004
    #9
  10. In article <>, <> wrote:
    .
    .
    .
    >I really wish there was a python-to-native compiler, a good one that
    >would produce fairly fast execution.

    .
    .
    .
    Maybe there is. There's enough ambiguity in "python-to-native
    compiler" that it's possible what you (think you) want already
    exists. See Leif K-Brooks's follow-up, especially.
    --

    Cameron Laird <>
    Business: http://www.Phaseit.net
     
    Cameron Laird, May 18, 2004
    #10
  11. Am Dienstag, 18. Mai 2004 13:41 schrieb Jacek Generowicz:
    > Native compilers for other languages just as dynamic as Python
    > exist. These compilers manage to achieve very significant speed
    > increases[*].


    You are again refering to LISP as an example of a dynamic language which when
    compiled gives huge speed increases. This is true in some respect, in others
    it isn't. LISP has the advantage that type-inference may be used throughout
    the program to create one version of each function, which can then be
    compiled. Of course it still has to call into runtime functions to do the
    high-level work, but there is actually only one representation of each
    finished LISP program, and only one set and one proper order of
    runtime-functions to call.

    In Python this isn't true. Python, instead of LISP, is "completely" dynamic,
    meaning that it's pretty impossible to do type-inference for each function
    that is called (even checking types isn't possible). E.g. how do you expect
    type-inference to work with the pickle module? string -> something/Error
    would be the best description what pickle does. For the function which calls
    pickle, do you want to create versions for each possible output of Pickle?
    Which outputs of Pickle are possible? (depends on the loaded modules, which
    can be loaded at runtime) There is no (sane) way to create machine-code which
    calls into the appropriate (low-level) Python-runtime functions (such as
    Py_List*, Py_Dict*, etc.) for such a method, at least not at compile-time.

    At runtime, this is possible. See what psyco does. There's a nice presentation
    on the psyco-website which explains what it does, and I guess you'll
    understand when you see that why writing a "compiler" for Python is pretty
    impossible.

    Heiko.
     
    Heiko Wundram, May 18, 2004
    #11
  12. On 2004-05-18, Svein Ove Aas <> wrote:

    > Seems to me that you want 'smarter', not 'worse'. I can't take a language
    > seriously if it says that 1/3 is 0.33333... .


    Dude, didn't you take high-school math? 1/3 _is_ 0.33333...

    --
    Grant Edwards grante Yow! My forehead feels
    at like a PACKAGE of moist
    visi.com CRANBERRIES in a remote
    FRENCH OUTPOST!!
     
    Grant Edwards, May 18, 2004
    #12
  13. Brian Quinlan wrote:

    > Svein Ove Aas wrote:
    >> Erk.
    >> Seems to me that you want 'smarter', not 'worse'. I can't take a
    >> language seriously if it says that 1/3 is 0.33333... .

    >
    > What is that? Are you arguing for an integer result, a fixed result or
    > a rational result?


    A rational result, of course. If I'm okay with losing precision, I'll
    *tell* the language I'm okay with losing precision.
     
    Svein Ove Aas, May 18, 2004
    #13
  14. Grant Edwards <> writes:

    > Dude, didn't you take high-school math? 1/3 _is_ 0.33333...


    No, because at some point you will stop writing 3's, either out of
    boredom, exhaustion or because you need to pee. At that instant, you
    introduce a rounding error, making 3 * 1/3 = 0.99999999999... instead
    of 1.0
     
    Tor Iver Wilhelmsen, May 18, 2004
    #14
  15. Svein Ove Aas

    Guest

    Leif K-Brooks wrote:
    >
    > Svein Ove Aas wrote:
    > > Is anyone working on a python-to-native compiler?
    > > I'd be interested in taking a look.

    >
    > What are you trying to achieve?
    >
    > If it's faster code execution, the primary slowdown with a very
    > high-level language like Python is caused by high-level data structures
    > (introspection, everything being an object, etc.), not the code itself.
    > A native compiler would still have to use high-level data structures to
    > work with all Python code, so the speed increase wouldn't be very much.


    Yes, fast execution. I have been using C. In my applications there is
    a population of "chromosomes" which are arrays of floats, about 2 to 5 k
    in length. Then there are subroutines which operate on a chromosome
    using pointers. For example, the "crossover" routine uses two pointers
    to swap portions of two chromosomes. My software sometimes runs for
    hours, perform many millions of operations like these. Clearly, speed
    of execution is of dramatic importance.

    A related problem is that it seems to be a big deal to call C routines
    from Python. I have not actually tried it, because when I read about
    how its done I was not able to understand it.

    Mitchell Timin

    --
    "Many are stubborn in pursuit of the path they have chosen, few in
    pursuit of the goal." - Friedrich Nietzsche

    http://annevolve.sourceforge.net is what I'm into nowadays.
    Humans may write to me at this address: zenguy at shaw dot ca
     
    , May 18, 2004
    #15
  16. Heiko Wundram wrote:

    > Am Dienstag, 18. Mai 2004 13:41 schrieb Jacek Generowicz:
    >> Native compilers for other languages just as dynamic as Python
    >> exist. These compilers manage to achieve very significant speed
    >> increases[*].

    >
    > You are again refering to LISP as an example of a dynamic language which
    > when compiled gives huge speed increases. This is true in some respect,
    > in others it isn't. LISP has the advantage that type-inference may be
    > used throughout the program to create one version of each function,
    > which can then be compiled. Of course it still has to call into runtime
    > functions to do the high-level work, but there is actually only one
    > representation of each finished LISP program, and only one set and one
    > proper order of runtime-functions to call.
    >
    > In Python this isn't true. Python, instead of LISP, is "completely"
    > dynamic, meaning that it's pretty impossible to do type-inference for
    > each function that is called (even checking types isn't possible). E.g.
    > how do you expect type-inference to work with the pickle module? string
    > -> something/Error would be the best description what pickle does. For
    > the function which calls pickle, do you want to create versions for each
    > possible output of Pickle? Which outputs of Pickle are possible?
    > (depends on the loaded modules, which can be loaded at runtime) There is
    > no (sane) way to create machine-code which calls into the appropriate
    > (low-level) Python-runtime functions (such as Py_List*, Py_Dict*, etc.)
    > for such a method, at least not at compile-time.


    I suppose that's true; pickle is an exception, and the compiler would pick
    that up. (The Lisp (print) function is approximately equivalent, at least
    when *print-readably* is true.)

    What you're claiming, though, is that it's possible to write Python code
    that can't easily be translated to equivalent Lisp code. Can you give an
    example?
     
    Svein Ove Aas, May 18, 2004
    #16
  17. * Tor Iver Wilhelmsen (2004-05-18 17:26 +0100)
    > Grant Edwards <> writes:
    >
    >> Dude, didn't you take high-school math? 1/3 _is_ 0.33333...

    >
    > No, because at some point you will stop writing 3's, either out of
    > boredom, exhaustion or because you need to pee. At that instant, you
    > introduce a rounding error, making 3 * 1/3 = 0.99999999999... instead
    > of 1.0


    Must have been a long time since you went to school... 1/3 is
    /exactly/ 0.3...: http://mathworld.wolfram.com/RepeatingDecimal.html

    Thorsten
     
    Thorsten Kampe, May 18, 2004
    #17
  18. Svein Ove Aas

    Yermat Guest

    Heiko Wundram wrote:
    > [...]
    >
    > In Python this isn't true. Python, instead of LISP, is "completely" dynamic,
    > meaning that it's pretty impossible to do type-inference for each function
    > that is called (even checking types isn't possible). E.g. how do you expect
    > type-inference to work with the pickle module? string -> something/Error
    > would be the best description what pickle does. For the function which calls
    > pickle, do you want to create versions for each possible output of Pickle?
    > Which outputs of Pickle are possible? (depends on the loaded modules, which
    > can be loaded at runtime) There is no (sane) way to create machine-code which
    > calls into the appropriate (low-level) Python-runtime functions (such as
    > Py_List*, Py_Dict*, etc.) for such a method, at least not at compile-time.
    >
    > At runtime, this is possible. See what psyco does. There's a nice presentation
    > on the psyco-website which explains what it does, and I guess you'll
    > understand when you see that why writing a "compiler" for Python is pretty
    > impossible.


    That is here where you are wrong !
    It can be known at compile time if you know every modules that will be
    imported. That is called "closed environment".

    If it is the case, you would be able to compile a program even if you
    would not be allowed to do incremental compilation... That just mean
    that you will need to recompile everything each time you modified something.

    The demonstration is quite easy:
    In a "closed environment" there is a finite number of classes. So you
    just have to create as much specialized functions as classes. Then where
    you can infere type, call directly the good function. Elsewhere, just
    call a runtime dispatcher. In fact, it is already used in some languages
    like Eiffel with certain optimizations.

    The problem is that many python programs are not "closed" at
    compile-time, ie they import or eval stuff only known at run-time.

    impossible n'est pas fran├žais ;-)

    --
    Yermat
     
    Yermat, May 18, 2004
    #18
  19. Svein Ove Aas

    Roger Binns Guest

    wrote:
    > A related problem is that it seems to be a big deal to call C routines
    > from Python. I have not actually tried it, because when I read about
    > how its done I was not able to understand it.


    Use SWIG. If your C header file is well written, you may not need to
    give any additional information to SWIG.

    Roger
     
    Roger Binns, May 18, 2004
    #19
  20. Svein Ove Aas

    simo Guest

    Leif K-Brooks <> wrote:

    > > Is anyone working on a python-to-native compiler?
    > > I'd be interested in taking a look.


    > If it's faster code execution, the primary slowdown with a very
    > high-level language like Python is caused by high-level data structures
    > (introspection, everything being an object, etc.), not the code itself.
    > A native compiler would still have to use high-level data structures to
    > work with all Python code, so the speed increase wouldn't be very much.


    I think the main slowdown most people would see is the startup time -
    the interpreter takes way too long to start (why the company where I
    work won't replace Perl/PHP with Python).

    > If it's ease of distribution you're looking for, I think distutils can
    > make standalone programs on Windows, and most Linux distros have Python
    > installed by default.


    Oh that one is funny - do most distro's have PyQT and wxPython (and
    their supporting Qt/GKT libs) installed by default, oh and which ones
    come with 2.3 by default?

    > If you just think a compiler would be cool (or would like to see how it
    > would be done), check out Psyco, Pyrex, and probably some other
    > projects. Pyrex even combats the speed issue by allowing native C types
    > to be used in addition to Python high-level types.


    I thought Pyrex was a hybrid of C and Python (like Jython/Java) not
    actually a Python-to-C convertor? And Pysco is just a different VM
    isn't it?

    I could really push Python where I work if there was a native
    compiler, my company uses C/C++/Java/Qt and were looking at QSA as a
    way to allow the user to script things, but as all of our products
    integrate with our software protection system, we can't be
    distributing source or easily decompiled bytecode!

    We could replace the whole lot with PyQt and use an embedded Python
    interpreter for the scripting! Ah the frustration :-(
     
    simo, May 18, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Julian Hershel

    Command line compilers

    Julian Hershel, Apr 29, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    495
    Julian Hershel
    Apr 30, 2004
  2. Rod Stephenson

    Embedding python with mingw compilers

    Rod Stephenson, Apr 3, 2004, in forum: Python
    Replies:
    0
    Views:
    457
    Rod Stephenson
    Apr 3, 2004
  3. project2501
    Replies:
    4
    Views:
    339
    Cameron Laird
    Apr 26, 2004
  4. geletine

    commercial c compilers vs free c compilers

    geletine, Jul 2, 2006, in forum: C Programming
    Replies:
    33
    Views:
    1,318
  5. John Ladasky
    Replies:
    19
    Views:
    239
    Stefan Behnel
    Apr 6, 2013
Loading...

Share This Page