Why not a Python compiler?

[Santiago Romero]

( Surely if this question has been asked for a zillion of times... )
( and sorry for my english! )

I'm impressed with python. I'm very happy with the language and I
find Python+Pygame a very powerful and productive way of writing 2D
games. I'm not, at this moment, worried about execution speed of the
small game I'm working on (it runs at full 60 fps even in an old AMD-
K6 450 Laptop computer), but I continue asking me the same question:

Why not a Python COMPILER?

It would be very nice to be able to output Linux, MAC or Windows
binaries of compiled (not bytecompiled) code. It would run faster, it
will be smaller in size (I think) and it will be easy to distribute to
people not having python installed. Yes, I know about py2exe, but I'm
not sure if that's the right aproach.

So, what's wrong with compiling python?

Maybe is not possible due to nature of the language? Is just a
decision?

What do you think about this?

 

[cokofreedom]

( Surely if this question has been asked for a zillion of times... )
( and sorry for my english! )

I'm impressed with python. I'm very happy with the language and I
find Python+Pygame a very powerful and productive way of writing 2D
games. I'm not, at this moment, worried about execution speed of the
small game I'm working on (it runs at full 60 fps even in an old AMD-
K6 450 Laptop computer), but I continue asking me the same question:

Why not a Python COMPILER?

It would be very nice to be able to output Linux, MAC or Windows
binaries of compiled (not bytecompiled) code. It would run faster, it
will be smaller in size (I think) and it will be easy to distribute to
people not having python installed. Yes, I know about py2exe, but I'm
not sure if that's the right aproach.

So, what's wrong with compiling python?

Maybe is not possible due to nature of the language? Is just a
decision?

What do you think about this?

I don't know the exact details but I think the issue is the dynamic
nature of Python makes it impossible to correctly store the various
types and changes into compiled code. Someone else will probably be
able to provide a good reason as to why it isn't very feasible, nor a
good idea. If you want to speed up your python look at Psyco.
http://psyco.sourceforge.net/

 

[Jeroen Ruigrok van der Werven]

Why not a Python COMPILER?

A lot of things within Python are very run-time dependent so creating a
compiler is not trivial work.

There are, however, endeavours underway like shed skin:

http://code.google.com/p/shedskin/

This provides a Python to C++ compiler, but it enforces some restrictions on
Python's code.

 

[Kay Schluehr]

( Surely if this question has been asked for a zillion of times... )

Sure. You can access comp.lang.python via google.google.com. It has a
search function.

 

[Dustan]

Sure. You can access comp.lang.python via
groups

.google.com.
It has a
search function.

 

[Bruno Desthuilliers]

Santiago Romero a écrit :

( Surely if this question has been asked for a zillion of times... )

Why not checking this by yourself ? google is down ?-)

I'm impressed with python. I'm very happy with the language and I
find Python+Pygame a very powerful and productive way of writing 2D
games. I'm not, at this moment, worried about execution speed of the
small game I'm working on (it runs at full 60 fps even in an old AMD-
K6 450 Laptop computer), but I continue asking me the same question:

Why not a Python COMPILER?
http://docs.python.org/lib/module-compiler.html

It would be very nice to be able to output Linux, MAC or Windows
binaries of compiled (not bytecompiled) code.

Ah, sorry.

It would run faster, it
will be smaller in size (I think)

These two points are very questionnable given Python's dynamicity. IIRC,
some guru already made the point, a couple or more years ago, that it
would be a lot of effort and buy *very* few wrt/ execution speed since
everything still have to happens at runtime. IOW, a JIT compiler seems a
far better solution, but this (so far) still impose some (serious)
limitations to be of any help.

 

[Ripter001]

Why not a Python COMPILER?

What about a Python JIT hardware chip, so the CPU doesn't have to
translate. Although it seems to me that with today's dual and quad
core processors that this might be a mute point because you could just
use one of the cores.

 

[Steve Holden]

What about a Python JIT hardware chip, so the CPU doesn't have to
translate. Although it seems to me that with today's dual and quad
core processors that this might be a mute point because you could just
use one of the cores.

What about a chip that reads your mind and does what you want it to?

I am sure that would be popular with all the frustrated computer users
there are in the world.

regards
Steve

 

[Jeff Schwab]

What about a chip that reads your mind and does what you want it to?
+1!

I am sure that would be popular with all the frustrated computer users
there are in the world.

I'll take two. (Chips, not frustrated users).

 

[Jeff Schwab]

What about a chip that reads your mind and does what you want it to?
+1!

I am sure that would be popular with all the frustrated computer users
there are in the world.

I'll take two. (Chips, not frustrated users).

 

[Jeff]

I'm surprised no one has mentioned Pyrex. It can be used to write
stand-alone C programs using near-Python syntax:

Pyrex: http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/
Stand-alone how-to: http://www.freenet.org.nz/python/embeddingpyrex/
Pyrex how-to: http://ldots.org/pyrex-guide/,
http://www.artfulcode.net/articles/extending-python-pyrex/ (shameless
plug, I know)

 

[Ripter001]

What about a chip that reads your mind and does what you want it to?

I am sure that would be popular with all the frustrated computer users
there are in the world.

regards
Steve

I'm not sure where that came from. Microsoft had talked about a
hardware JIT for .NET a few years back. I was just wondering if anyone
had thought of it for python or anything like that.

 

[Steven D'Aprano]

I'm not sure where that came from. Microsoft had talked about a hardware
JIT for .NET a few years back. I was just wondering if anyone had
thought of it for python or anything like that.

Okay, you know how hard it is to create a software JIT compiler for a
language as dynamic as Python? It's REALLY HARD, which is why it hasn't
already been done[1]. Now you want that done in *hardware*, which is much
harder. Who's going to spend the money on R&D?

I'm sure there are thousands of l33t hax0rs out there who have thought
"Wouldn't it be c00l if there was a chip I could put into my PC to make
Python run a million times faster!!!". When I was younger and more
stupi^W innocent I had a mad fegairy for Forth and Lisp chips, but their
lack of financial viability and their unfortunate habit of actually being
*slower* than running the language in software put a big dent in the
idea. As general purpose CPUs got faster, the idea of making a specialist
language chip is now pretty much dead. Even if you could find a niche
market prepared to pay for it, "people who run Python programs" is
probably not that market.






[1] Apart from some specializing compilers like Pysco
http://psyco.sourceforge.net/introduction.html

 

[Steve Holden]

I'm not sure where that came from. Microsoft had talked about a hardware
JIT for .NET a few years back. I was just wondering if anyone had
thought of it for python or anything like that.

Okay, you know how hard it is to create a software JIT compiler for a
language as dynamic as Python? It's REALLY HARD, which is why it hasn't
already been done[1]. Now you want that done in *hardware*, which is much
harder. Who's going to spend the money on R&D?

I'm sure there are thousands of l33t hax0rs out there who have thought
"Wouldn't it be c00l if there was a chip I could put into my PC to make
Python run a million times faster!!!". When I was younger and more
stupi^W innocent I had a mad fegairy for Forth and Lisp chips, but their
lack of financial viability and their unfortunate habit of actually being
*slower* than running the language in software put a big dent in the
idea. As general purpose CPUs got faster, the idea of making a specialist
language chip is now pretty much dead. Even if you could find a niche
market prepared to pay for it, "people who run Python programs" is
probably not that market.

[1] Apart from some specializing compilers like Pysco
http://psyco.sourceforge.net/introduction.html

Quite.

regards
Steve

 

[josephoswald+gg]

Okay, you know how hard it is to create a software JIT compiler for a
language as dynamic as Python? It's REALLY HARD, which is why it hasn't
already been done[1]. Now you want that done in *hardware*, which is much
harder. Who's going to spend the money on R&D?

To be clear, compiling a language that is dynamic *in the particular
way Python is designed to be dynamic*, for example, documenting and
exporting as an API the dictionary lookup mechanism for method
dispatch, is really hard.

Compiling languages such as Lisp, which are quite dynamic, and in some
ways much *more* dynamic than Python, to native machine code has been
standard practice since the 1970s.

The issue is how Python, for whatever reason, was designed as a
language. Not with whether it is dynamic or static.

 

[Luis M. González]

( Surely if this question has been asked for a zillion of times... )
( and sorry for my english! )

I'm impressed with python. I'm very happy with the language and I
find Python+Pygame a very powerful and productive way of writing 2D
games. I'm not, at this moment, worried about execution speed of the
small game I'm working on (it runs at full 60 fps even in an old AMD-
K6 450 Laptop computer), but I continue asking me the same question:

Why not a Python COMPILER?

It would be very nice to be able to output Linux, MAC or Windows
binaries of compiled (not bytecompiled) code. It would run faster, it
will be smaller in size (I think) and it will be easy to distribute to
people not having python installed. Yes, I know about py2exe, but I'm
not sure if that's the right aproach.

So, what's wrong with compiling python?

Maybe is not possible due to nature of the language? Is just a
decision?

What do you think about this?

There are some projects aimed to speed up Python by a large margin.
Right now you can use psyco, which is considered to be feature
complete, and whose future relies on the Pypy project.

Pypy is a very ambitious project and it aims, amongst many other
goals, to provide a fast just-in-time python implementation.
They even say that the "secret goal is being faster than c, which is
nonsense, isn´t it?" (I still didn´t get the joke though...).

And finally, you have ShedSkin, a project developed by one lonely and
heroic coder (Mark Dufour).
Shedskin aims at being a static python compiler, which can translate a
subset of python to stand alone executables.
It also can compile extension modules for cpython.
It works by translating python to c++ and then to machine code.
The python code must be done in a static way (getting rid of dynamic
features like, for example, not asigning different types to the same
variable).

Luis

 

[John Nagle]

Okay, you know how hard it is to create a software JIT compiler for a
language as dynamic as Python? It's REALLY HARD, which is why it hasn't
already been done[1]. Now you want that done in *hardware*, which is much
harder. Who's going to spend the money on R&D?

To be clear, compiling a language that is dynamic *in the particular
way Python is designed to be dynamic*, for example, documenting and
exporting as an API the dictionary lookup mechanism for method
dispatch, is really hard.

Compiling languages such as Lisp, which are quite dynamic, and in some
ways much *more* dynamic than Python, to native machine code has been
standard practice since the 1970s.

The issue is how Python, for whatever reason, was designed as a
language. Not with whether it is dynamic or static.

Exactly. Or "when the only tool you have is a hash, everything
looks like a dictionary". The performance problems of Python stem
from the difficulty of telling whether a hash lookup is really necessary.
The killer is that Python lets you modify the namespaces of all functions,
objects, and modules from the outside. Given that language spec,
you're stuck with an implementation like CPython where everything really
is a hash internally.

If the namespaces of objects, functions, and modules could only be
modified from within the object, function, or module, then a compiler
could examine the module and see if there was anything that required
the general late-binding case. Everything else could then be bound early,
avoiding huge numbers of unnecessary lookups and yielding a considerable
speedup. It's necessary to provide for the late-binding case to keep
the dynamism of the language, but late-binding everything kills
performance.

Shed Skin has restrictions like that, but Shed Skin is being
developed by one guy, which isn't much for an optimizing compiler.

John Nagle

 

[Reedick, Andrew]

-----Original Message-----
From: [email protected] [mailtoython-
[email protected]] On Behalf Of Luis M. González
Sent: Tuesday, February 05, 2008 6:44 PM
To: (e-mail address removed)
Subject: Re: Why not a Python compiler?


Pypy is a very ambitious project and it aims, amongst many other
goals, to provide a fast just-in-time python implementation.
They even say that the "secret goal is being faster than c, which is
nonsense, isn´t it?" (I still didn´t get the joke though...).

'c' is also the speed of light. And since nothing can travel faster than light...

One demerit has been marked against your geek card for missing an obvious science pun. Additionally, your membership to the Star Trek Lifestyle Adventure Club has been put on probationary status for the next twelve parsecs.



*****

The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers. GA621

 

[Grant Edwards]

'c' is also the speed of light.

'c' is the speed of light _in_a_vacuum_.

And since nothing can travel faster than light...

Nothing can travel faster than the speed of light
_in_a_vacuum_. There are situtaitons where things can (and
regularly do) travel faster than light:
http://en.wikipedia.org/wiki/Cherenkov_radiation

Half a demerit.

One demerit has been marked against your geek card for missing
an obvious science pun. Additionally, your membership to the
Star Trek Lifestyle Adventure Club has been put on
probationary status for the next twelve parsecs.

Ouch. Two demerits for using the distance unit "parsec" in a
context where a quantity of time was required.

 

[Reedick, Andrew]

-----Original Message-----

From: [email protected] [mailtoython-
[email protected]] On Behalf Of Grant Edwards
Sent: Wednesday, February 06, 2008 10:35 AM
To: (e-mail address removed)
Subject: Re: Why not a Python compiler?

'c' is also the speed of light.

'c' is the speed of light _in_a_vacuum_.

True.

Nothing can travel faster than the speed of light
_in_a_vacuum_. There are situtaitons where things can (and
regularly do) travel faster than light:
http://en.wikipedia.org/wiki/Cherenkov_radiation

Nope. It propagates, not travels, faster than light. Go ask a physicist to explain it. It's odd...

Ouch. Two demerits for using the distance unit "parsec" in a
context where a quantity of time was required.

Ten demerits for not catching the Star Wars Kessel Run reference.