Make Python Compilable, convert to Python source to Go

[bookaa bookaa]

Make Python Compilable

This tool can be called 'Python to GoLang', which translate Python source to Golang source. And then you can compile the Go files to executable binary.. (btw: Go is a new C-like compilable language, open source).

Generally, people consider Python as a script language. It has high development efficiency, but run too slowly, interpret running, and can not compile.. It depends to Python environment, can not employ alone.

For years, many people have do a lot of job try to improve running speed ofPython, include Pypy, Cython. But all of these are not satisfied.

In my opinion, Python only define a development rule. You can easily express your logic idea with this rule. It doesnot confines that it must be run as script. Only that we only have this C Python interpretor(CPython). So we take it for granted that Python is a script language.

I think no matter how you program your work, it only represent some logics.There should be possible to convert from one computer language to another.Based on this, after some months of work, we have this utility.

I consider Python as the most efficient langrage. Many genius programmers love Python and Go. Now with this tool, we can make Python compilable, and run more quickly. Python will be perfectly! We do not need C and C++ anymore! Python is not as before!

Q1: Why do we need convert Python source to Go ? Why not coding with Go directly ?
First, Python is good at develop efficiency. And Go is too simply and do not have many really-need functions. I do not suggest you choose Go as your primary development langrage. Second, Python has a lot of 3rd part libraries.. And Go been as a new language, if can inherit all libraries from Python, it will be more powerful.

This tool can work in 2 options:
op1: If you want your product run under Python environment, or debug, you can choose op1. With this option, I will only convert your Python source files to Go, and ignore 'import' of system libraries. Mostly one line of Python will produce 1.5 lines of Go source, basicly preserve function, function names, variable names, calling graphics. The output Go source is easy to understand, optimize.
op2: Convert all Python source to Go including system libraries. Generally this will product more than 150 thousand lines of Go source. After compile,your execute binary file can run without Python installation. You should include Python27.dll and some PYD file if need. Now its possible to deply your Python product.

Q2: Why do you need python27.dll ? If you include this dll, why just run with one line of 'exec' ?
If you run with 'exec', it is run with Python interpretor. You can not improve running speed. And its not safe.
I will convert all function of Python to a correspond Go function. All variable becomes a Go variable. I will try to identify data type of each variable. If I find a var is integer, I will use int in Go. This make it possibleto improve running speed. For variables I do not know data type, or list, tuple etc, I use PyObject. So there is no choice but call correspond operation functions from python27.dll.
And many 3rd part libraries of Python is PYD compiled from C. This PYD fileis actually DLL which dependance to python27.dll.

With this tool, we can benefit all libraries of Python in Go. This makes Gomore powful. For example, we still do not have good GUI libraries for Go, now we can try GUI libraries of Python and its possible to begin GUI development with Go now.

It takes me more than 7 month to do this job, with more than 30000 lines ofsource, mostly in Python with some Go and C. As you may think, I can convert myself to Go! During the convertion, I must deal with complex Python logics. I must carefully understand any Python acknownledge point and try to find solution in Go. Its very difficault! Especially for some concept of Python which missing in Go, like try/exception, yield, var scope, no use var, var declare.

This tool will public and open source later. Up for now, only support Python 2.7.6, do not support Python3. I have tested it on Win32, Win64, and I amsure its easy to run on all OS support Python and Go. Python is open source in C, Go is also open source in G. There can run on almost all operation system. With my Python to Go, I believe Python+Go will become popular development trend.

You can download this to have a look how Python to Go work. This is a smalltool for listen and dictation of learning Foreigin languages by me.
http://pan.baidu.com/s/1eQADiHc
including Python source, and converted Go sourced(more than 170000 lines), and compiled executable binary(win32).

I may help you:
1. convert your Python project to Go. Its quick and safe and easy to deploy..
2. optimize running speed of your Python project
3. Make a speech.
LiuTaoTao (e-mail address removed)

 

[Chris Angelico]

This tool can be called 'Python to GoLang', which translate Python source to Golang source. And then you can compile the Go files to executable binary. (btw: Go is a new C-like compilable language, open source).

Sounds like you're writing a Python implementation that uses a Go
backend. As Pythons go, this is comparable to using Java, or Mono, or
RPython, or C, or anything else. So there are two questions:

1) How compatible is your Python-to-Golang converter with all the
nuances of Python code? Does it work perfectly on any arbitrary Python
script? And, what version of Python is it aimed at?

2) What's performance like? Presumably significantly better than
CPython, as that's what you're boasting here. Have you run a
standardized benchmark? How do the numbers look?

If the answer is "It'll work on anything, but it's only faster if you
restrict yourself to a specific subset of Python syntax", that's still
useful. But we'd need to see figures that tell us when it's worth
adding a separate dependency and another translation layer (after all,
every layer adds its own bug potential).

ChrisA

 

[wxjmfauth]

My opinions about Go.

i) go build XXX that creates an exe, one can put on
a usb stick and run (distribute) it, is a feature hard
to beat.

I do not know, if it will be rendered correctly.

D:\jm\jmgo>hello3.exe
ASCII abcde xyz
German äöü ÄÖÜ ß
Polish ąęźżńł
Russian абвгдеж ÑÑŽÑ
CJK 你好
French œÿéà
Misc ሴé€㑖Ѓ⌴*

D:\jm\jmgo>

ii) From a unicode perspective, excellent, coherent and
very well constructed, but...

iii) Having to work with an utf-8 arithmetic
(strings <-> runes conversion), no thanks.

jmf

 

[bookaa bookaa]

Sounds like you're writing a Python implementation that uses a Go
backend. As Pythons go, this is comparable to using Java, or Mono, or
RPython, or C, or anything else. So there are two questions:

1) How compatible is your Python-to-Golang converter with all the
nuances of Python code? Does it work perfectly on any arbitrary Python
script? And, what version of Python is it aimed at?

I try to support all Python syntax, any arbitray script. From the example
attached,you can see I translate all import system libraries needed and
produce up to 170000　lines of Go. Up to now, only support Python 2.7.6.
Maybe I will work on Python 3 later.

2) What's performance like? Presumably significantly better than
CPython, as that's what you're boasting here. Have you run a
standardized benchmark? How do the numbers look?

I must admit that after automaticaly convert Python source to Go, compile
to EXE, the running speed is just as before. For compatible reason, I must
make it behave　just as it before, support any Python feathers. Takea
example, if we find a function　call func1(2), I can not simply convert it
to Go function call as func1_in_go(2), but　something like this: current_module_scope.GetAttributeString("func1").CallObject(2)
because func1 maybe overwrited.

I think the significance of Python to Go, is it give us opportunity to
make Python project run fast. We may edit the output Go source. or We
may add some Python decorator to tell the converter its safe to convert
it in simple form.

If the answer is "It'll work on anything, but it's only faster if you
restrict yourself to a specific subset of Python syntax", that's still
useful. But we'd need to see figures that tell us when it's worth
adding a separate dependency and another translation layer (after all,
every layer adds its own bug potential).

ChrisA

thanks, LiuTaoTao

 

[Chris Angelico]

I try to support all Python syntax, any arbitray script. From the example
attached,you can see I translate all import system libraries needed and
produce up to 170000　lines of Go. Up to now, only support Python 2.7.6.
Maybe I will work on Python 3 later.


I must admit that after automaticaly convert Python source to Go, compile
to EXE, the running speed is just as before. For compatible reason, I must
make it behave　just as it before, support any Python feathers. Take a
example, if we find a function　call func1(2), I can not simply convert it
to Go function call as func1_in_go(2), but　something like this: current_module_scope.GetAttributeString("func1").CallObject(2)
because func1 maybe overwrited.

I think the significance of Python to Go, is it give us opportunity to
make Python project run fast. We may edit the output Go source. or We
may add some Python decorator to tell the converter its safe to convert
it in simple form.

This is extremely unsurprising. Everyone who says "Python is so slow"
is comparing against a less dynamic language. Python lets you change
any name *at any time*, so all lookups must be done at the time
they're asked for. (By comparison, Pike binds all global names at
compile time - effectively, when you import the module. If you want to
change one, you need to reimport code that's using it. C, of course,
binds everything early, and that's that.)

There have been a variety of proposals to remove some of Python's
dynamism with markers saying "This is read-only". Victor Stinner
started a thread on python-ideas this week with some serious proposals
and decent argument (backed by a POC fork of CPython 3.5). Also, I'm
not 100% sure but I suspect that PyPy quite possibly optimizes on the
basis of "this probably hasn't changed the meaning of len()", and does
a quick check ("if len has been rebound, go to the slow path,
otherwise run the fast path") rather than checking each time. Both of
these options are viable, both have their trade-offs... and neither
requires actually compiling via an unrelated language.

I have never liked any system that involves converting code from one
language to another and then hand-editing the resulting code. There is
a reason the languages are different; they have different strengths
and different weaknesses. There's always going to be something that's
messy in the target language. Sometimes you don't care (you can
probably write an 8086 Assembler interpreter in Python and have it run
at 4.77MHz as long as the Python interpreter is running on fast enough
hardware), but if your goal is an overall speed improvement, you're up
against a number of Python interpreters that have specifically looked
at performance (I know CPython may be considered slow, but the devs do
care about running time; and PyPy touts running times of 16% of
CPython's), so you're going to have to boast some pretty good numbers.

Strong recommendation: If you want to move forward with this, compare
against Python 3.x. New projects want to be written for Py3 rather
than Py2, and you're limiting your project's usefulness if it's
compatible only with Py2. As an added "bonus", Py3 is currently a bit
slower than Py2 in a lot of benchmarks, so you get yourself a slightly
easier target

ChrisA

 

[Stefan Behnel]

bookaa bookaa, 25.05.2014 10:17:

I think the significance of Python to Go, is it give us opportunity to
make Python project run fast.

You shouldn't make that your only goal, because you'll have a really hard
time achieving it (to put it mildly).

Stefan

 

[Mark Lawrence]

Maybe I will work on Python 3 later.

That's good to know, it'll save me wasting my time looking at it now.

 

[mm0fmf]

That's good to know, it'll save me wasting my time looking at it now.

OT:
Mark, you've been pro-Python3 enough in your recent postings you have
forced me to act. I've just upgraded my 1st Python2 app to Python3. 2to3
did 99.999% of the work and I had to get a more modern version of a
package which was Python3 compatible.

From Tuesday (Monday is a holiday), all new Python code at work will be
Python3

Andy

 

[Mark Lawrence]

OT:
Mark, you've been pro-Python3 enough in your recent postings you have
forced me to act. I've just upgraded my 1st Python2 app to Python3. 2to3
did 99.999% of the work and I had to get a more modern version of a
package which was Python3 compatible.

From Tuesday (Monday is a holiday), all new Python code at work will be
Python3

Andy

I merely think Python 3 is the way to go, and that the Python 2.8 crew
don't so much have loose screws, but never had them fitted in the first
place.

FTR I entirely agree with Roy Smith about sticking with Python 2 in his
situation, strikes me as a complete no brainer.

 

[Devin Jeanpierre]

That's good to know, it'll save me wasting my time looking at it now.

This seems like an unnecessarily harsh way of putting it, Mark. Could
you be less dismissive of the hard work others put into improving the
Python world?

-- Devin

 

[Stefan Behnel]

Ben Finney, 26.05.2014 05:20:

Which Python implementation are you talking about? Run time is not a
property of the language. It is a property of the language
implementation.

That, plus the fact that talking about a language or language
implementation "being slow" actually makes no sense at all without
referring to a specific piece of code or at least an application scenario
in which it is (provably) slow when compared to something (specific) else.

Stefan

 

[Mark Lawrence]

This seems like an unnecessarily harsh way of putting it, Mark. Could
you be less dismissive of the hard work others put into improving the
Python world?

-- Devin

Like Python 2.8?

 

[Steven D'Aprano]

Like Python 2.8?

I think you may have drifted into an alternate universe. There is no
Python 2.8 and nobody has done any hard work on writing it, so I have no
idea what point you think you are making. (That's the trouble with snappy
one-line answers to serious questions -- they often make no sense to
anyone outside of your head.)

The OP has done a lot of hard work at writing a Python 2 to Go translator
(allegedly -- we haven't actually been shown his project) and assuming it
works and done what he says it does, that's valuable. Python 2.7 will be
part of the Python ecosystem until at least 2022. Furthermore, adapting
an existing Python2 to Go compiler to work with Python3 will be much
easier than writing one from scratch.