Issues in compiling Python to machine code

Discussion in 'Python' started by Johnathan Doe, May 23, 2004.

  1. I've been thinking about what the issues would be in compiling
    Python into native machine code, and since type information is
    important in Python, it seems possible that Python code can
    be compiled into native machine code (albeit with a lot of
    extra effort).

    For instance, type information is discovered when something is
    assigned to a variable or an anonymous piece of data is used
    in a program. Compiling Python bytecode into native machine
    code could involve a pass over the bytecode to look at all the
    uses of a variable or anonymous variable, call some Python
    function to discover its variable, then record it in the symbol
    table.

    Another issue is what to do with objects. Finding the offset
    to a pointer which is the object's constructor, for instance,
    if that's the way it's done internally in Python.

    And then freeing the memory used by an object when it's no
    longer needed. Perhaps freeing memory for an object can be
    done by inserting machine code to return memory back to the
    heap after the first pass over the bytecode and recording the
    place a variable is last used (hopefully then it won't be
    used in some other way later of course... perhaps it can
    record all cases where that variable is referred to by another
    name and track them, too.)

    Seems like compiled Python would need a lot of support code,
    in any case, that gets run at certain times: before the program
    starts, whenever a variable is accessed, before an object is
    used (to allocate memory), whenever something happens to data
    within an object (allocate/reallocate/destroy), and so on...

    Are there any reasons why Python would be *impossible* to compile?

    Thanks :)
     
    Johnathan Doe, May 23, 2004
    #1
    1. Advertising

  2. Johnathan Doe

    Harald Massa Guest

    Johnathan

    > Are there any reasons why Python would be *impossible* to compile?


    there is quite a FAQ to this question :)))

    Why is there no Python compiler ?
    - because when it was time to do so, it was not possible to get a
    diploma for creating a compiler, so sufficent motivation was not
    available.


    Actually there are some projects towards a "python compiler":

    look at:

    http://codespeak.net/pypy/

    -> Python in Python

    http://psyco.sourceforge.net/

    -> Python special compiler. Creates x86 machine code out of Python "on
    the fly"

    http://web.mit.edu/msalib/www/urop/

    Starkiller is some project to deal with types in Python

    http://ironpython.com/

    Jim Hugunin is doing a projekt for "Python 'compilation' to the .NET /
    MONO platform"


    Maybe it could be valuable to you to visit EuroPython
    (http://www.europython.org/, the biggest Python Conference without US
    travelling restrictions. Especially

    http://www.europython.org/conferences/epc2004/info/talks/refereed/msalib0
    1

    http://www.europython.org/conferences/epc2004/info/talks/python_language/
    arigo02

    http://www.europython.org/conferences/epc2004/info/talks/python_language/
    arigo01

    may be of great interest for you. Registration is still open, you can
    attend for 220€ normal fee.


    Harald
     
    Harald Massa, May 23, 2004
    #2
    1. Advertising

  3. Johnathan Doe

    Ryan Paul Guest

    On Sun, 23 May 2004 21:23:19 +1000, Johnathan Doe wrote:

    > I've been thinking about what the issues would be in compiling
    > Python into native machine code, and since type information is
    > important in Python, it seems possible that Python code can
    > be compiled into native machine code (albeit with a lot of
    > extra effort).
    >
    > For instance, type information is discovered when something is
    > assigned to a variable or an anonymous piece of data is used
    > in a program. Compiling Python bytecode into native machine
    > code could involve a pass over the bytecode to look at all the
    > uses of a variable or anonymous variable, call some Python
    > function to discover its variable, then record it in the symbol
    > table.
    >
    > Another issue is what to do with objects. Finding the offset
    > to a pointer which is the object's constructor, for instance,
    > if that's the way it's done internally in Python.
    >
    > And then freeing the memory used by an object when it's no
    > longer needed. Perhaps freeing memory for an object can be
    > done by inserting machine code to return memory back to the
    > heap after the first pass over the bytecode and recording the
    > place a variable is last used (hopefully then it won't be
    > used in some other way later of course... perhaps it can
    > record all cases where that variable is referred to by another
    > name and track them, too.)
    >
    > Seems like compiled Python would need a lot of support code,
    > in any case, that gets run at certain times: before the program
    > starts, whenever a variable is accessed, before an object is
    > used (to allocate memory), whenever something happens to data
    > within an object (allocate/reallocate/destroy), and so on...
    >
    > Are there any reasons why Python would be *impossible* to compile?
    >
    > Thanks :)


    we have talked about this here before, I think. Native compilation wont
    improve runtime speed of a dynamically typed language. Despite this, there
    are actually a few projects out there endevouring to do it anyway. If you
    want better speed from a python application, try a customized python
    runtime engine like Psyco.

    --SegPhault
     
    Ryan Paul, May 23, 2004
    #3
  4. Thanks for all the links!

    Obviously I've just shown myself to think out loud, and not to think
    very much at all before doing the "out loud" part... :)

    I'll take a look through all these.

    Incidentally, this doesn't have anything to do with speeding up Python.
    It's already fast enough for what I want. My main issue is writing
    low level code in Python. It's possible to do great things like
    autodetect hardware in Python for an OS, e.g. Anaconda/Kudzu, but why
    not be able to write an OS completely in Python? I am sure it's
    possible somehow. Just not sure yet :)

    Possibly, GRUB can load the startup code, which contains an embedded
    Python interpeter. Then the interpreter defines the kernel_main
    function and takes over. I'd need some way of gluing up all the low
    level stuff to the interpreted Python "platform" : interrupts and page
    faults and whatnot.

    Anyway, I should talk about this on alt.os.development :)

    Johnathan
     
    Johnathan Doe, May 24, 2004
    #4
  5. > we have talked about this here before, I think. Native compilation wont
    > improve runtime speed of a dynamically typed language. Despite this, there
    > are actually a few projects out there endevouring to do it anyway. If you
    > want better speed from a python application, try a customized python
    > runtime engine like Psyco.


    That might be a good idea actually, to investigate Psyco and others.
    I'm not interested in speeding up Python as such, I'm just interested in
    being able to write system-level software in Python. I would like to be
    able to write an OS in Python for instance.

    Johnathan
     
    Johnathan Doe, May 24, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. johnny
    Replies:
    5
    Views:
    532
    Antonio Cuni
    Jul 11, 2007
  2. Garrett Cooper
    Replies:
    0
    Views:
    568
    Garrett Cooper
    Feb 24, 2009
  3. Garrett Cooper
    Replies:
    0
    Views:
    504
    Garrett Cooper
    Feb 24, 2009
  4. mlt
    Replies:
    9
    Views:
    393
    Alf P. Steinbach
    Mar 12, 2009
  5. Lon Baker
    Replies:
    1
    Views:
    228
    Csaba Henk
    Mar 21, 2005
Loading...

Share This Page