per-function jit compiler

Discussion in 'Python' started by Luis M. González, Apr 6, 2010.

  1. This post gave me an idea: http://groups.google.com/group/comp.lang.python/msg/5d75080707104b76

    What if I write a simple decorator to figure out the types of every
    function, and then we use it as a base for a simple method-jit
    compiler for python?

    example:

    def typer(f):
    def wrap(*args):
    a = f.func_code.co_varnames
    b = [type(i) for i in args]
    return dict(zip(a,b))
    return wrap

    @typer
    def spam(a, b, c=3, d=4):
    pass

    >>> spam(8,'hello',9.9, 10)

    {'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
    'int'>}

    So by using this information, we record all the argument types used
    the first time each function/method is executed, and then we generate
    optimized code for them.
    From this point on, a guard should check if all arguments remain the
    same and, if so, the optimized code is run.
    Otherwise, just fall back to the interpreter.

    He! I have no idea how to implement it...
    Any guru out there?
    Luis
    Luis M. González, Apr 6, 2010
    #1
    1. Advertising

  2. Luis M. González

    Chris Rebert Guest

    2010/4/5 Luis M. González <>:
    > This post gave me an idea: http://groups.google.com/group/comp.lang.python/msg/5d75080707104b76
    >
    > What if I write a simple decorator to figure out the types of every
    > function, and then we use it as a base for a simple method-jit
    > compiler for python?
    >
    > example:
    >
    > def typer(f):
    >        def wrap(*args):
    >            a = f.func_code.co_varnames
    >            b = [type(i) for i in args]
    >            return dict(zip(a,b))
    >        return wrap
    >
    > @typer
    > def spam(a, b, c=3, d=4):
    >        pass
    >
    >>>> spam(8,'hello',9.9, 10)

    > {'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
    > 'int'>}
    >
    > So by using this information, we record all the argument types used
    > the first time each function/method is executed, and then we generate
    > optimized code for them.
    > >From this point on, a guard should check if all arguments remain the

    > same and, if so, the optimized code is run.
    > Otherwise, just fall back to the interpreter.
    >
    > He! I have no idea how to implement it...


    Guido's been lending out his time machine again:
    http://psyco.sourceforge.net/introduction.html

    Cheers,
    Chris
    --
    http://blog.rebertia.com
    Chris Rebert, Apr 6, 2010
    #2
    1. Advertising

  3. On 6 abr, 03:40, Chris Rebert <> wrote:
    > 2010/4/5 Luis M. González <>:
    >
    >
    >
    >
    >
    > > This post gave me an idea:http://groups.google.com/group/comp.lang.python/msg/5d75080707104b76

    >
    > > What if I write a simple decorator to figure out the types of every
    > > function, and then we use it as a base for a simple method-jit
    > > compiler for python?

    >
    > > example:

    >
    > > def typer(f):
    > >        def wrap(*args):
    > >            a = f.func_code.co_varnames
    > >            b = [type(i) for i in args]
    > >            return dict(zip(a,b))
    > >        return wrap

    >
    > > @typer
    > > def spam(a, b, c=3, d=4):
    > >        pass

    >
    > >>>> spam(8,'hello',9.9, 10)

    > > {'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
    > > 'int'>}

    >
    > > So by using this information, we record all the argument types used
    > > the first time each function/method is executed, and then we generate
    > > optimized code for them.
    > > >From this point on, a guard should check if all arguments remain the

    > > same and, if so, the optimized code is run.
    > > Otherwise, just fall back to the interpreter.

    >
    > > He! I have no idea how to implement it...

    >
    > Guido's been lending out his time machine again:http://psyco.sourceforge.net/introduction.html
    >
    > Cheers,
    > Chris
    > --http://blog.rebertia.com


    Well, psyco is a complex beast. Actually it's a Just-In-Time
    Specializer, not really a compiler at all (Guido's machine told me
    so).
    It's been superseded by pypy, which is a way more complex beast, and
    it's a tracing compiler (and many things more).
    What I mean is, why not using python's simple introspection
    capabilities (instead of partial evaluation, type inference or other
    complex mechanism) to gather type information?
    I can imagine that this, coupled with pyrex, cython or shedskin could
    give some results perhaps... by selectively compiling some function or
    all of them.

    But probably someone will throw Guido's time machine again at me. If
    it was that easier, someone would have done it already...
    I just wonder why this simple approach is not feasible.

    Luis
    Luis M. González, Apr 6, 2010
    #3
  4. En Tue, 06 Apr 2010 07:23:19 -0300, Luis M. González <>
    escribió:

    > On 6 abr, 03:40, Chris Rebert <> wrote:
    >> 2010/4/5 Luis M. González <>:
    >>
    >>
    >>
    >>
    >>
    >> > This post gave me an

    >> idea:http://groups.google.com/group/comp.lang.python/msg/5d75080707104b76
    >>
    >> > What if I write a simple decorator to figure out the types of every
    >> > function, and then we use it as a base for a simple method-jit
    >> > compiler for python?

    >>
    >> > example:

    >>
    >> > def typer(f):
    >> > def wrap(*args):
    >> > a = f.func_code.co_varnames
    >> > b = [type(i) for i in args]
    >> > return dict(zip(a,b))
    >> > return wrap

    >>
    >> > @typer
    >> > def spam(a, b, c=3, d=4):
    >> > pass

    >>
    >> >>>> spam(8,'hello',9.9, 10)
    >> > {'a': <type 'int'>, 'c': <type 'float'>, 'b': <type 'str'>, 'd':<type
    >> > 'int'>}

    >>
    >> > So by using this information, we record all the argument types used
    >> > the first time each function/method is executed, and then we generate
    >> > optimized code for them.
    >> > >From this point on, a guard should check if all arguments remain the
    >> > same and, if so, the optimized code is run.
    >> > Otherwise, just fall back to the interpreter.

    >>
    >> > He! I have no idea how to implement it...

    >>
    >> Guido's been lending out his time machine
    >> again:http://psyco.sourceforge.net/introduction.html
    >>

    >
    > Well, psyco is a complex beast. Actually it's a Just-In-Time
    > Specializer, not really a compiler at all (Guido's machine told me
    > so).
    > It's been superseded by pypy, which is a way more complex beast, and
    > it's a tracing compiler (and many things more).
    > What I mean is, why not using python's simple introspection
    > capabilities (instead of partial evaluation, type inference or other
    > complex mechanism) to gather type information?
    > I can imagine that this, coupled with pyrex, cython or shedskin could
    > give some results perhaps... by selectively compiling some function or
    > all of them.


    That *is* exactly what psyco does. A specialized function (or block) is a
    pre-compiled block with a known combination of variable types, detected at
    runtime.
    psyco has not been superseded by pypy, they're two separate projects.
    (psyco's author focus has migrated, though).

    --
    Gabriel Genellina
    Gabriel Genellina, Apr 6, 2010
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. drv

    JIT Compiler

    drv, Jul 9, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    1,521
  2. RickMuller
    Replies:
    4
    Views:
    694
    Alexey Shamrin
    Mar 26, 2005
  3. Ken

    Failed to load JIT compiler

    Ken, May 7, 2007, in forum: ASP .Net
    Replies:
    0
    Views:
    911
  4. Luis M. González

    per-method jit compiler

    Luis M. González, Apr 6, 2010, in forum: Python
    Replies:
    1
    Views:
    466
    Irmen de Jong
    Apr 6, 2010
  5. 1989lzhh
    Replies:
    0
    Views:
    36
    1989lzhh
    Jun 12, 2014
Loading...

Share This Page