How to get dynamically-created fxn's source?

Discussion in 'Python' started by gb345, Nov 5, 2010.

  1. gb345

    gb345 Guest

    For a project I'm working on I need a way to retrieve the source
    code of dynamically generated Python functions. (These functions
    are implemented dynamically in order to simulate "partial application"
    in Python.[1]) The ultimate goal is to preserve a textual record
    of transformations performed on data, along with all the data (both
    pre- and post- transformation) itself.

    These transformation functions could be dynamically generated as
    closures, but I suspect that this would make it more difficult to
    extract source code that could serve as a reasonably self-contained
    description of the transformation (because this source code would
    refer to variables defined outside of the function). An alternative
    would be to generate the *source code* for the functions dynamically,
    from a template having slots (within the function's definition)
    that gets filled in with actual parameter values, and pass this
    source code to exec.

    In any case, the problem remains of how to extract the
    dynamically-generated function's source code.

    One possibility would be to define a Transformation wrapper class
    whose __init__ takes the dynamically-generated source code (a
    string) as argument, keeps it around as an instance attribute for
    future reference, and exec's it to define its __call__ method.

    Is this overkill/reinventing the wheel? IOW, does Python already
    have a built-in way to achieve this same effect?

    (Also, the wrapper scheme above is somewhat loose: it's relatively
    easy for the source code instance attribute (as described) to fall
    out of sync with the function that actually executes when __call__
    runs. Maybe a tighter connection between the obtained source code
    and the code that actually executes when __call__ runs is possible.)

    I'm aware of the inspect module, but from reading its source code
    I gather that it is designed for inspecting source code that is
    explicitly written in files, and would not be too good at inspecting
    functions that are generated dynamically (i.e. not from source code
    explicitly given in a source file--I hope that made sense).

    Your comments and suggestions would be much appreciated. Many
    thanks in advance!

    G

    [1] For example, given a base function spam that has the signature
    (typeT, typeT, typeT, int, int, int) and three specific integer
    values X, Y, and Z, dynamically generate a new function spamXYZ
    with signature (typeT, typeT, typeT) such that spamXYZ(A, B, C) is
    identical to spam(A, B, C, X, Y, Z), for all possible values of A,
    B, C.
     
    gb345, Nov 5, 2010
    #1
    1. Advertising

  2. On 11/5/2010 9:55 AM gb345 said...

    > In any case, the problem remains of how to extract the
    > dynamically-generated function's source code.
    >


    Are you looking for the source code of the dynamically created wrapper
    function (effectively the piece that calls the original function) or of
    the wrapped function? (the piece that ultimately gets called?)

    I'm pretty sure you can't get source from the executable function
    (python 2.x anyway), but with your naming scheme, it may be possible to
    trace into the wrapped function's source.

    Emile
     
    Emile van Sebille, Nov 5, 2010
    #2
    1. Advertising

  3. gb345

    Peter Otten Guest

    gb345 wrote:

    > For a project I'm working on I need a way to retrieve the source
    > code of dynamically generated Python functions. (These functions
    > are implemented dynamically in order to simulate "partial application"
    > in Python.[1]) The ultimate goal is to preserve a textual record
    > of transformations performed on data, along with all the data (both
    > pre- and post- transformation) itself.


    Are you aware of functools.partial?

    >>> from functools import partial
    >>> def f(a, b, c, x, y, z):

    .... return a*x + b*y*y + c*z*z*z
    ....
    >>> fstar = partial(f, x=1, y=2, z=3)
    >>> fstar(1, 0, 0)

    1
    >>> fstar(0, 1, 0)

    4
    >>> fstar(0, 0, 1)

    27
    >>> fstar.args, fstar.keywords

    ((), {'y': 2, 'x': 1, 'z': 3})
    >>> fstar.func

    <function f at 0x7fc4868e6848>
     
    Peter Otten, Nov 5, 2010
    #3
  4. gb345

    gb345 Guest

    In <ib1fai$8g3$02$-online.com> Peter Otten <> writes:

    >gb345 wrote:


    >> For a project I'm working on I need a way to retrieve the source
    >> code of dynamically generated Python functions. (These functions
    >> are implemented dynamically in order to simulate "partial application"
    >> in Python.[1])


    >Are you aware of functools.partial?


    >>>> from functools import partial
    >>>> def f(a, b, c, x, y, z):

    >... return a*x + b*y*y + c*z*z*z
    >...
    >>>> fstar = partial(f, x=1, y=2, z=3)
    >>>> fstar(1, 0, 0)

    >1
    >>>> fstar(0, 1, 0)

    >4
    >>>> fstar(0, 0, 1)

    >27
    >>>> fstar.args, fstar.keywords

    >((), {'y': 2, 'x': 1, 'z': 3})
    >>>> fstar.func

    ><function f at 0x7fc4868e6848>


    I was not aware of functools.partial. This makes the problem a
    lot easier. Thanks!

    G
     
    gb345, Nov 8, 2010
    #4
  5. On Nov 5, 5:55 pm, gb345 <> wrote:
    > For a project I'm working on I need a way to retrieve the source
    > code of dynamically generated Python functions.  (These functions
    > are implemented dynamically in order to simulate "partial application"
    > in Python.[1])  The ultimate goal is to preserve a textual record
    > of transformations performed on data, along with all the data (both
    > pre- and post- transformation) itself.
    >
    > These transformation functions could be dynamically generated as
    > closures, but I suspect that this would make it more difficult to
    > extract source code that could serve as a reasonably self-contained
    > description of the transformation (because this source code would
    > refer to variables defined outside of the function).  An alternative
    > would be to generate the *source code* for the functions dynamically,
    > from a template having slots (within the function's definition)
    > that gets filled in with actual parameter values, and pass this
    > source code to exec.
    >
    > In any case, the problem remains of how to extract the
    > dynamically-generated function's source code.
    >
    > One possibility would be to define a Transformation wrapper class
    > whose __init__ takes the dynamically-generated source code (a
    > string) as argument, keeps it around as an instance attribute for
    > future reference, and exec's it to define its __call__ method.
    >
    > Is this overkill/reinventing the wheel?  IOW, does Python already
    > have a built-in way to achieve this same effect?
    >
    > (Also, the wrapper scheme above is somewhat loose: it's relatively
    > easy for the source code instance attribute (as described) to fall
    > out of sync with the function that actually executes when __call__
    > runs.  Maybe a tighter connection between the obtained source code
    > and the code that actually executes when __call__ runs is possible.)
    >
    > I'm aware of the inspect module, but from reading its source code
    > I gather that it is designed for inspecting source code that is
    > explicitly written in files, and would not be too good at inspecting
    > functions that are generated dynamically (i.e. not from source code
    > explicitly given in a source file--I hope that made sense).
    >
    > Your comments and suggestions would be much appreciated.  Many
    > thanks in advance!
    >
    > G
    >
    > [1] For example, given a base function spam that has the signature
    > (typeT, typeT, typeT, int, int, int) and three specific integer
    > values X, Y, and Z, dynamically generate a new function spamXYZ
    > with signature (typeT, typeT, typeT) such that spamXYZ(A, B, C) is
    > identical to spam(A, B, C, X, Y, Z), for all possible values of A,
    > B, C.


    I wondered a lot about this issue when writing my own decorator
    module.
    You may look at the source code of the FunctionMaker class which tries
    to address the problem by storing the source code of the generated
    function into an attribute. Using inspect.getsource does not work
    (see http://bugs.python.org/issue1764286). Of course if you
    functools.partial
    is enough to solve your problem use it and be done with it.
     
    Michele Simionato, Nov 9, 2010
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Raed Sawalha
    Replies:
    2
    Views:
    1,527
    =?Utf-8?B?UHN5Y2hv?=
    Oct 5, 2004
  2. Karthick Kumar
    Replies:
    1
    Views:
    799
  3. A man

    pure virtual fxn decl

    A man, Sep 22, 2003, in forum: C++
    Replies:
    4
    Views:
    360
    jeffc
    Sep 22, 2003
  4. msimmons
    Replies:
    0
    Views:
    507
    msimmons
    Jul 16, 2009
  5. noydb
    Replies:
    3
    Views:
    261
Loading...

Share This Page