Functions and code objects

Discussion in 'Python' started by Fuzzyman, Jul 27, 2006.

  1. Fuzzyman

    Fuzzyman Guest

    Hello all,

    I'm trying to extract the code object from a function, and exec it
    without explicitly passing parameters.

    The code object 'knows' it expects to receive paramaters. It's
    'arg_count' attribute is readonly.

    How can I set the arg_count to 0, or pass parameters to the code object
    when I exec it ?

    >>> def f(x):

    .... print x
    ....
    >>> c = f.func_code
    >>> type(c)

    <type 'code'>
    >>> exec c

    Traceback (most recent call last):
    File "<input>", line 1, in ?
    TypeError: f() takes exactly 1 argument (0 given)
    >>> c.co_argcount

    1
    >>> c.co_argcount = 0

    Traceback (most recent call last):
    File "<input>", line 1, in ?
    TypeError: readonly attribute
    >>>


    Fuzzyman
    http://www.voidspace.org.uk/python/index.shtml
    Fuzzyman, Jul 27, 2006
    #1
    1. Advertising

  2. Fuzzyman

    Fuzzyman Guest

    Fuzzyman wrote:
    > Hello all,
    >
    > I'm trying to extract the code object from a function, and exec it
    > without explicitly passing parameters.
    >
    > The code object 'knows' it expects to receive paramaters. It's
    > 'arg_count' attribute is readonly.
    >
    > How can I set the arg_count to 0, or pass parameters to the code object
    > when I exec it ?
    >


    Ok, so now I'm getting somewhere, without really knowing what I'm
    doing. Using the CodeType I can create a new code object with identical
    attributes, except an 'argcount' of 0.

    It doesn't quite work, so I probably need to set some of the attributes
    *differently*.

    The code I use is :

    >>> def f(x):

    ... print x
    ...
    >>> c = f.func_code
    >>> CodeType = type(c)
    >>> a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

    ... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
    c.co_name,
    ... c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars)
    >>> a

    <code object f at 01C707A0, file "<input>", line 1>
    >>> exec a

    Traceback (most recent call last):
    File "<input>", line 1, in ?
    File "<input>", line 2, in f
    UnboundLocalError: local variable 'x' referenced before assignment

    (the first argument, 0, becomes the 'arg_count' attribute)

    So the code object is still creating a local scope, and knows that x is
    a local variable. ('co_nlocals' and 'co_varnames' ?).

    I'd like to construct the code object so that it takes the parameters
    from the enclosing scope (the context I pass into exec I guess),
    without clobbering any local variables that may be defined in the code
    object.

    Anyone got any clues ?


    The attributes mean (from http://pyref.infogami.com/type-code ) :

    * co_name gives the function name
    * co_argcount is the number of positional arguments (including
    arguments with default values)
    * co_nlocals is the number of local variables used by the function
    (including arguments)
    * co_varnames is a tuple containing the names of the local
    variables (starting with the argument names)
    * co_cellvars is a tuple containing the names of local variables
    that are referenced by nested functions
    * co_freevars is a tuple containing the names of free variables
    * co_code is a string representing the sequence of bytecode
    instructions
    * co_consts is a tuple containing the literals used by the bytecode
    * co_names is a tuple containing the names used by the bytecode
    * co_filename is the filename from which the code was compiled
    * co_firstlineno is the first line number of the function
    * co_lnotab is a string encoding the mapping from byte code offsets
    to line numbers (for details see the source code of the interpreter)
    * co_stacksize is the required stack size (including local
    variables)
    * co_flags is an integer encoding a number of flags for the
    interpreter.

    In case anyone wonders, this is purely an experiment in creating
    'annoymous code blocks'. :)

    What is a 'free variable' ?

    'cell_vars' also looks interesting, but probably better not to mess
    with it :) (I guess it is only relevant for functions defined in the
    code block)

    Fuzzyman
    http://www.voidspace.org.uk/python/index.shtml
    Fuzzyman, Jul 27, 2006
    #2
    1. Advertising

  3. Fuzzyman

    Fuzzyman Guest

    Fuzzyman wrote:
    > Fuzzyman wrote:
    > > Hello all,
    > >
    > > I'm trying to extract the code object from a function, and exec it
    > > without explicitly passing parameters.
    > >
    > > The code object 'knows' it expects to receive paramaters. It's
    > > 'arg_count' attribute is readonly.
    > >
    > > How can I set the arg_count to 0, or pass parameters to the code object
    > > when I exec it ?
    > >

    >
    > Ok, so now I'm getting somewhere, without really knowing what I'm
    > doing. Using the CodeType I can create a new code object with identical
    > attributes, except an 'argcount' of 0.
    >
    > It doesn't quite work, so I probably need to set some of the attributes
    > *differently*.
    >
    > The code I use is :
    >
    > >>> def f(x):

    > ... print x
    > ...
    > >>> c = f.func_code
    > >>> CodeType = type(c)
    > >>> a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

    > ... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
    > c.co_name,
    > ... c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars)
    > >>> a

    > <code object f at 01C707A0, file "<input>", line 1>
    > >>> exec a

    > Traceback (most recent call last):
    > File "<input>", line 1, in ?
    > File "<input>", line 2, in f
    > UnboundLocalError: local variable 'x' referenced before assignment
    >
    > (the first argument, 0, becomes the 'arg_count' attribute)
    >
    > So the code object is still creating a local scope, and knows that x is
    > a local variable. ('co_nlocals' and 'co_varnames' ?).
    >
    > I'd like to construct the code object so that it takes the parameters
    > from the enclosing scope (the context I pass into exec I guess),
    > without clobbering any local variables that may be defined in the code
    > object.
    >
    > Anyone got any clues ?
    >


    *Damn* I've extracted the code object and told it that it has no
    arguments. Executing the code object results in the function object !

    The code object is obviously the code object for the function
    definition. *sigh*

    I was hoping I could get to the code object for the *body* of the
    function. Looks like that won't be possible without dis-assembling the
    bytecode or other tricks even more hackish than what I've already done.

    For the record, the code I was using was :

    x = 3
    def f(x):
    print x

    CodeType = type(f.func_code)

    def convert_function(f):
    code = f.func_code
    nlocals = max(code.co_nlocals - code.co_argcount, 0)
    newCode = CodeType(0, nlocals, code.co_stacksize, code.co_flags,
    code.co_code, code.co_consts, code.co_names,
    code.co_varnames, code.co_filename,
    code.co_name,
    code.co_firstlineno, code.co_lnotab,
    code.co_freevars,
    code.co_cellvars)
    return newCode

    print convert_function(f)
    exec convert_function(f)

    Fuzzyman
    http://www.voidspace.org.uk/python/index.shtml
    Fuzzyman, Jul 27, 2006
    #3
  4. Fuzzyman

    Simon Forman Guest

    Fuzzyman wrote:
    > Fuzzyman wrote:
    > > Fuzzyman wrote:
    > > > Hello all,
    > > >
    > > > I'm trying to extract the code object from a function, and exec it
    > > > without explicitly passing parameters.
    > > >
    > > > The code object 'knows' it expects to receive paramaters. It's
    > > > 'arg_count' attribute is readonly.
    > > >
    > > > How can I set the arg_count to 0, or pass parameters to the code object
    > > > when I exec it ?
    > > >

    > >
    > > Ok, so now I'm getting somewhere, without really knowing what I'm
    > > doing. Using the CodeType I can create a new code object with identical
    > > attributes, except an 'argcount' of 0.
    > >
    > > It doesn't quite work, so I probably need to set some of the attributes
    > > *differently*.
    > >
    > > The code I use is :
    > >
    > > >>> def f(x):

    > > ... print x
    > > ...
    > > >>> c = f.func_code
    > > >>> CodeType = type(c)
    > > >>> a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

    > > ... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
    > > c.co_name,
    > > ... c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars)
    > > >>> a

    > > <code object f at 01C707A0, file "<input>", line 1>
    > > >>> exec a

    > > Traceback (most recent call last):
    > > File "<input>", line 1, in ?
    > > File "<input>", line 2, in f
    > > UnboundLocalError: local variable 'x' referenced before assignment
    > >
    > > (the first argument, 0, becomes the 'arg_count' attribute)
    > >
    > > So the code object is still creating a local scope, and knows that x is
    > > a local variable. ('co_nlocals' and 'co_varnames' ?).
    > >
    > > I'd like to construct the code object so that it takes the parameters
    > > from the enclosing scope (the context I pass into exec I guess),
    > > without clobbering any local variables that may be defined in the code
    > > object.
    > >
    > > Anyone got any clues ?
    > >

    >
    > *Damn* I've extracted the code object and told it that it has no
    > arguments. Executing the code object results in the function object !
    >
    > The code object is obviously the code object for the function
    > definition. *sigh*
    >
    > I was hoping I could get to the code object for the *body* of the
    > function. Looks like that won't be possible without dis-assembling the
    > bytecode or other tricks even more hackish than what I've already done.
    >
    > For the record, the code I was using was :
    >
    > x = 3
    > def f(x):
    > print x
    >
    > CodeType = type(f.func_code)
    >
    > def convert_function(f):
    > code = f.func_code
    > nlocals = max(code.co_nlocals - code.co_argcount, 0)
    > newCode = CodeType(0, nlocals, code.co_stacksize, code.co_flags,
    > code.co_code, code.co_consts, code.co_names,
    > code.co_varnames, code.co_filename,
    > code.co_name,
    > code.co_firstlineno, code.co_lnotab,
    > code.co_freevars,
    > code.co_cellvars)
    > return newCode
    >
    > print convert_function(f)
    > exec convert_function(f)
    >
    > Fuzzyman
    > http://www.voidspace.org.uk/python/index.shtml


    Out of curiosity, why are you doing this?

    Peace,
    ~Simon
    Simon Forman, Jul 27, 2006
    #4
  5. Fuzzyman

    Duncan Booth Guest

    Fuzzyman wrote:

    >> I'd like to construct the code object so that it takes the parameters
    >> from the enclosing scope (the context I pass into exec I guess),
    >> without clobbering any local variables that may be defined in the code
    >> object.
    >>
    >> Anyone got any clues ?


    Does this do what you wanted? Instead of messing about with the code object
    just work out which values from the namespace the function actually
    expects.

    >>> def callfromnamespace(fn, namespace):

    names = fn.func_code.co_varnames[:fn.func_code.co_argcount]
    fn(**dict((name, namespace[name])
    for name in names if name in namespace))


    >>> def f(x, y=99):

    z = 2
    print x, y, z


    >>> x = 42
    >>> callfromnamespace(f, globals())

    42 99 2
    >>> y = 37
    >>> callfromnamespace(f, globals())

    42 37 2
    >>> def testme():

    x = 3
    callfromnamespace(f, vars())
    y = 9
    callfromnamespace(f, vars())


    >>> testme()

    3 99 2
    3 9 2
    Duncan Booth, Jul 27, 2006
    #5
  6. Fuzzyman

    Fuzzyman Guest

    Simon Forman wrote:
    > Fuzzyman wrote:

    [snip..]
    > > I was hoping I could get to the code object for the *body* of the
    > > function. Looks like that won't be possible without dis-assembling the
    > > bytecode or other tricks even more hackish than what I've already done.
    > >
    > > For the record, the code I was using was :
    > >
    > > x = 3
    > > def f(x):
    > > print x
    > >
    > > CodeType = type(f.func_code)
    > >
    > > def convert_function(f):
    > > code = f.func_code
    > > nlocals = max(code.co_nlocals - code.co_argcount, 0)
    > > newCode = CodeType(0, nlocals, code.co_stacksize, code.co_flags,
    > > code.co_code, code.co_consts, code.co_names,
    > > code.co_varnames, code.co_filename,
    > > code.co_name,
    > > code.co_firstlineno, code.co_lnotab,
    > > code.co_freevars,
    > > code.co_cellvars)
    > > return newCode
    > >
    > > print convert_function(f)
    > > exec convert_function(f)
    > >
    > > Fuzzyman
    > > http://www.voidspace.org.uk/python/index.shtml

    >
    > Out of curiosity, why are you doing this?
    >


    In Ruby anonymous code blocks (which take parameters and can presumably
    return values) are executed within the scope in which they are used,
    rather than the scope that they are defined.

    I wanted to see how close to that I could get in Python. Obviously code
    could be compiled from a string, but I thought extracting it from the
    body of a function was closer.

    All the best,

    Fuzzyman
    http://www.voidspace.org.uk/python/index.shtml

    > Peace,
    > ~Simon
    Fuzzyman, Jul 27, 2006
    #6
  7. Fuzzyman

    Fuzzyman Guest

    Duncan Booth wrote:
    > Fuzzyman wrote:

    [snip..]
    >
    > Does this do what you wanted? Instead of messing about with the code object
    > just work out which values from the namespace the function actually
    > expects.
    >
    > >>> def callfromnamespace(fn, namespace):

    > names = fn.func_code.co_varnames[:fn.func_code.co_argcount]
    > fn(**dict((name, namespace[name])
    > for name in names if name in namespace))
    >
    >
    > >>> def f(x, y=99):

    > z = 2
    > print x, y, z
    >
    >
    > >>> x = 42
    > >>> callfromnamespace(f, globals())

    > 42 99 2
    > >>> y = 37
    > >>> callfromnamespace(f, globals())

    > 42 37 2
    > >>> def testme():

    > x = 3
    > callfromnamespace(f, vars())
    > y = 9
    > callfromnamespace(f, vars())
    >
    >
    > >>> testme()

    > 3 99 2
    > 3 9 2


    Hmmm... it may do, thanks. :)

    I'll play around, and this may become a blog entry.

    Fuzzyman
    http://www.voidspace.org.uk/python/index.shtml
    Fuzzyman, Jul 27, 2006
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Xiangliang Meng
    Replies:
    1
    Views:
    1,577
    Victor Bazarov
    Jun 21, 2004
  2. John Goche
    Replies:
    10
    Views:
    724
    Marcus Kwok
    Dec 8, 2006
  3. 7stud
    Replies:
    11
    Views:
    673
    Dennis Lee Bieber
    Mar 20, 2007
  4. mafiQ
    Replies:
    0
    Views:
    465
    mafiQ
    Apr 22, 2009
  5. py
    Replies:
    5
    Views:
    289
    Daniel DeLorme
    Mar 24, 2007
Loading...

Share This Page