Converting functions

Discussion in 'Python' started by iu2, Jan 24, 2011.

  1. iu2

    iu2 Guest

    Hi all,

    I'm trying to convert functions - pass a few functions to a converting
    function, which change their behaviour and return the changed
    functions:

    >>> def cfuncs(*funcs):

    n = []
    for f in funcs:
    def ff(*args, **key):
    print 'Start!', f.func_name
    res = f(*args, **key)
    print 'End', f.func_name
    return res
    n.append(ff)
    return n

    then I try it using two functions:

    >>> def f1():

    print 'hello'


    >>> def f2(x):

    return 2 * x

    Eventually:
    >>> newfuncs = cfuncs(f1, f2)


    I would expect newfuncs to hold changed versions of f1 and f2, but
    what is actually contained in newfuncs is twice the changed version of
    f2.

    That is:

    >>> newfuncs[1](100)

    Start! f2
    End f2
    200

    which is what I expected,

    but:

    >>> newfuncs[0]()

    Start! f2

    Traceback (most recent call last):
    File "<pyshell#267>", line 1, in <module>
    newfuncs[0]()
    File "<pyshell#261>", line 6, in ff
    res = f(*args, **key)
    TypeError: f2() takes exactly 1 argument (0 given)

    which is not.

    I'll appreciate your help in pointing out the mistake in defining
    cfuncs and how to fix it.
    Thank you very much!
    iu2, Jan 24, 2011
    #1
    1. Advertising

  2. iu2

    Peter Otten Guest

    iu2 wrote:

    > I'm trying to convert functions - pass a few functions to a converting
    > function, which change their behaviour and return the changed
    > functions:
    >
    > >>> def cfuncs(*funcs):

    > n = []
    > for f in funcs:
    > def ff(*args, **key):
    > print 'Start!', f.func_name
    > res = f(*args, **key)
    > print 'End', f.func_name
    > return res
    > n.append(ff)
    > return n
    >
    > then I try it using two functions:
    >
    > >>> def f1():

    > print 'hello'
    >
    >
    > >>> def f2(x):

    > return 2 * x
    >
    > Eventually:
    > >>> newfuncs = cfuncs(f1, f2)

    >
    > I would expect newfuncs to hold changed versions of f1 and f2, but
    > what is actually contained in newfuncs is twice the changed version of
    > f2.


    That is because the inner ff() references f which is a local variable of
    cfuncs(). By the time you invoke your newly created functions cfuncs() and
    thus the 'for f in funcs' loop has finished and the value of f is that of
    the last item in the funcs tuple. You can avoid the problem with another
    indirection

    def make_ff(f):
    def ff(*args, **key):
    print 'Start!', f.func_name
    res = f(*args, **key)
    print 'End', f.func_name
    return res
    return ff

    def cfuncs(*funcs):
    return [make_ff(f) for f in funcs]

    Peter
    Peter Otten, Jan 24, 2011
    #2
    1. Advertising

  3. iu2

    iu2 Guest

    On Jan 24, 9:51 am, Peter Otten <> wrote:
    > iu2 wrote:
    > > I'm trying to convert functions - pass a few functions to a converting
    > > function, which change their behaviour and return the changed
    > > functions:

    >
    > > >>> def cfuncs(*funcs):

    > >         n = []
    > >         for f in funcs:
    > >                 def ff(*args, **key):
    > >                         print 'Start!', f.func_name
    > >                         res = f(*args, **key)
    > >                         print 'End', f.func_name
    > >                         return res
    > >                 n.append(ff)
    > >         return n

    >
    > > then I try it using two functions:

    >
    > > >>> def f1():

    > >         print 'hello'

    >
    > > >>> def f2(x):

    > >         return 2 * x

    >
    > > Eventually:
    > > >>> newfuncs = cfuncs(f1, f2)

    >
    > > I would expect newfuncs to hold changed versions of f1 and f2, but
    > > what is actually contained in newfuncs is twice the changed version of
    > > f2.

    >
    > That is because the inner ff() references f which is a local variable of
    > cfuncs(). By the time you invoke your newly created functions cfuncs() and
    > thus the 'for f in funcs' loop has finished and the value of f is that of
    > the last item in the funcs tuple. You can avoid the problem with another
    > indirection
    >
    > def make_ff(f):
    >     def ff(*args, **key):
    >         print 'Start!', f.func_name
    >         res = f(*args, **key)
    >         print 'End', f.func_name
    >         return res
    >     return ff        
    >
    > def cfuncs(*funcs):
    >     return [make_ff(f) for f in funcs]
    >
    > Peter- Hide quoted text -
    >
    > - Show quoted text -


    Thanks!

    I thought a function definition creates a closure around all used
    vars.
    As I understand now only variables that are passed as function
    arguments can participate in a closure.
    iu2, Jan 24, 2011
    #3
  4. iu2

    Peter Otten Guest

    iu2 wrote:

    > I thought a function definition creates a closure around all used
    > vars.
    > As I understand now only variables that are passed as function
    > arguments can participate in a closure.


    No, it's just that all closures see the value of a variable at the time when
    the closure is run, not when it's defined.

    I don't know how to express it more clearly, so here's another example:

    >>> def f():

    .... def g(): return a * a
    .... def h(): return a + a
    .... a = 5
    .... return g, h
    ....
    >>> g, h = f()
    >>> g(), h()

    (25, 10)

    As you can see the local variable is not yet set when g() and h() are
    defined; but only the value by the time they are invoked matters.

    Here's a more involved generator version where the value may change between
    invocations:

    >>> def f(items):

    .... def g(): return a * a
    .... def h(): return a + a
    .... yield g, h
    .... for a in items:
    .... yield a
    ....
    >>> ff = f([2,3,4])
    >>> g, h = next(ff)
    >>> g()

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "<stdin>", line 2, in g
    NameError: free variable 'a' referenced before assignment in enclosing scope
    >>> next(ff)

    2
    >>> g(), h()

    (4, 4)
    >>> next(ff)

    3
    >>> g(), h()

    (9, 6)

    I think this behaviour is also called "late binding".

    Peter
    Peter Otten, Jan 24, 2011
    #4
  5. iu2

    Edmunds Cers Guest

    Peter Otten <> writes:

    > I don't know how to express it more clearly, so here's another example:
    >
    >>>> def f():

    > ... def g(): return a * a
    > ... def h(): return a + a
    > ... a = 5
    > ... return g, h
    > ...
    >>>> g, h = f()
    >>>> g(), h()

    > (25, 10)


    IMHO this whole confusion just shows that mingling assignment and
    binding makes understanding scope harder. In Python, the first
    assignment inside a function body also creates a binding (unless told
    not to do so by global) the scope of which is the _whole_ of the
    function body. A variable reference refers to the lexically innermost
    surrounding binding of said variable. Now, while it might seem that some
    magic happens in the example on the return of the function, this is in
    fact not so, since the assignment "a = 5" actually creates a binding for
    /a/ that is visible from the body of /g/, because the lexical scope of
    the binding is the whole body of /f/, so that the capture of the
    variable happens inside of the def expression (as one would expect) and
    not on return as you seem to imply.

    Slightly OT -- the above also explains why closed over variables are
    read only in Python. An assignment inside a closure would implicitly
    create a binding, so that all (even previous) references to that
    variable would refer to this new binding.

    > I think this behaviour is also called "late binding".


    "Retroactive implicit scope" would be closer.

    --
    A change in perspective is worth 80 IQ points. --- Alan Kay
    Edmunds Cers, Jan 24, 2011
    #5
  6. iu2

    Peter Otten Guest

    Edmunds Cers wrote:

    > Peter Otten <> writes:
    >
    >> I don't know how to express it more clearly, so here's another example:
    >>
    >>>>> def f():

    >> ... def g(): return a * a
    >> ... def h(): return a + a
    >> ... a = 5
    >> ... return g, h
    >> ...
    >>>>> g, h = f()
    >>>>> g(), h()

    >> (25, 10)

    >
    > IMHO this whole confusion just shows that mingling assignment and
    > binding makes understanding scope harder. In Python, the first
    > assignment inside a function body also creates a binding (unless told
    > not to do so by global) the scope of which is the _whole_ of the
    > function body. A variable reference refers to the lexically innermost
    > surrounding binding of said variable. Now, while it might seem that some
    > magic happens in the example on the return of the function, this is in
    > fact not so, since the assignment "a = 5" actually creates a binding for
    > /a/ that is visible from the body of /g/, because the lexical scope of
    > the binding is the whole body of /f/, so that the capture of the
    > variable happens inside of the def expression (as one would expect) and
    > not on return as you seem to imply.
    >
    > Slightly OT -- the above also explains why closed over variables are
    > read only in Python. An assignment inside a closure would implicitly
    > create a binding, so that all (even previous) references to that
    > variable would refer to this new binding.


    Well, in Python 3 they no longer are, courtesy of the nonlocal statement:

    >>> def f():

    .... def set(x):
    .... nonlocal a
    .... a = x
    .... def get():
    .... return a
    .... return get, set
    .... a = 42
    ....
    >>> get, set = f()
    >>> get()

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "<stdin>", line 6, in get
    NameError: free variable 'a' referenced before assignment in enclosing scope
    >>> set(42)
    >>> get()

    42

    That closed-over variables are read-only by default is just to avoid the
    ambiguity about the scope they are supposed to live in, just like global
    variables that are to be changed from within a function.

    Peter
    Peter Otten, Jan 24, 2011
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. tshad
    Replies:
    11
    Views:
    819
    tshad
    May 27, 2005
  2. Timothy Wong
    Replies:
    3
    Views:
    410
    Timothy Wong
    May 20, 2004
  3. Xiangliang Meng
    Replies:
    1
    Views:
    1,593
    Victor Bazarov
    Jun 21, 2004
  4. jamihuq

    converting inline functions to C functions

    jamihuq, May 16, 2006, in forum: C Programming
    Replies:
    7
    Views:
    344
  5. Steven D'Aprano
    Replies:
    3
    Views:
    203
    Daniel Fetchinson
    Feb 23, 2010
Loading...

Share This Page