Reasoning behind nested scope

Discussion in 'Python' started by Andy Baker, Aug 3, 2004.

  1. Andy Baker

    Andy Baker Guest

    Hi there,

    I'm learning Python at the moment and trying to grok the thinking behind
    it's scoping and nesting rules.

    I was googling for nested functions and found this Guido quote:
    (http://www.python.org/search/hypermail/python-1993/0343.html)

    "This is because nested function definitions don't have access to the
    local variables of the surrounding block -- only to the globals of the
    containing module. This is done so that lookup of globals doesn't
    have to walk a chain of dictionaries -- as in C, there are just two
    nested scopes: locals and globals (and beyond this, built-ins).
    Therefore, nested functions have only a limited use. This was a
    deliberate decision, based upon experience with languages allowing
    arbitraries nesting such as Pascal and both Algols -- code with too
    many nested scopes is about as readable as code with too many GOTOs.
    And, of course, in Python, the "proper" way is to use an
    object-oriented programming style"

    This sounds reasonable to me although nested scope always struck me as more
    natural and intuitive ('Don't be surprising')

    I was wondering how the balance changed in favour of nested scope? What
    changed people's minds about 'readability' vs other factors? Are nested
    scopes still regarded as leading to spagetti code etc?

    (On a side note is there any way to call a nested function from outside the
    parent? I was kind of expecting nested functions to be addressable through
    dot notation like methods are but I can see why that wouldn't be quite
    right. This might be a better question for the tutor list...)

    Andy Baker
     
    Andy Baker, Aug 3, 2004
    #1
    1. Advertising

  2. Andy Baker

    Mel Wilson Guest

    In article <>,
    "Andy Baker" <> wrote:
    >"This is because nested function definitions don't have access to the
    >local variables of the surrounding block -- only to the globals of the
    >containing module. This is done so that lookup of globals doesn't
    >have to walk a chain of dictionaries -- as in C, there are just two
    >nested scopes: locals and globals (and beyond this, built-ins).
    >Therefore, nested functions have only a limited use. This was a
    >deliberate decision, based upon experience with languages allowing
    >arbitraries nesting such as Pascal and both Algols -- code with too
    >many nested scopes is about as readable as code with too many GOTOs.
    >And, of course, in Python, the "proper" way is to use an
    >object-oriented programming style"
    >
    >This sounds reasonable to me although nested scope always struck me as more
    >natural and intuitive ('Don't be surprising')
    >
    >I was wondering how the balance changed in favour of nested scope? What
    >changed people's minds about 'readability' vs other factors? Are nested
    >scopes still regarded as leading to spagetti code etc?


    I guess a consensus built up that the power/safety
    tradeoff had given too much emphasis to safety.

    My own poster child for nested scopes is in some old post
    on c.l.p . I wanted to build a lot of similar labelled text
    controls in wxPython, following a naming convention. I
    could have written a function, but pre-nested-scope, any
    change I made to the information used in building the
    controls would have meant changing the function parameter
    list, and all the function calls. I came up with a template
    string which got parameter substituted via the '%' operator,
    and then `exec`ed so as code it had full access to all the
    info that was in the contemporary scope.

    Post nested scope, any info the function needed from the
    surrounding scope could simply be read from there. The code
    was much less unusual.

    I believe (without proof) that any language that won't
    let me shoot myself in the foot will also prevent me doing
    anything useful. Debate is heating up now about write
    access to enclosing scopes. We'll see how strong my faith
    really is. People will be able to write function nests
    instead of classes.

    >(On a side note is there any way to call a nested function from outside the
    >parent? I was kind of expecting nested functions to be addressable through
    >dot notation like methods are but I can see why that wouldn't be quite
    >right. This might be a better question for the tutor list...)


    Yup.

    def f1 (a):
    if a:
    x = 2
    else:
    x = 3

    def f2 (y):
    return y % x

    return f2

    import os
    b = f1 (os.name.startswith ('p'))
    for c in range (10):
    print b (c)



    Regards. Mel.
     
    Mel Wilson, Aug 3, 2004
    #2
    1. Advertising

  3. Andy Baker

    Larry Bates Guest

    Ever since my first Fortran class some 30+ years ago
    I've always liked nested scopes. If you want something
    to be global, say so in your code. In Fortran we used
    common blocks, in Python you use a global directive.
    Making everything global can lead to many name clashes
    or you must have lots of unique variable names which
    lengthens code and makes reuse more difficult.

    Judicious use of OOP and globals seems to be the best
    approach. The more I learn/use OOP the less I seem
    to require globals, but there are some times they
    seem a preferable solution.

    I don't believe there is any way to call nested
    functions from outside their parent. If you need
    something like that, make the function a class and
    make the nested function a method of the class.

    HTH,
    Larry Bates
    Syscon, Inc.


    "Andy Baker" <> wrote in message
    news:...
    > Hi there,
    >
    > I'm learning Python at the moment and trying to grok the thinking behind
    > it's scoping and nesting rules.
    >
    > I was googling for nested functions and found this Guido quote:
    > (http://www.python.org/search/hypermail/python-1993/0343.html)
    >
    > "This is because nested function definitions don't have access to the
    > local variables of the surrounding block -- only to the globals of the
    > containing module. This is done so that lookup of globals doesn't
    > have to walk a chain of dictionaries -- as in C, there are just two
    > nested scopes: locals and globals (and beyond this, built-ins).
    > Therefore, nested functions have only a limited use. This was a
    > deliberate decision, based upon experience with languages allowing
    > arbitraries nesting such as Pascal and both Algols -- code with too
    > many nested scopes is about as readable as code with too many GOTOs.
    > And, of course, in Python, the "proper" way is to use an
    > object-oriented programming style"
    >
    > This sounds reasonable to me although nested scope always struck me as

    more
    > natural and intuitive ('Don't be surprising')
    >
    > I was wondering how the balance changed in favour of nested scope? What
    > changed people's minds about 'readability' vs other factors? Are nested
    > scopes still regarded as leading to spagetti code etc?
    >
    > (On a side note is there any way to call a nested function from outside

    the
    > parent? I was kind of expecting nested functions to be addressable through
    > dot notation like methods are but I can see why that wouldn't be quite
    > right. This might be a better question for the tutor list...)
    >
    > Andy Baker
    >
     
    Larry Bates, Aug 3, 2004
    #3
  4. Andy Baker

    Peter Otten Guest

    Andy Baker wrote:

    > (On a side note is there any way to call a nested function from outside
    > the parent? I was kind of expecting nested functions to be addressable
    > through dot notation like methods are but I can see why that wouldn't be
    > quite right. This might be a better question for the tutor list...)


    When you are nesting functions, you don't get one function sitting inside
    another function. Instead, the function creation code of the "inner"
    function is executed every time to the effect that you get a new inner
    function every time you call the outer one:

    >>> def make():

    .... def f(): return "shoobidoo"
    .... return f
    ....
    >>> f1 = make()
    >>> f2 = make()
    >>> f1(), f2()

    ('shoobidoo', 'shoobidoo')
    >>> f1 is f2

    False

    You wouldn't do that in cases like the above, when you get nothing in return
    for the extra overhead, but somtimes nesting is useful - because of the
    change in the scoping rules you now get readonly-closures:

    >>> def make(s):

    .... def f(): return s
    .... return f
    ....
    >>> f1 = make("now")
    >>> f2 = make("what")
    >>> f1(), f2()

    ('now', 'what')

    Peter
     
    Peter Otten, Aug 3, 2004
    #4
  5. Andy Baker

    Nigel Rowe Guest

    Re: Overhead of (was Reasoning behind) nested scope

    Peter Otten wrote:

    > Andy Baker wrote:
    >
    >> (On a side note is there any way to call a nested function from outside
    >> the parent? I was kind of expecting nested functions to be addressable
    >> through dot notation like methods are but I can see why that wouldn't be
    >> quite right. This might be a better question for the tutor list...)

    >
    > When you are nesting functions, you don't get one function sitting inside
    > another function. Instead, the function creation code of the "inner"
    > function is executed every time to the effect that you get a new inner
    > function every time you call the outer one:
    >
    >>>> def make():

    > ... def f(): return "shoobidoo"
    > ... return f
    > ...
    >>>> f1 = make()
    >>>> f2 = make()
    >>>> f1(), f2()

    > ('shoobidoo', 'shoobidoo')
    >>>> f1 is f2

    > False
    >
    > You wouldn't do that in cases like the above, when you get nothing in
    > return for the extra overhead, but somtimes nesting is useful - because of
    > the change in the scoping rules you now get readonly-closures:
    >
    >>>> def make(s):

    > ... def f(): return s
    > ... return f
    > ...
    >>>> f1 = make("now")
    >>>> f2 = make("what")
    >>>> f1(), f2()

    > ('now', 'what')
    >
    > Peter


    What work is actually done when the
    'nested function creation code of the "inner" function'
    is executed?

    Given a quick test:-
    <code>
    def outer():
    def inner():
    pass
    return inner

    b1 = outer()
    b2 = outer()

    attrs=[a for a in dir(b1) if not a.startswith('_')]
    for a, a1, a2 in zip(attrs,
    [getattr(b1,a) for a in attrs],
    [getattr(b2,a) for a in attrs]):
    print a, a1 is a2

    </code>
    <result>
    func_closure True
    func_code True
    func_defaults True
    func_dict True
    func_doc True
    func_globals True
    func_name True
    </result>

    it appears that all the components of the inner function are the same, which
    just leaves the binding of the code object to 'inner'.

    Am I missing something, or is the overhead no worse than, say, foo=self.foo,
    where self.foo is a method?


    --
    Nigel Rowe
    A pox upon the spammers that make me write my address like..
    rho (snail) swiftdsl (stop) com (stop) au
     
    Nigel Rowe, Aug 4, 2004
    #5
  6. Andy Baker

    Peter Otten Guest

    Re: Overhead of (was Reasoning behind) nested scope

    Nigel Rowe wrote:

    > Peter Otten wrote:
    >
    >> Andy Baker wrote:
    >>
    >>> (On a side note is there any way to call a nested function from outside
    >>> the parent? I was kind of expecting nested functions to be addressable
    >>> through dot notation like methods are but I can see why that wouldn't be
    >>> quite right. This might be a better question for the tutor list...)

    >>
    >> When you are nesting functions, you don't get one function sitting inside
    >> another function. Instead, the function creation code of the "inner"
    >> function is executed every time to the effect that you get a new inner
    >> function every time you call the outer one:
    >>
    >>>>> def make():

    >> ... def f(): return "shoobidoo"
    >> ... return f
    >> ...
    >>>>> f1 = make()
    >>>>> f2 = make()
    >>>>> f1(), f2()

    >> ('shoobidoo', 'shoobidoo')
    >>>>> f1 is f2

    >> False
    >>
    >> You wouldn't do that in cases like the above, when you get nothing in
    >> return for the extra overhead, but somtimes nesting is useful - because
    >> of the change in the scoping rules you now get readonly-closures:
    >>
    >>>>> def make(s):

    >> ... def f(): return s
    >> ... return f
    >> ...
    >>>>> f1 = make("now")
    >>>>> f2 = make("what")
    >>>>> f1(), f2()

    >> ('now', 'what')
    >>
    >> Peter

    >
    > What work is actually done when the
    > 'nested function creation code of the "inner" function'
    > is executed?
    >
    > Given a quick test:-
    > <code>
    > def outer():
    > def inner():
    > pass
    > return inner
    >
    > b1 = outer()
    > b2 = outer()
    >
    > attrs=[a for a in dir(b1) if not a.startswith('_')]
    > for a, a1, a2 in zip(attrs,
    > [getattr(b1,a) for a in attrs],
    > [getattr(b2,a) for a in attrs]):
    > print a, a1 is a2
    >
    > </code>
    > <result>
    > func_closure True
    > func_code True
    > func_defaults True
    > func_dict True
    > func_doc True
    > func_globals True
    > func_name True
    > </result>
    >
    > it appears that all the components of the inner function are the same,
    > which just leaves the binding of the code object to 'inner'.


    Not the binding, a new function object is created every time outer is run,
    i. e. (faked, but I did it for real above with f1 and f2)

    >>> b1 is b2

    False

    > Am I missing something, or is the overhead no worse than, say,
    > foo=self.foo, where self.foo is a method?


    If it were pure Python it would rather be something like

    foo = Function(func_closure=..., func_code=..., ...)

    Another perspective is to look at the byte code:

    >>> import dis
    >>> def outer():

    .... def inner(): pass
    .... return inner
    ....
    >>> dis.dis(outer)

    2 0 LOAD_CONST 1 (<code object inner at
    0x4028eee0, file "<stdin>", line 2>)
    3 MAKE_FUNCTION 0
    6 STORE_FAST 0 (inner)

    3 9 LOAD_FAST 0 (inner)
    12 RETURN_VALUE
    13 LOAD_CONST 0 (None)
    16 RETURN_VALUE
    >>> def inner():

    .... pass
    ....
    >>> def outer2():

    .... return inner
    ....
    >>> dis.dis(outer2)

    2 0 LOAD_GLOBAL 0 (inner)
    3 RETURN_VALUE
    4 LOAD_CONST 0 (None)
    7 RETURN_VALUE

    In the example you get three extra instructions, LOAD_CONST, MAKE_FUNCTION
    and STORE_FAST (and earn the slight benefit that inner is now a local
    instead of a global). A constant code object is used, meaning compilation
    takes place at most once when the module is loaded. There is a dedicated
    op-code, so function creation should be faster than "normal" creation of a
    class instance.

    Now this is all nice and dandy, but how do the two contenders perform?

    $ timeit.py -s"def inner(): pass" -s"def outer(): return inner" "outer()"
    1000000 loops, best of 3: 0.469 usec per loop
    $ timeit.py -s"def outer():" -s" def inner(): pass" -s" return inner"
    "outer()"
    1000000 loops, best of 3: 1.12 usec per loop

    i. e. nesting the two functions roughly doubles execution time.
    However, creation of an inner function often will only take a small fraction
    of the total time spent in the outer function - in the end it's just a
    matter of style.

    I use inner functions only when they depend on the local context because I
    think it nicely discriminates closures from helpers. I tend to omit
    implicit parameters even for helper funtions, therefore nesting them would
    gain me nothing.

    Peter
     
    Peter Otten, Aug 4, 2004
    #6
  7. Andy Baker

    Nigel Rowe Guest

    Re: Overhead of (was Reasoning behind) nested scope

    Peter Otten wrote:

    > Nigel Rowe wrote:
    >
    >> Peter Otten wrote:
    >>
    >>> Andy Baker wrote:
    >>>

    >


    <snip>

    >> What work is actually done when the
    >> 'nested function creation code of the "inner" function'
    >> is executed?


    < snip >

    >
    > In the example you get three extra instructions, LOAD_CONST, MAKE_FUNCTION
    > and STORE_FAST (and earn the slight benefit that inner is now a local
    > instead of a global). A constant code object is used, meaning compilation
    > takes place at most once when the module is loaded. There is a dedicated
    > op-code, so function creation should be faster than "normal" creation of a
    > class instance.
    >
    > Now this is all nice and dandy, but how do the two contenders perform?
    >
    > $ timeit.py -s"def inner(): pass" -s"def outer(): return inner" "outer()"
    > 1000000 loops, best of 3: 0.469 usec per loop
    > $ timeit.py -s"def outer():" -s" def inner(): pass" -s" return inner"
    > "outer()"
    > 1000000 loops, best of 3: 1.12 usec per loop
    >
    > i. e. nesting the two functions roughly doubles execution time.
    > However, creation of an inner function often will only take a small
    > fraction of the total time spent in the outer function - in the end it's
    > just a matter of style.
    >
    > I use inner functions only when they depend on the local context because I
    > think it nicely discriminates closures from helpers. I tend to omit
    > implicit parameters even for helper funtions, therefore nesting them would
    > gain me nothing.
    >
    > Peter


    Thanks Peter, I didn't know about timeit.py (it's now on my $PATH).

    I don't think I'm going to worry about the overhead (except in deeply nested
    loops). From some quick testing it looks like the overhead is about the
    same as 1.5 times
    a=b[10:20] # where b="the quick brown jox jumps over the lazy dog"
    and less than 1/2 of
    a=math.sin(0)

    So if the nested code is easier to read, in it goes.
    --
    Nigel Rowe
    A pox upon the spammers that make me write my address like..
    rho (snail) swiftdsl (stop) com (stop) au
     
    Nigel Rowe, Aug 5, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. blu4899
    Replies:
    4
    Views:
    1,750
    Bob Foster
    Oct 31, 2003
  2. Rick Spencer

    what's wrong with my popen reasoning?

    Rick Spencer, Feb 5, 2006, in forum: Python
    Replies:
    1
    Views:
    343
    Rick Spencer
    Feb 5, 2006
  3. GHUM
    Replies:
    1
    Views:
    266
    Marc 'BlackJack' Rintsch
    Oct 17, 2008
  4. Mark

    reasoning for a macro

    Mark, May 28, 2010, in forum: C Programming
    Replies:
    14
    Views:
    549
  5. small Pox
    Replies:
    5
    Views:
    268
    small Pox
    Dec 5, 2010
Loading...

Share This Page