How to modify local variables from internal functions?

Discussion in 'Python' started by kj, Oct 24, 2009.

  1. kj

    kj Guest

    I like Python a lot, and in fact I'm doing most of my scripting in
    Python these days, but one thing that I absolutely *****DETEST*****
    about Python is that it does allow an internal function to modify
    variables in the enclosing local scope. This willful hobbling of
    internal functions seems to me so perverse and unnecessary that it
    delayed my adoption of Python by about a decade. Just thinking
    about it brings me to the brink of blowing a gasket... I must go
    for a walk...



    OK, I'm better now.

    Anyway, I recently wanted to write a internal helper function that
    updates an internal list and returns True if, after this update,
    the list is empty, and once more I bumped against this hated
    "feature". What I wanted to write, if Python did what I wanted it
    to, was this:

    def spam():
    jobs = None
    def check_finished():
    jobs = look_for_more_jobs()
    return not jobs

    if check_finished():
    return

    process1(jobs)

    if check_finished():
    return

    process2(jobs)

    if check_finished():
    return

    process3(jobs)

    In application in question, the availability of jobs can change
    significantly over the course of the function's execution (jobs
    can expire before they are fully processed, and new ones can arise),
    hence the update-and-check prior to the calls to process1, process2,
    process3.

    But, of course, the above does not work in Python, because the jobs
    variable local to spam does not get updated by check_finished.
    Grrrr!

    I ended up implementing check_finished as this stupid-looking
    monstrosity:

    def spam():
    jobs = []
    def check_finished(jobs):
    while jobs:
    jobs.pop()
    jobs.extend(look_for_more_jobs())
    return not jobs

    if check_finished(jobs):
    return

    # etc.

    Is there some other trick to modify local variables from within
    internal functions?

    TIA!

    kynn
     
    kj, Oct 24, 2009
    #1
    1. Advertising

  2. kj

    Chris Rebert Guest

    On Fri, Oct 23, 2009 at 5:19 PM, kj <> wrote:
    > I like Python a lot, and in fact I'm doing most of my scripting in
    > Python these days, but one thing that I absolutely *****DETEST*****
    > about Python is that it does allow an internal function to modify
    > variables in the enclosing local scope.  This willful hobbling of
    > internal functions seems to me so perverse and unnecessary that it
    > delayed my adoption of Python by about a decade.  Just thinking
    > about it brings me to the brink of blowing a gasket...  I must go
    > for a walk...

    <snip>
    > Anyway, I recently wanted to write a internal helper function that
    > updates an internal list and returns True if, after this update,
    > the list is empty, and once more I bumped against this hated
    > "feature".  What I wanted to write, if Python did what I wanted it
    > to, was this:
    >
    > def spam():
    >    jobs = None
    >    def check_finished():
    >       jobs = look_for_more_jobs()
    >       return not jobs
    >
    >    if check_finished():
    >        return
    >
    >    process1(jobs)
    >
    >    if check_finished():
    >        return
    >
    >    process2(jobs)
    >
    >    if check_finished():
    >        return
    >
    >    process3(jobs)

    <snip>
    > Is there some other trick to modify local variables from within
    > internal functions?


    The `nonlocal` statement?:
    http://www.python.org/dev/peps/pep-3104/

    Cheers,
    Chris
    --
    http://blog.rebertia.com
     
    Chris Rebert, Oct 24, 2009
    #2
    1. Advertising

  3. kj

    MRAB Guest

    kj wrote:
    >
    >
    > I like Python a lot, and in fact I'm doing most of my scripting in
    > Python these days, but one thing that I absolutely *****DETEST*****
    > about Python is that it does allow an internal function to modify
    > variables in the enclosing local scope. This willful hobbling of
    > internal functions seems to me so perverse and unnecessary that it
    > delayed my adoption of Python by about a decade. Just thinking
    > about it brings me to the brink of blowing a gasket... I must go
    > for a walk...
    >
    >
    >
    > OK, I'm better now.
    >
    > Anyway, I recently wanted to write a internal helper function that
    > updates an internal list and returns True if, after this update,
    > the list is empty, and once more I bumped against this hated
    > "feature". What I wanted to write, if Python did what I wanted it
    > to, was this:
    >
    > def spam():
    > jobs = None
    > def check_finished():
    > jobs = look_for_more_jobs()
    > return not jobs
    >
    > if check_finished():
    > return
    >
    > process1(jobs)
    >
    > if check_finished():
    > return
    >
    > process2(jobs)
    >
    > if check_finished():
    > return
    >
    > process3(jobs)
    >
    > In application in question, the availability of jobs can change
    > significantly over the course of the function's execution (jobs
    > can expire before they are fully processed, and new ones can arise),
    > hence the update-and-check prior to the calls to process1, process2,
    > process3.
    >
    > But, of course, the above does not work in Python, because the jobs
    > variable local to spam does not get updated by check_finished.
    > Grrrr!
    >

    So you didn't try this?

    def spam():
    jobs = []

    def check_finished():
    jobs[:] = look_for_more_jobs()
    return not jobs

    if check_finished():
    return

    process1(jobs)

    if check_finished():
    return

    process2(jobs)

    if check_finished():
    return

    process3(jobs)


    Actually, I'd refactor and reduce it to this:

    def spam():
    for proc in [process1, process2, process3]:
    jobs = look_for_more_jobs()
    if not jobs:
    return
    proc(jobs)
     
    MRAB, Oct 24, 2009
    #3
  4. On Sat, 24 Oct 2009 00:19:12 +0000, kj wrote:

    > I like Python a lot, and in fact I'm doing most of my scripting in
    > Python these days, but one thing that I absolutely *****DETEST*****
    > about Python is that it does allow an internal function to modify
    > variables in the enclosing local scope.


    You can't rebind names in the enclosing scope (not without the nonlocal
    keyword, which I believe is introduced in Python 3.0):


    >>> def func():

    .... x = 1
    .... def inner():
    .... x = 2
    .... inner()
    .... print x
    ....
    >>> func()

    1


    However, naturally you can call methods on objects bound in the outer
    scope. It would be rather inconvenient if you couldn't access them from
    inner functions, but that's how Python was prior to the introduction of
    nested scopes in 2.1.


    Given this code:

    def f():
    L = [1, 2, 3]
    def inner():
    return L.index(1)
    return inner()

    f()


    This is what you get in Python 1.5:

    Traceback (innermost last):
    File "<stdin>", line 1, in ?
    File "<stdin>", line 5, in f
    File "<stdin>", line 4, in inner
    NameError: L


    while in current versions of Python, you get 0.

    Of course, once you allow inner functions to call methods on objects in
    the outer scope, you allow methods which mutate the object:

    >>> def f():

    .... L = []
    .... def inner():
    .... L.append(None)
    .... inner()
    .... print L
    ....
    >>> f()

    [None]

    What other behaviour would you expect?



    > This willful hobbling of
    > internal functions seems to me so perverse and unnecessary that it
    > delayed my adoption of Python by about a decade. Just thinking about it
    > brings me to the brink of blowing a gasket... I must go for a walk...


    I think you don't understand the object and assignment model of Python if
    you think this is "hobbling of internal functions".


    > Anyway, I recently wanted to write a internal helper function that
    > updates an internal list and returns True if, after this update, the
    > list is empty, and once more I bumped against this hated "feature".
    > What I wanted to write, if Python did what I wanted it to, was this:

    [...]
    > But, of course, the above does not work in Python, because the jobs
    > variable local to spam does not get updated by check_finished. Grrrr!


    Okay, now I'm completely confused. You "DETEST" (shouting was yours) that
    Python objects can be modified by inner functions, and yet here you are
    writing a function which relies on that -- and gets it wrong, because
    you're trying to rebind a name rather than modify an object.

    Others have suggested the nonlocal keyword in 3.x. Here's another
    solution:


    def spam():
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process1(jobs)
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process2(jobs)
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process3(jobs)



    which immediately suggests refactoring:


    def spam():
    for process in (process1, process2, process3):
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process(jobs)



    --
    Steven
     
    Steven D'Aprano, Oct 26, 2009
    #4
  5. On Sat, 24 Oct 2009 00:19:12 +0000, kj wrote:

    > I like Python a lot, and in fact I'm doing most of my scripting in
    > Python these days, but one thing that I absolutely *****DETEST*****
    > about Python is that it does allow an internal function to modify
    > variables in the enclosing local scope.


    You can't rebind names in the enclosing scope (not without the nonlocal
    keyword, which I believe is introduced in Python 3.0):


    >>> def func():

    .... x = 1
    .... def inner():
    .... x = 2
    .... inner()
    .... print x
    ....
    >>> func()

    1


    However, naturally you can call methods on objects bound in the outer
    scope. It would be rather inconvenient if you couldn't access them from
    inner functions, but that's how Python was prior to the introduction of
    nested scopes in 2.1.


    Given this code:

    def f():
    L = [1, 2, 3]
    def inner():
    return L.index(1)
    return inner()

    f()


    This is what you get in Python 1.5:

    Traceback (innermost last):
    File "<stdin>", line 1, in ?
    File "<stdin>", line 5, in f
    File "<stdin>", line 4, in inner
    NameError: L


    while in current versions of Python, you get 0.

    Of course, once you allow inner functions to call methods on objects in
    the outer scope, you allow methods which mutate the object:

    >>> def f():

    .... L = []
    .... def inner():
    .... L.append(None)
    .... inner()
    .... print L
    ....
    >>> f()

    [None]

    What other behaviour would you expect?



    > This willful hobbling of
    > internal functions seems to me so perverse and unnecessary that it
    > delayed my adoption of Python by about a decade. Just thinking about it
    > brings me to the brink of blowing a gasket... I must go for a walk...


    I think you don't understand the object and assignment model of Python if
    you think this is "hobbling of internal functions".


    > Anyway, I recently wanted to write a internal helper function that
    > updates an internal list and returns True if, after this update, the
    > list is empty, and once more I bumped against this hated "feature".
    > What I wanted to write, if Python did what I wanted it to, was this:

    [...]
    > But, of course, the above does not work in Python, because the jobs
    > variable local to spam does not get updated by check_finished. Grrrr!


    Okay, now I'm completely confused. You "DETEST" (shouting was yours) that
    Python objects can be modified by inner functions, and yet here you are
    writing a function which relies on that -- and gets it wrong, because
    you're trying to rebind a name rather than modify an object.

    Others have suggested the nonlocal keyword in 3.x. Here's another
    solution:


    def spam():
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process1(jobs)
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process2(jobs)
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process3(jobs)



    which immediately suggests refactoring:


    def spam():
    for process in (process1, process2, process3):
    jobs = look_for_more_jobs()
    if not jobs:
    return
    process(jobs)





    --
    Steven
     
    Steven D'Aprano, Oct 26, 2009
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Razvan
    Replies:
    11
    Views:
    938
    Andrew Thompson
    Jul 17, 2004
  2. The Colonel
    Replies:
    6
    Views:
    4,444
    Hans Kesting
    Jun 7, 2006
  3. Daniel Vallstrom
    Replies:
    2
    Views:
    2,004
    Kevin Bracey
    Nov 21, 2003
  4. Wolfgang
    Replies:
    1
    Views:
    278
    Bruno Desthuilliers
    Jul 17, 2006
  5. Sullivan WxPyQtKinter
    Replies:
    10
    Views:
    696
    Antoon Pardon
    Nov 8, 2007
Loading...

Share This Page