Unexpected python exception

[Richard Purdie]

I've been having problems with an unexpected exception from python which
I can summarise with the following testcase:

def A():
import __builtin__
import os

__builtin__.os = os

def B():
os.stat("/")
import os

A()
B()

which results in:

Traceback (most recent call last):
File "./test.py", line 12, in <module>
B()
File "./test.py", line 8, in B
os.stat("/")
UnboundLocalError: local variable 'os' referenced before assignment

If I remove the "import os" from B(), it works as expected.

From what I've seen, its very unusual to have something operate

"backwards" in scope in python. Can anyone explain why this happens?

Cheers,

Richard

 

[Diez B. Roggisch]

I've been having problems with an unexpected exception from python which
I can summarise with the following testcase:

def A():
import __builtin__
import os

__builtin__.os = os

def B():
os.stat("/")
import os

A()
B()

which results in:

Traceback (most recent call last):
File "./test.py", line 12, in <module>
B()
File "./test.py", line 8, in B
os.stat("/")
UnboundLocalError: local variable 'os' referenced before assignment

If I remove the "import os" from B(), it works as expected.

"backwards" in scope in python. Can anyone explain why this happens?

As the import-statement in a function/method-scope doesn't leak the
imported names into the module scope, python treats them as locals.
Which makes your code equivalent to


x = 1000

def foo():
print x
x = 10

Throws the same error. The remedy is to inform python that a specific
name belongs to global scope, using the "global"-statement.

def foo():
global x
print x
x = 10


Beware though that then of course *assigning* to x is on global level.
This shouldn't be of any difference in your case though, because of the
import-only-once-mechanics of python.

Diez

 

[Andreas Löscher]

Hi,
unfortunatley I cannot reproduce your error. Which Python Version do you
use?

The expected case in this scenario is that the exception is thrown, as
you import os in A() where it is stored in the local namespace of the
function.

I tested it with Python 2.4, 2.5 and 2.6 and in both cases an exception
is thrown.

Best,
Andreas

 

[Chris Rebert]

Em Qua 11 Nov 2009, às 03:21:55, Diez B. Roggisch escreveu:

So...it should not work

def A():
    import __builtin__
    import os
    __builtin__.os = os

A()
os.stat("/")

but it does.  Why ? B() cannot see the import, but the global level can ?

The optimization which results in the behavior in question is only
done on functions scopes, not global scope.

Cheers,
Chris

 

[Andreas Löscher]

Python searches for Variables not only in local or global scoop but also
in __builtins__. If you do something like __builtins__.os = os, than
this variable should be accessible global.

If you then write something like:
def B():
os.stat("/")
import os

Python recognises on compile time, that os is a local variable in B and
allocates memory for it. All reading or writing now goes to the local
variable and not the one in __builtins__. And the local variable has
never been assigned to a value and an Exception is thrown.

Best

 

[Richard Purdie]

As the import-statement in a function/method-scope doesn't leak the
imported names into the module scope, python treats them as locals.
Which makes your code equivalent to


x = 1000

def foo():
print x
x = 10

Aha, thanks. This makes it clear whats happening.

Throws the same error. The remedy is to inform python that a specific
name belongs to global scope, using the "global"-statement.

def foo():
global x
print x
x = 10


Beware though that then of course *assigning* to x is on global level.
This shouldn't be of any difference in your case though, because of the
import-only-once-mechanics of python.

Is there a way to make the "global x" apply to all functions without
adding it to each one?

I suspect this equates to intentionally "leaking the imported names into
the module scope"?

What I'm trying to do is to avoid having "import X" statements
everywhere by changing __builtin__. It seems my approach doesn't have
quite the same effect as a true import though.

Cheers,

Richard

 

[Chris Rebert]

Is there a way to make the "global x" apply to all functions without
adding it to each one?

Thankfully, no.

What I'm trying to do is to avoid having "import X" statements
everywhere by changing __builtin__. It seems my approach doesn't have
quite the same effect as a true import though.

And you can't just put all your imports together at the top of the
file because...?
If you're importing the same stuff across multiple modules, I'd say
you have a code smell on your hands. Injecting stuff into the builtin
namespace purely for convenience's sake is Just Evil (tm). At the
least, you can put all the imports in one module and then use `from
all_imports import *`, which is just slightly less evil.

Cheers,
Chris

 

[Ralax]

I've been having problems with an unexpected exception from python which
I can summarise with the following testcase:

def A():
    import __builtin__
    import os

    __builtin__.os = os

def B():
    os.stat("/")
    import os

A()
B()

which results in:

Traceback (most recent call last):
  File "./test.py", line 12, in <module>
    B()
  File "./test.py", line 8, in B
    os.stat("/")
UnboundLocalError: local variable 'os' referenced before assignment

If I remove the "import os" from B(), it works as expected.


"backwards" in scope in python. Can anyone explain why this happens?

Cheers,

Richard

One word, Python treat objects in a function or method-scope as
locals.
So os is not defined in B(), is it right?

 

[Richard Purdie]

Thankfully, no.

Hmm .

And you can't just put all your imports together at the top of the
file because...?

The application in question is bitbake, the parsing tool for the
OpenEmbedded and Poky projects.

We're talking about python code fragments spread across about 8,000
files which make up the "metadata" some of which is public and some of
which is not. Rightly or wrongly bitbake does some interesting things
with its execution contexts for these code fragments.

Lets just say its not a traditional use case, we know the way bitbake
does some things is evil but it does other things rather well and we
can't break those.

Cheers,

Richard

 

[Eduardo Lenz]

Em Qua 11 Nov 2009, às 03:21:55, Diez B. Roggisch escreveu:

As the import-statement in a function/method-scope doesn't leak the
imported names into the module scope, python treats them as locals.
Which makes your code equivalent to


x = 1000

def foo():
print x
x = 10

Throws the same error. The remedy is to inform python that a specific
name belongs to global scope, using the "global"-statement.

def foo():
global x
print x
x = 10


Beware though that then of course *assigning* to x is on global level.
This shouldn't be of any difference in your case though, because of the
import-only-once-mechanics of python.

Diez

So...it should not work

def A():
import __builtin__
import os
__builtin__.os = os

A()
os.stat("/")

but it does. Why ? B() cannot see the import, but the global level can ?

Thanks.

Eduardo.

 

[Gabriel Genellina]

One word, Python treat objects in a function or method-scope as
locals.

Not exactly; from the Language Reference [1]:

"The following constructs bind names: formal parameters to functions,
import statements, class and function definitions (these bind the class or
function name in the defining block), and targets that are identifiers if
occurring in an assignment, for loop header, or in the second position of
an except clause header. The import statement of the form “from ...import
*†binds all names defined in the imported module, except those beginning
with an underscore. This form may only be used at the module level. [...]
If a name binding operation occurs anywhere within a code block, all uses
of the name within the block are treated as references to the current
block. This can lead to errors when a name is used within a block before
it is bound. This rule is subtle. Python lacks declarations and allows
name binding operations to occur anywhere within a code block. The local
variables of a code block can be determined by scanning the entire text of
the block for name binding operations."

So os is not defined in B(), is it right?

"os" is a local name due to import, and starts uninitialized; using it
before it is initialized raises UnboundLocalError

[1] http://docs.python.org/reference/executionmodel.html