Is this a bug, or is it me?

[cptnwillard]

Hello all,
For some reason, the following does not work :


class C:
TYPES = [None]
DICT = {}
for Type in TYPES:
DICT.update((E,Type) for E in [1])


What do you think? Is this a bug?

 

[cokofreedom]

Hello all,
For some reason, the following does not work :

class C:
TYPES = [None]
DICT = {}
for Type in TYPES:
DICT.update((E,Type) for E in [1])

What do you think? Is this a bug?

Do not use those names...really poor choice...

thingydingy = [None]
my = {}
for dingaling in thingydingy:
my.update([(E,dingaling ) for E in [1]])
print my

 

[cptnwillard]

Do not use those names...really poor choice...

This is not the way it looks it my code. I simplified it, with generic
names, in order to point out something that does not work... The only
question here is why?

 

[Neil Cerutti]

Hello all,
For some reason, the following does not work :


class C:
TYPES = [None]
DICT = {}
for Type in TYPES:
DICT.update((E,Type) for E in [1])


What do you think? Is this a bug?

You cannot access a class's class variables in it's class-statement
scope, since the name of the type is not bound until after the class
statement is completed.

Try:

class C:
TYPES = [None]
DICT = {}

for Type in C.TYPES:
C.DICT.update((E, Type) for E in [1])

 

[Neil Cerutti]

You cannot access a class's class variables in it's class-statement
scope, since the name of the type is not bound until after the class
statement is completed.

Arrgh! I hate making the "its" error. But I wanted to add that this
property of Python's class statement bothers me only when I think it
would be good to use class variables as default method argument
values. Luckily, I don't think very often.

 

[cptnwillard]

You cannot access a class's class variables in it's class-statement
scope, since the name of the type is not bound until after the class
statement is completed.

Thanks for the answer, but then why is there no error with the
variable 'TYPES'? This one is accessed first...

 

[Hrvoje Niksic]

You cannot access a class's class variables in it's class-statement
scope, since the name of the type is not bound until after the class
statement is completed.

But they are still available as locals, so you can access them using
their names, like this:
.... a = 1
.... b = 2
.... print a+b
....
3

The question is, why doesn't the OP's code snippet work? It seems
that the generator expression can't read the surrounding locals().
But when you change the generator expression to a list comprehension
using a pair of [] around it, it starts working. Compare:

class C(object):
d = {}
for a in 1, 2, 3:
ignore = list((a, b) for b in (4, 5, 6))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in C
File "<stdin>", line 4, in <genexpr>
NameError: global name 'a' is not defined

with:

class C(object):
d = {}
for a in 1, 2, 3:
ignore = [(a, b) for b in (4, 5, 6)]

It seems that generator expressions and list comprehensions have
subtly different scoping rules. Whether this is a bug or not is a
good question.

 

[Hrvoje Niksic]

Hello all,
For some reason, the following does not work :


class C:
TYPES = [None]
DICT = {}
for Type in TYPES:
DICT.update((E,Type) for E in [1])


What do you think? Is this a bug?

It works if you change the generator expression to a list
comprehension, by adding [] around it. Feels like a bug to me, but
there might be subtleties involved.

 

[Peter Otten]

Hello all,
For some reason, the following does not work :


class C:
TYPES = [None]
DICT = {}
for Type in TYPES:
DICT.update((E,Type) for E in [1])


What do you think? Is this a bug?

Here is a simpler example:
.... a = 42
.... list(a for _ in "a")
....
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in A
File "<stdin>", line 3, in <genexpr>
NameError: global name 'a' is not defined

The equivalent code using a list comprehension instead of the generator
expression works without complaint:
.... a = 42
.... [a for _ in "a"]
....
So it seems that Python gets puzzled by the extra scope introduced by the
genexp, i. e. you are seeing an obscure variant of the following
(expected) behaviour:
.... a = 42
.... def f(): a
.... f()
....
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in B
File "<stdin>", line 3, in f
NameError: global name 'a' is not defined

I think you should file a bug report, though making the genexp recognizing
the class scope probably isn't worth the effort.

Peter

 

[Neil Cerutti]

But they are still available as locals, so you can access them using
their names, like this:

... a = 1
... b = 2
... print a+b
...
3

Thanks! I had never tried that before. That actually solved my default
method argument problem.

The question is, why doesn't the OP's code snippet work? It seems
that the generator expression can't read the surrounding locals().
But when you change the generator expression to a list comprehension
using a pair of [] around it, it starts working. Compare:

class C(object):
d = {}
for a in 1, 2, 3:
ignore = list((a, b) for b in (4, 5, 6))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in C
File "<stdin>", line 4, in <genexpr>
NameError: global name 'a' is not defined

with:

class C(object):
d = {}
for a in 1, 2, 3:
ignore = [(a, b) for b in (4, 5, 6)]

It seems that generator expressions and list comprehensions have
subtly different scoping rules. Whether this is a bug or not is a
good question.

Generator expressions, unlike list comprehensions, have their own
scope so that they don't "leak" names to the enclosing scope. The
Python rule forbidding access to the locals of enclosing scopes is
preventing the class names from working in the generator expression.

 

[cokofreedom]

But they are still available as locals, so you can access them using
their names, like this:
... a = 1
... b = 2
... print a+b
...
3

Thanks! I had never tried that before. That actually solved my default
method argument problem.

The question is, why doesn't the OP's code snippet work? It seems
that the generator expression can't read the surrounding locals().
But when you change the generator expression to a list comprehension
using a pair of [] around it, it starts working. Compare:

class C(object):
d = {}
for a in 1, 2, 3:
ignore = list((a, b) for b in (4, 5, 6))

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in C
File "<stdin>", line 4, in <genexpr>
NameError: global name 'a' is not defined

with:

class C(object):
d = {}
for a in 1, 2, 3:
ignore = [(a, b) for b in (4, 5, 6)]

It seems that generator expressions and list comprehensions have
subtly different scoping rules. Whether this is a bug or not is a
good question.

Generator expressions, unlike list comprehensions, have their own
scope so that they don't "leak" names to the enclosing scope. The
Python rule forbidding access to the locals of enclosing scopes is
preventing the class names from working in the generator expression.

Ah, that explains why my random suggestion worked but was not
helpful I feel like I am learning a lot today!

 

[Neil Cerutti]

Ah, that explains why my random suggestion worked but was not
helpful I feel like I am learning a lot today!

Me, too. Especially about wrongly oversimplifying stuff that's
complicated. My statement above isn't exactly right, since generator
expressions usually *can* access stuff in the enclosing local scope.
They wouldn't be very useful if they couldn't.

 

[cptnwillard]

I filed a bug report, and here is the short answer to my question:
genexps are code blocks, and code blocks cannot see variables in class
scopes. Congrats to Neil Cerutti who figured it out.

Now here is another one for your enjoyment:

class C:
@staticmethod
def f1(): pass
F = { '1' : f1 }

C().F['1']()


What do you think of this one?

 

[Neil Cerutti]

Now here is another one for your enjoyment:

class C:
@staticmethod
def f1(): pass
F = { '1' : f1 }

C().F['1']()


What do you think of this one?

I'll get the ball rolling again, and hopefully it won't roll over me.

The decoration is setting the class type's f1 attribute correctly, but
doing something strange in the local namespace.
.... @staticmethod
.... def f1(): pass
.... print f1
....
<function f1 at 0x00A60830>

The class statement's local namespace is pretty strange. I think I
mightl go back to pretending there isn't one.

 

[Ross Ridge]

The decoration is setting the class type's f1 attribute correctly, but
doing something strange in the local namespace.

... @staticmethod
... def f1(): pass
... print f1
...

<function f1 at 0x00A60830>

It might help understand the problem if you do the something similar
without using @staticmethod:

class C:
def f1(): pass
print "f1", f1

print "C.f1", C.f1
print "C().f1", C().f1

You should see output something like:

f1 <function f1 at 0x2ac08d6377d0>
C.f1 <unbound method C.f1>

The class statement's local namespace is pretty strange. I think I
mightl go back to pretending there isn't one.

When class attributes are referenced, functions are turned into unbound
methods and staticmethod objects get turned into functions.

Something like the following should work:

class C:
def f1(): pass
F = {'1': f1}
f1 = staticmethod(f1) # if you need C.f1() to work as well

If you don't need C.f1() to work you can replace the last line with
"del f1".

Ross Ridge

 

[Neil Cerutti]

When class attributes are referenced, functions are turned into unbound
methods and staticmethod objects get turned into functions.

OK, thank you. So the local namespace is just fine, you just can't
call the staticmethod using that name directly.

 

[Peter Otten]

I filed a bug report, and here is the short answer to my question:
genexps are code blocks, and code blocks cannot see variables in class
scopes. Congrats to Neil Cerutti who figured it out.

Now here is another one for your enjoyment:

class C:
@staticmethod
def f1(): pass
F = { '1' : f1 }

C().F['1']()


What do you think of this one?

If you want it to be callable you can subclass:
.... def __call__(self, *args, **kw):
.... return self.__get__(object)(*args, **kw)
.... .... @static
.... def f(x="yadda"): print x
.... f()
....
yaddayadda

Peter

 

[Steven D'Aprano]

Now here is another one for your enjoyment:

class C:
@staticmethod
def f1(): pass
F = { '1' : f1 }

C().F['1']()


What do you think of this one?

The trick is to realise that the object stored in the dict is not the
same object that you get back when you call C.f1().

C().f1 is C().F['1'] False
C().f1

C().F['1']

<staticmethod object at 0xb7ee57dc>


What you've done in inadvertently bypassed Python's method-access
mechanism:

C.__dict__['f1'] is C().f1 False
C.__dict__['f1'] is C().F['1']

True


Analogous results will occur without the decorator (although the error
you get is different):
.... def foo(self): pass
.... F = {'oo': foo}
....

D().foo()
D.F['oo']()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() takes exactly 1 argument (0 given)


So... not a bug, a feature

 

[Arnaud Delobelle]

I filed a bug report, and here is the short answer to my question:
genexps are code blocks, and code blocks cannot see variables in class
scopes. Congrats to Neil Cerutti who figured it out.

Now here is another one for your enjoyment:

class C:
        @staticmethod
        def f1(): pass
        F = { '1' : f1 }

C().F['1']()

What do you think of this one?

It's normal, the staticmethod decorator's role is to 'flag up' the
method as static for further processing when type(C).__new__ is
called.

 

[Arnaud Delobelle]

Here is a simpler example:

...     a = 42
...     list(a for _ in "a")
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in A
  File "<stdin>", line 3, in <genexpr>
NameError: global name 'a' is not defined [...]
So it seems that Python gets puzzled by the extra scope introduced by the
genexp, i. e. you are seeing an obscure variant of the following
(expected) behaviour:

...     a = 42
...     def f(): a
...     f()
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in B
  File "<stdin>", line 3, in f
NameError: global name 'a' is not defined

This is exactly the problem:

class A:
a = 42
list(a for _ in "a")

is in fact compiled as:

class A:
a = 42
def _genexpr():
for _ in "a":
yield a
list(_genexpr())

Except that the _genexpr function is not bound to any name, just left
... a = 42
... def _genexpr():
... for _ in "a":
... yield a
... list(_genexpr())
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in A

I think you should file a bug report, though making the genexp recognizing
the class scope probably isn't worth the effort.

It isn't necessary to do that to fix the problem. I think that the
semantics of genexprs concerning free variables are confusing (and
this is a good example!). I suggested a change on python-ideas a
while ago that would solve this particular issue.

In short, all free variables inside a generator expression could be
bound at the creation of the expression, not when it is evaluated.

So for example

((c, f(x)) for x in L)

would be the equivalent of what one could currently write as.

(lambda _c, _f: ((c, f(x) for x in L))(c, f)

so your example would compile to:

class A:
a = 42
def _genexpr(_a):
for _ in "a":
yield _a
list(_genexpr(a))

Which would behave as the OP expected (and I think as it is reasonable
to expect if one doesn't know about how genexprs are implemented).

Other reasons why I think this is desirable are exposed (maybe not
very convincingly) in this post to python-ideas.

http://mail.python.org/pipermail/python-ideas/2007-December/001260.html