Asymmetry in globals __getitem__/__setitem__

[Robert Lehmann]

Hi all,

I have noticed there is a slight asymmetry in the way the interpreter
(v3.3.5, reproduced also in v3.5.x) loads and stores globals. While
loading globals from a custom mapping triggers __getitem__ just fine,
writing seems to silently ignore __setitem__.

class Namespace(dict):
def __getitem__(self, key):
print("getitem", key)
def __setitem__(self, key, value):
print("setitem", key, value)

def fun():
global x, y
x # should call globals.__getitem__
y = 1 # should call globals.__setitem__

exec(fun.__code__, Namespace())
# => getitem x

I would have expected "setitem y 1" to show up as well, but to no avail.
Am I doing something wrong? Is this on purpose?

Cheers,
Robert

PS. I found a 3.3.x commit (e3ab8aa
<http://hg.python.org/cpython/rev/e3ab8aa0216c>) which fixed the
LOAD_GLOBAL opcode to support other types than dict, but STORE_GLOBAL seems
to use bare PyDict_SetItem instead of dispatching to PyObject_SetItem.

 

[Gregory Ewing]

I have noticed there is a slight asymmetry in the way the interpreter
(v3.3.5, reproduced also in v3.5.x) loads and stores globals. While
loading globals from a custom mapping triggers __getitem__ just fine,
writing seems to silently ignore __setitem__.

I didn't think that using a custom mapping object for
globals was officially supported. Has that changed?

 

[Marko Rauhamaa]

I didn't think that using a custom mapping object for globals was
officially supported. Has that changed?

The documentation is a bit vague about it:

If only globals is provided, it must be a dictionary, which will be
used for both the global and the local variables. If globals and
locals are given, they are used for the global and local variables,
respectively. If provided, locals can be any mapping object.

Now, if you define:

class Namespace(dict):
...

A Namespace *is* a dict[ionary] in the classic OOP sense.


Marko

 

[robert]

The documentation is a bit vague about it:

If only globals is provided, it must be a dictionary, which will be
used for both the global and the local variables. If globals and
locals are given, they are used for the global and local variables,
respectively. If provided, locals can be any mapping object.

Interesting. This paragraph explicitly states "locals can be any mapping object," but that seems to be false:


class Namespace(dict):
def __getitem__(self, key):
print("getitem", key)
def __setitem__(self, key, value):
print("setitem", key, value)

def fun():
x # should call locals.__getitem__
y = 1 # should call locals.__setitem__

exec(fun.__code__, {}, Namespace())


Neither __getitem__ nor __setitem__ seem to be called on the local variables.

 

[Peter Otten]

Interesting. This paragraph explicitly states "locals can be any mapping
object," but that seems to be false:


class Namespace(dict):
def __getitem__(self, key):
print("getitem", key)
def __setitem__(self, key, value):
print("setitem", key, value)

def fun():
x # should call locals.__getitem__

No, x is a global here.

y = 1 # should call locals.__setitem__

exec(fun.__code__, {}, Namespace())


Neither __getitem__ nor __setitem__ seem to be called on the local
variables.

Accessing fun.__code__ is clever, but unfortunately the compiler produces
different bytecodes for loading/storing variables inside a function.
Compare:
.... x
.... y = 1
....2 0 LOAD_FAST 0 (x)
3 POP_TOP

3 4 LOAD_CONST 1 (1)
7 STORE_FAST 1 (y)
10 LOAD_CONST 0 (None)
13 RETURN_VALUE1 0 LOAD_NAME 0 (x)
3 POP_TOP

2 4 LOAD_CONST 0 (2)
7 STORE_NAME 1 (y)
10 LOAD_CONST 1 (None)
13 RETURN_VALUE

Only the latter works as advertised:
getitem x
setitem y 1

 

[Paul Sokolovsky]

Hello,

On Fri, 13 Jun 2014 12:53:54 +0200

[]

Accessing fun.__code__ is clever, but unfortunately the compiler
produces different bytecodes for loading/storing variables inside a
function. Compare:

Compiler produces different bytecodes, and allocates local variables on
stack (just like C) very fortunately, steps like that allowed Python to
drop moniker of "ridiculously slow language". And people should decide
what they really want - fast language which can stand against the
competition, or language with dynamicity and reflection capabilities
beyond any practical need. I make first choice any time. And then it
makes sense to just accept that any function can be JIT (or AOT)
compiled, and there's nothing to fish inside of it (but for the raw
machine code of unknown architecture).

 

[Marko Rauhamaa]

And people should decide what they really want - fast language which
can stand against the competition, or language with dynamicity and
reflection capabilities beyond any practical need. I make first choice
any time.

I'm in the latter camp, absolutely, except that I have a lot of
practical needs for much of that dynamism.

Admittedly, the topic of this thread is a bit funky. I'm wondering what
the application is: a profiler? a debugger? malware? self-awareness?

And then it makes sense to just accept that any function can be JIT
(or AOT) compiled, and there's nothing to fish inside of it (but for
the raw machine code of unknown architecture).

I've been talking about the need for effective JIT so we can get rid of
Java et co. I wouldn't dream of taking away any of Python's dynamism,
though. In particular, type annotations etc are a big no in my book.


Marko