How do functions get access to builtins?

Discussion in 'Python' started by Steven D'Aprano, Jan 20, 2013.

  1. I've been playing around with ChainedMap in Python 3.3, and run into
    something which perplexes me. Let's start with an ordinary function that
    accesses one global and one builtin.


    x = 42
    def f():
    print(x)


    If you call f(), it works as expected. But let's make a version with no
    access to builtins, and watch it break:

    from types import FunctionType
    g = FunctionType(f.__code__, {'x': 23})


    If you call g(), you get an exception:

    py> g()
    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "<stdin>", line 2, in f
    NameError: global name 'print' is not defined


    (Don't be fooled by the traceback referring to "f" rather than g.
    That's because g's code was copied from f.)

    We can add support for builtins:

    import builtins # use __builtin__ with no "s" in Python 2
    g.__globals__['__builtins__'] = builtins # Note the "s" in the key.


    and now calling g() prints 23, as expected.

    Now let me try the same thing using Python 3.3's ChainMap. Unfortunately,
    functions insist that their __global__ is a dict, so we fool it into
    accepting a ChainMap with some multiple inheritance trickery:


    from collections import ChainMap
    class ChainedDict(ChainMap, dict):
    pass

    d = ChainedDict({}, {'x': 23}, {'x': 42})
    assert d['x'] == 23
    g = FunctionType(f.__code__, d)


    As before, calling g() raises NameError, "global name 'print' is not
    defined". So I expected to be able to fix it just as I did before:

    g.__globals__['__builtins__'] = builtins


    But it doesn't work -- I still get the same NameError. Why does this not
    work here, when it works for a regular dict?


    I can fix it by adding the builtins into the ChainMap:

    g.__globals__.maps.append(builtins.__dict__)


    And now calling g() prints 23 as expected.



    --
    Steven
    Steven D'Aprano, Jan 20, 2013
    #1
    1. Advertising

  2. On 01/19/2013 09:59 PM, Steven D'Aprano wrote:
    > I've been playing around with ChainedMap in Python 3.3, and run into
    > something which perplexes me. Let's start with an ordinary function that
    > accesses one global and one builtin.
    >
    >
    > x = 42
    > def f():
    > print(x)
    >
    >
    > If you call f(), it works as expected. But let's make a version with no
    > access to builtins, and watch it break:
    >
    > from types import FunctionType
    > g = FunctionType(f.__code__, {'x': 23})
    >
    >
    > If you call g(), you get an exception:
    >
    > py> g()
    > Traceback (most recent call last):
    > File "<stdin>", line 1, in <module>
    > File "<stdin>", line 2, in f
    > NameError: global name 'print' is not defined
    >
    >
    > (Don't be fooled by the traceback referring to "f" rather than g.
    > That's because g's code was copied from f.)
    >
    > We can add support for builtins:
    >
    > import builtins # use __builtin__ with no "s" in Python 2
    > g.__globals__['__builtins__'] = builtins # Note the "s" in the key.
    >
    >
    > and now calling g() prints 23, as expected.
    >
    > Now let me try the same thing using Python 3.3's ChainMap. Unfortunately,
    > functions insist that their __global__ is a dict, so we fool it into
    > accepting a ChainMap with some multiple inheritance trickery:
    >
    >
    > from collections import ChainMap
    > class ChainedDict(ChainMap, dict):
    > pass
    >
    > d = ChainedDict({}, {'x': 23}, {'x': 42})
    > assert d['x'] == 23
    > g = FunctionType(f.__code__, d)
    >
    >
    > As before, calling g() raises NameError, "global name 'print' is not
    > defined". So I expected to be able to fix it just as I did before:
    >
    > g.__globals__['__builtins__'] = builtins
    >
    >
    > But it doesn't work -- I still get the same NameError. Why does this not
    > work here, when it works for a regular dict?
    >


    I found the answer in Python's source code. When you execute a code
    object, PyFrame_New is called which gets 'bultins' from 'globals', but
    inside PyFrame_New (defined on line 596 of Objects/frameobject.c) is the
    following (line 613):

    builtins = PyDict_GetItem(globals, builtin_object);

    Unlike PyObject_GetItem, PyDict_GetItem is specialized for dict objects.
    Your ChainedDict class uses ChainMaps's storage and leaves dict's
    storage empty, so PyDict_GetItem doesn't find anything.


    > I can fix it by adding the builtins into the ChainMap:
    >
    > g.__globals__.maps.append(builtins.__dict__)
    >
    >
    > And now calling g() prints 23 as expected.
    >


    The reason this works is unlike PyFrame_New, the LOAD_GLOBAL opcode
    first checks if globals' type is dict (and not a subclass), and falls
    back to using PyObject_GetItem if it's anything else.


    Interestingly: it looks like it could be fixed easily enough. Unless
    there are other places where globals is assumed to be a dict object, it
    would just be a matter of doing the same check and fallback in
    PyFrame_New that is done in LOAD_GLOBAL (technically, you could just use
    PyObject_GetItem; obviously, this is an optimization).
    Rouslan Korneychuk, Jan 21, 2013
    #2
    1. Advertising

  3. Rouslan Korneychuk wrote:

    > I found the answer in Python's source code. When you execute a code
    > object, PyFrame_New is called which gets 'bultins' from 'globals', but
    > inside PyFrame_New (defined on line 596 of Objects/frameobject.c) is the
    > following (line 613):
    >
    > builtins = PyDict_GetItem(globals, builtin_object);
    >
    > Unlike PyObject_GetItem, PyDict_GetItem is specialized for dict objects.
    > Your ChainedDict class uses ChainMaps's storage and leaves dict's
    > storage empty, so PyDict_GetItem doesn't find anything.

    [...]
    > Interestingly: it looks like it could be fixed easily enough. Unless
    > there are other places where globals is assumed to be a dict object, it
    > would just be a matter of doing the same check and fallback in
    > PyFrame_New that is done in LOAD_GLOBAL (technically, you could just use
    > PyObject_GetItem; obviously, this is an optimization).



    Thanks for the reply Rouslan.

    Perhaps I should report this as a bug.




    --
    Steven
    Steven D'Aprano, Jan 24, 2013
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Sean 'Shaleh' Perry

    Re: Least used builtins?

    Sean 'Shaleh' Perry, Jul 4, 2003, in forum: Python
    Replies:
    2
    Views:
    315
    Lulu of the Lotus-Eaters
    Jul 5, 2003
  2. Raymond Hettinger
    Replies:
    8
    Views:
    295
    Daniel 'Dang' Griffith
    Apr 21, 2004
  3. Raymond Hettinger
    Replies:
    0
    Views:
    296
    Raymond Hettinger
    Apr 19, 2004
  4. Kamilche
    Replies:
    5
    Views:
    331
    Kamilche
    Jun 10, 2004
  5. Kenneth McDonald
    Replies:
    2
    Views:
    278
    Russell Blau
    Jun 16, 2004
Loading...

Share This Page