surprising behaviour of global dictionaries

Discussion in 'Python' started by Michele Simionato, Oct 9, 2012.

  1. I have the following module implementing a registry of functions with a decorator:

    $ cat x.py
    registry = {} # global dictionary

    def dec(func):
    registry[func.__name__] = func
    print registry, id(registry)
    return func

    if __name__ == '__main__':
    import xlib
    print registry, id(registry)

    The library xlib just defines two dummy functions:

    $ cat xlib.py
    from x import dec

    @dec
    def f1():
    pass

    @dec
    def f2():
    pass

    Then I get the following output:

    $ python x.py
    {'f1': <function f1 at 0x7f7bce0cd668>} 27920352
    {'f1': <function f1 at 0x7f7bce0cd668>, 'f2': <function f2 at 0x7f7bce0cd6e0>} 27920352
    {} 27395472

    This is surprising since I would expect to have a single global dictionary, not two: how comes the registry inside the ``if __name__ == '__main__'`` block is different from the one seen in the library?

    This is python 2.7.3 on Ubuntu.
     
    Michele Simionato, Oct 9, 2012
    #1
    1. Advertising

  2. Michele Simionato

    Peter Otten Guest

    Michele Simionato wrote:

    > I have the following module implementing a registry of functions with a
    > decorator:
    >
    > $ cat x.py
    > registry = {} # global dictionary
    >
    > def dec(func):
    > registry[func.__name__] = func
    > print registry, id(registry)
    > return func
    >
    > if __name__ == '__main__':
    > import xlib
    > print registry, id(registry)
    >
    > The library xlib just defines two dummy functions:
    >
    > $ cat xlib.py
    > from x import dec
    >
    > @dec
    > def f1():
    > pass
    >
    > @dec
    > def f2():
    > pass
    >
    > Then I get the following output:
    >
    > $ python x.py
    > {'f1': <function f1 at 0x7f7bce0cd668>} 27920352
    > {'f1': <function f1 at 0x7f7bce0cd668>, 'f2': <function f2 at
    > {0x7f7bce0cd6e0>} 27920352 } 27395472
    >
    > This is surprising since I would expect to have a single global
    > dictionary, not two: how comes the registry inside the ``if __name__ ==
    > '__main__'`` block is different from the one seen in the library?
    >
    > This is python 2.7.3 on Ubuntu.


    Welcome to python -- this is a trap every newbie falls into ;)

    Seriously, you shouldn't use the main script as a library; it is put into
    the sys.modules cache under the "__main__" key. Subsequent imports under its
    real name will not find that name in the cache and import another instance
    of the module, with puzzling effects, like

    $ cat x.py
    import x
    class A: pass
    a = A()
    assert isinstance(a, x.A)

    $ python x.py
    Traceback (most recent call last):
    File "x.py", line 4, in <module>
    assert isinstance(a, x.A)
    AssertionError
    $
     
    Peter Otten, Oct 9, 2012
    #2
    1. Advertising

  3. On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
    > Seriously, you shouldn't use the main script as a library; it is put into
    >
    > the sys.modules cache under the "__main__" key. Subsequent imports under its
    >
    > real name will not find that name in the cache and import another instance
    >
    > of the module, with puzzling effects


    Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.
     
    Michele Simionato, Oct 9, 2012
    #3
  4. On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
    > Seriously, you shouldn't use the main script as a library; it is put into
    >
    > the sys.modules cache under the "__main__" key. Subsequent imports under its
    >
    > real name will not find that name in the cache and import another instance
    >
    > of the module, with puzzling effects


    Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.
     
    Michele Simionato, Oct 9, 2012
    #4
  5. On 2012-10-09, Peter Otten <> wrote:

    > Welcome to python -- this is a trap every newbie falls into ;)
    >
    > Seriously, you shouldn't use the main script as a library;


    There must be something wrong with me. It never even occurred to me
    to try to import a file from within that same file. I don't think
    I've ever even heard of that before...

    --
    Grant Edwards grant.b.edwards Yow! The Korean War must
    at have been fun.
    gmail.com
     
    Grant Edwards, Oct 9, 2012
    #5
  6. Michele Simionato

    Dave Angel Guest

    On 10/09/2012 11:36 AM, Michele Simionato wrote:
    > On Tuesday, October 9, 2012 5:24:17 PM UTC+2, Peter Otten wrote:
    >> Seriously, you shouldn't use the main script as a library; it is put into
    >>
    >> the sys.modules cache under the "__main__" key. Subsequent imports under its
    >>
    >> real name will not find that name in the cache and import another instance
    >>
    >> of the module, with puzzling effects

    > Actually I usually never use the main script as a library, this is why I never experience this puzzling behavior before. But now it is clear, thanks.

    More generally, you should arrange your imports so that there are no
    cycles. If module/script "a" imports "b", directly or indirectly, "b"
    should not try to import "a." Move the common code to a third place,
    and import it from both "a" and from "b".

    The other symptoms you can get are more subtle than this one, but just
    as surprising.


    --

    DaveA
     
    Dave Angel, Oct 9, 2012
    #6
  7. Michele Simionato

    Peter Otten Guest

    Grant Edwards wrote:

    > On 2012-10-09, Peter Otten <> wrote:
    >
    >> Welcome to python -- this is a trap every newbie falls into ;)
    >>
    >> Seriously, you shouldn't use the main script as a library;

    >
    > There must be something wrong with me. It never even occurred to me
    > to try to import a file from within that same file.


    It is typically done in two steps:

    (1) Write module x, use it in module y.
    (2) For convenience add "if __name__ == '__main__'" and import module y.

    Hilarity ensues.

    > I don't think I've ever even heard of that before...


    As I was poking fun at Michele who really is an expert and likely knows more
    about Python than I do I may have exaggerated a bit ;)

    But it does come up, see

    "Singleton implementation problems"
    http://mail.python.org/pipermail/python-list/2008-July/470770.html

    "Dynamically declared shared constant/variable imported twice problem"
    http://mail.python.org/pipermail/python-list/2009-May/535619.html

    There are more, but I don't know a convenient way to find the posts.
    Related problems with reload() or paths into a package occur even more
    frequently.
     
    Peter Otten, Oct 9, 2012
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. lysdexia
    Replies:
    6
    Views:
    505
    John Machin
    Dec 2, 2007
  2. Joe P. Cool
    Replies:
    7
    Views:
    281
    Benjamin
    Jun 29, 2008
  3. Brandon
    Replies:
    12
    Views:
    491
    Brandon
    Aug 15, 2008
  4. candide
    Replies:
    15
    Views:
    527
    Stephen Hansen
    Jun 18, 2010
  5. Frank Schmitt
    Replies:
    3
    Views:
    168
    Frank Schmitt
    Dec 9, 2003
Loading...

Share This Page