obj.__dict__ expected behavior or bug?

Discussion in 'Python' started by Ed Young, Aug 10, 2003.

  1. Ed Young

    Ed Young Guest

    Here is an example of the behavior:
    ------- code start -----------------------------------
    #!/usr/bin/python
    #bugtest - test of class attribute initiation


    class Config:
    a = 1
    b = 2
    c = 3
    d = None
    e = None
    h = {'d' : 22, 'e' : 33}

    def __init__(self, factor):
    for attr in self.h.keys():
    self.__dict__[attr] = self.h[attr] * factor

    def moda(self):
    self.a *= 5


    c = Config(2)
    print c.a, c.b, c.c, c.d, c.e
    for attr in c.__dict__:
    print 'c.%s = %s' % (attr, c.__dict__[attr])
    print

    c.moda()
    print c.a, c.b, c.c, c.d, c.e
    for attr in c.__dict__:
    print 'c.%s = %s' % (attr, c.__dict__[attr])
    print
    ------- code ends -----------------------------------
    ------- output starts -------------------------------
    $ bugtest
    1 2 3 44 66
    c.e = 66
    c.d = 44

    5 2 3 44 66
    c.a = 5
    c.e = 66
    c.d = 44
    ------- output ends ---------------------------------
    What happened to c.a, c.b, and c.c when iterating thru
    c.__dict__ ?

    It appears that __dict__ members are not instantiated
    until they are changed.

    This precludes using __dict__ as the dictionary in
    a formatted print statement. e.g.

    print "c.a=%(a)s, c.b=%(b)s, c.c=%(c)s" % c.__dict__

    Is this a bug or expected behavior?
     
    Ed Young, Aug 10, 2003
    #1
    1. Advertising

  2. Ed Young wrote:

    > What happened to c.a, c.b, and c.c when iterating thru
    > c.__dict__ ?
    >
    > It appears that __dict__ members are not instantiated
    > until they are changed.
    >
    > This precludes using __dict__ as the dictionary in
    > a formatted print statement. e.g.
    >
    > print "c.a=%(a)s, c.b=%(b)s, c.c=%(c)s" % c.__dict__
    >
    > Is this a bug or expected behavior?


    Expected behavior. What you're missing is the general way that Python
    does attribute lookup. When c is an instance and you say c.x, Python
    looks in c's __dict__ for an 'x' entry, then it looks in c's class's
    __dict__ for an 'x' entry, then it looks (in a well-defined way) through
    c's class's base classes, if any, for an 'x' entry in their __dict__
    members.

    When you defined

    class C:
    a = ...
    b = ...

    and so on, these are all _class_ attributes. When you instantiate a C
    and then wrote self.a = ... in its methods, you instantiated _instance_
    attributes on that instance. Class attributes are analogous to static
    members/fields in other languages:

    >>> class C: # class with two class attributes

    .... a = 1
    .... b = 2
    ....
    >>> c = C()
    >>> d = C()
    >>> c.a

    1
    >>> d.a

    1
    >>> c.a = 10 # change an instance attribute
    >>> c.a

    10
    >>> d.a

    1
    >>> C.b = 20 # change a class attribute
    >>> c.b

    20
    >>> d.b

    20
    >>> C.__dict__

    {'a': 1, '__module__': '__main__', 'b': 20, '__doc__': None}
    >>> c.__dict__

    {'a': 10}
    >>> d.__dict__

    {}

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ Nobody's on nobody's side
    \__/ Florence, _Chess_
     
    Erik Max Francis, Aug 10, 2003
    #2
    1. Advertising

  3. "Ed Young
    > Here is an example of the behavior:
    > ------- code start -----------------------------------
    > #!/usr/bin/python
    > #bugtest - test of class attribute initiation
    >
    >
    > class Config:
    > a = 1
    > b = 2
    > c = 3
    > d = None
    > e = None
    > h = {'d' : 22, 'e' : 33}
    >
    > def __init__(self, factor):
    > for attr in self.h.keys():
    > self.__dict__[attr] = self.h[attr] * factor
    >
    > def moda(self):
    > self.a *= 5
    >
    >
    > c = Config(2)
    > print c.a, c.b, c.c, c.d, c.e
    > for attr in c.__dict__:
    > print 'c.%s = %s' % (attr, c.__dict__[attr])
    > print
    >
    > c.moda()
    > print c.a, c.b, c.c, c.d, c.e
    > for attr in c.__dict__:
    > print 'c.%s = %s' % (attr, c.__dict__[attr])
    > print
    > ------- code ends -----------------------------------
    > ------- output starts -------------------------------
    > $ bugtest
    > 1 2 3 44 66
    > c.e = 66
    > c.d = 44
    >
    > 5 2 3 44 66
    > c.a = 5
    > c.e = 66
    > c.d = 44
    > ------- output ends ---------------------------------
    > What happened to c.a, c.b, and c.c when iterating thru
    > c.__dict__ ?


    They are up in C.__dict__



    > This precludes using __dict__ as the dictionary in
    > a formatted print statement. e.g.
    >
    > print "c.a=%(a)s, c.b=%(b)s, c.c=%(c)s" % c.__dict__



    Not really. Use a wrapper to forward dict lookup requests
    to getattr() which knows how/where to search for attributes:

    class AttrDict:
    def __init__(self, obj):
    self.obj = obj
    def __getitem__(self, key):
    return getattr(self.obj, key)

    print "c.a=%(a)s, c.b=%(b)s, c.c=%(c)s" % AttrDict(c)


    > Is this a bug or expected behavior?


    Expected.



    Raymond Hettinger
     
    Raymond Hettinger, Aug 10, 2003
    #3
  4. Ed Young

    Sean Ross Guest

    Hi.
    It's expected behaviour.

    Let's go through your code with a few additional print statements to see if
    we can demonstrate what's happening;

    class Config:
    a = 1
    b = 2
    c = 3
    d = None
    e = None
    h = {'d' : 22, 'e' : 33}

    def __init__(self, factor):
    for attr in self.h.keys():
    self.__dict__[attr] = self.h[attr] * factor

    def moda(self):
    self.a *= 5


    c = Config(2)

    # Here's what to pay attention to ...........
    print "c.__dict__: ", c.__dict__ #
    this is the instance dictionary
    print "c.__class__.__dict__: ", c.__class__.__dict__ # this is the class
    dictionary


    print c.a, c.b, c.c, c.d, c.e
    for attr in c.__dict__:
    print 'c.%s = %s' % (attr, c.__dict__[attr])
    print

    c.moda()

    print "c.moda() --------------"
    print "c.__dict__: ", c.__dict__
    print "c.__class__.__dict__: ", c.__class__.__dict__

    print c.a, c.b, c.c, c.d, c.e
    for attr in c.__dict__:
    print 'c.%s = %s' % (attr, c.__dict__[attr])
    print



    Now, here's the output with #annotations:


    c.__dict__: {'e': 66, 'd': 44}
    c.__class__.__dict__: {'a': 1, 'moda': <function moda at 0x015209B0>,
    '__module__': '__main__', 'b': 2, 'e': None, 'd': None, 'h': {'e': 33, 'd':
    22}, 'c': 3, '__init__': <function __init__ at 0x01520B30>, '__doc__': None}
    1 2 3 44 66
    c.e = 66
    c.d = 44

    # Okay. We can see that the values for 'a', 'b', 'c' were all found
    # in the class dictionary of instance c, while 'd', and 'e' were
    # found in the instance dictionary of c. More on this later....


    # Now we're about to call moda() ....
    c.moda() --------------

    # What's changed?
    c.__dict__: {'a': 5, 'e': 66, 'd': 44}
    c.__class__.__dict__: {'a': 1, 'moda': <function moda at 0x015209B0>,
    '__module__': '__main__', 'b': 2, 'e': None, 'd': None, 'h': {'e': 33, 'd':
    22}, 'c': 3, '__init__': <function __init__ at 0x01520B30>, '__doc__': None}
    5 2 3 44 66
    c.a = 5
    c.e = 66
    c.d = 44

    # This time only 'b' and 'c''s values were pulled from instance c's class'
    dictionary.
    # What about 'a'? 'a' was pulled from c's instance dictionary. Nothing's
    changed
    # for 'd' and 'e'.


    Okay then. What's going on?

    class Config:
    a = 1
    b = 2
    c = 3
    d = None
    e = None
    h = {'d' : 22, 'e' : 33}

    The code above adds class variables a - h to the class Config. So, if you
    have an instance c of Config,
    variables a-h are stored in c's class dictionary (c.__class__.__dict__) and
    NOT c's instance dictionary
    (c.__dict__). Moving on...

    def __init__(self, factor):
    for attr in self.h.keys():
    self.__dict__[attr] = self.h[attr] * factor

    Inside the constructor, you call self.h.keys(). To find self.h, Python looks
    first in self.__dict__. But 'h' isn't there.
    Next it looks in self.__class__.__dict__. That's were 'h' is! Now this:

    self.__dict__[attr] = self.h[attr] * factor

    Here, you're assigning NEW attributes 'd' and 'e' to self's __dict__. What
    you are not doing is assigning new values to class variables 'd' and 'e' in
    self.__class__.__dict__ .


    def moda(self):
    self.a *= 5

    Something similar is happening in here. This one is a bit more complicated.

    self.a *= 5

    is the same as

    self.a = self.a * 5

    What does this really mean? Well,

    self.a = ....

    is equivalent to

    self.__dict__['a'] = ....

    But

    self.a = self.a ....

    is not necessarily equivalent to

    self.__dict__['a'] = self.__dict__['a']


    because the self.a on the right hand side of the assignment has to be looked
    up by Python. And, as we showed earlier,
    look up starts with self.__dict__. But 'a' is not yet a key in that
    dictionary, so we move up to self.__class__.__dict__.
    That's where 'a' is! It's value is '1', so we get

    self.__dict__['a'] = 1*5
    ^
    self.__class__.__dict__['a']

    We finish the evaluation, and assign 5 to self.__dict__['a'], creating a new
    instance variable.
    The class variable 'a' is unchanged. If you call c.moda() again later then,
    that time, Python's lookup
    would find 'a' in self.__dict__, and the expression self.a *= 5 would be
    equivalent to

    self.__dict__['a'] = 5*5
    ^
    self.__dict__['a']

    So, the thing is, yes the behaviour is expected, if you know what behaviour
    to expect ...


    Okay, then. Hopefully that was helpful.
    Sean
     
    Sean Ross, Aug 10, 2003
    #4
  5. Ed Young

    Ed Young Guest

    Thank you all for the kind and detailed explanations.
    I now have a thorough understanding of the mechanism
    behind attribute lookup.
     
    Ed Young, Aug 10, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Kamoski
    Replies:
    3
    Views:
    15,852
    Jay B. Harlow [MVP - Outlook]
    Aug 9, 2003
  2. Phil Endecott

    Obj* ptr = new Obj(X)

    Phil Endecott, Jun 3, 2005, in forum: C++
    Replies:
    5
    Views:
    487
    Mark P
    Jun 3, 2005
  3. Shalabh Chaturvedi
    Replies:
    2
    Views:
    461
    Mike C. Fletcher
    Feb 20, 2004
  4. Derek Fountain

    When is a __dict__ not a __dict__?

    Derek Fountain, Apr 21, 2004, in forum: Python
    Replies:
    1
    Views:
    350
    John Roth
    Apr 21, 2004
  5. Steven D'Aprano
    Replies:
    8
    Views:
    354
    Christos Georgiou
    Feb 16, 2006
Loading...

Share This Page