Re: __dict__ attribute for built-in types

Discussion in 'Python' started by candide, Oct 27, 2011.

  1. candide

    candide Guest

    Le 27/10/2011 13:03, Duncan Booth a écrit :

    >
    >> -- where the official documentation refers to this point ?
    >>

    > See http://docs.python.org/reference/datamodel.html for the docs about
    > __slots__
    >
    > There is also the API documentation which describes at a low level how
    > to control whether or not instances have a dict:
    > http://docs.python.org/c-api/typeobj.html#tp_dictoffset
    >
    > I'm not sure though where you find a higher level statement of which
    > builtin types have a __dict__.
    >



    OK, thanks for the information abouts the slots. Nevertheless, this
    cannot answer completely my question. Some builtin types like string,
    lists, integer, float, dictionaries, etc have the property that
    instances of those types don't provide a __dict__ attribute. I can't
    imagine the documentation lets pass silently this point.

    But beside this, how to recognise classes whose object doesn't have a
    __dict__ attribute ?
     
    candide, Oct 27, 2011
    #1
    1. Advertising

  2. On Thu, 27 Oct 2011 16:01:25 +0200, candide wrote:

    > OK, thanks for the information abouts the slots. Nevertheless, this
    > cannot answer completely my question. Some builtin types like string,
    > lists, integer, float, dictionaries, etc have the property that
    > instances of those types don't provide a __dict__ attribute. I can't
    > imagine the documentation lets pass silently this point.


    What, you think it goes against the laws of physics that nobody thought
    to mention it in the docs? <wink>

    > But beside this, how to recognise classes whose object doesn't have a
    > __dict__ attribute ?


    The same way as you would test for any other attribute.

    >>> hasattr(42, '__dict__')

    False


    --
    Steven
     
    Steven D'Aprano, Oct 27, 2011
    #2
    1. Advertising

  3. candide

    candide Guest

    Le 28/10/2011 00:19, Steven D'Aprano a écrit :
    >
    > What, you think it goes against the laws of physics that nobody thought
    > to mention it in the docs?<wink>



    No but I'm expecting from Python documentation to mention the laws of
    Python ...


    >
    >> But beside this, how to recognise classes whose object doesn't have a
    >> __dict__ attribute ?

    >
    > The same way as you would test for any other attribute.
    >
    >>>> hasattr(42, '__dict__')

    > False
    >
    >



    OK but I'm talking about classes, not instances : 42 has no __dict__
    attribute but, may be, 43 _has_ such attribute, who knows in advance ? ;)

    Let'have a try :

    >>> hasattr(43, '__dict__')

    False
    >>>



    so we have proved by induction that no integer instance has a
    dictionnary attribute ;)
     
    candide, Oct 27, 2011
    #3
  4. candide <> writes:

    > But beside this, how to recognise classes whose object doesn't have a
    > __dict__ attribute ?


    str, list and others aren't classes, they are types. While all
    (new-style) classes are types, not all types are classes. It's
    instances of classes (types created by executing the "class" statement
    or its equivalent) that automatically get a __dict__, unless __slots__
    was used at class definition time to suppress it. Built-in and
    extension types can choose whether to implement __dict__.

    (Mechanics of defining built-in and extension types are of course
    implementation-specific. CPython allows adding __dict__ to any
    extension type by setting the tp_dictoffset member of the type
    definition struct to the appropriate offset into the instance struct.)
     
    Hrvoje Niksic, Oct 27, 2011
    #4
  5. candide

    candide Guest

    Le 28/10/2011 00:57, Hrvoje Niksic a écrit :

    > was used at class definition time to suppress it. Built-in and
    > extension types can choose whether to implement __dict__.
    >


    Is it possible in the CPython implementation to write something like this :

    "foo".bar = 42

    without raising an attribute error ?
     
    candide, Oct 28, 2011
    #5
  6. candide

    MRAB Guest

    On 28/10/2011 00:36, candide wrote:
    > Le 28/10/2011 00:57, Hrvoje Niksic a écrit :
    >
    >> was used at class definition time to suppress it. Built-in and
    >> extension types can choose whether to implement __dict__.
    >>

    >
    > Is it possible in the CPython implementation to write something like this :
    >
    > "foo".bar = 42
    >
    > without raising an attribute error ?


    No, built-in classes written in C have certain limitations, but why
    would you want to do that anyway?
     
    MRAB, Oct 28, 2011
    #6
  7. candide

    Terry Reedy Guest

    On 10/27/2011 6:52 PM, candide wrote:

    > No but I'm expecting from Python documentation to mention the laws of
    > Python ...


    The details of CPython builtin classes are not laws of Python. It *is* a
    'law of Python' that classes can use 'slots = ' to restrict the
    attributes of instances. By implication, builtin classes in any
    implementation do not have to allow attribute assignment. I do not
    believe it would be a violation if some implementation did so.

    None of this is to say that we could not say something on the subject at
    the beginning of the 'built-in types' chapter of the lib manual.

    > OK but I'm talking about classes, not instances :


    Yes you are. The class determines whether its instances have assignable
    new attributes.

    > 42 has no __dict__ > attribute but,
    > may be, 43 _has_ such attribute, who knows in advance ? ;)


    True, in a sense, but if the class allowed a user to execute
    "42.__dict__ = {}" then you could safely assume that "43.xxx = z" should
    work also.

    --
    Terry Jan Reedy
     
    Terry Reedy, Oct 28, 2011
    #7
  8. candide

    candide Guest

    Le 28/10/2011 02:02, MRAB a écrit :

    >
    > No, built-in classes written in C have certain limitations, but why
    > would you want to do that anyway?





    Mainly for learning purpose and Python better understanding.

    Actually, I have a class of mine for drawing graphs with the Graphviz
    software. The nodes of the graph to be represented was supposed to have
    2 attributes, say title and shortName. Now, I want to plot a graph whose
    nodes are pure string. So to fit the class interface, I was trying to
    add title and shortName attribute to every string node.
     
    candide, Oct 28, 2011
    #8
  9. candide

    alex23 Guest

    On Oct 28, 8:52 am, candide <> wrote:
    > No but I'm expecting from Python documentation to mention the laws of
    > Python ...


    It's not a "law", it's an _implementation detail_. The docs don't tend
    to mention every such decision made because that's what the source is
    for.

    > But beside this, how to recognise classes whose object doesn't have a
    > __dict__ attribute ?


    The better question is: why do you need to be able to?

    > Is it possible in the CPython implementation to write something like this:
    > "foo".bar = 42
    > without raising an attribute error ?


    Why are you trying to modify an immutible object?

    If you really want to assign attributes to string objects, subclass
    str.
     
    alex23, Oct 28, 2011
    #9
  10. On Oct 27, 9:46 pm, candide <> wrote:
    > Le 28/10/2011 02:02, MRAB a crit :
    >
    >
    >
    > > No, built-in classes written in C have certain limitations, but why
    > > would you want to do that anyway?

    >
    > Mainly for learning purpose and Python better understanding.
    >
    > Actually, I have a class of mine for drawing graphs with the Graphviz
    > software. The nodes of the graph to be represented was supposed to have
    > 2 attributes, say title and shortName. Now, I want to plot a graph whose
    > nodes are pure string. So to fit the class interface, I was trying to
    > add title and shortName attribute to every string node.


    You can easily do that by subclassing a string:

    class AnnotatedStr(str):
    pass

    x = AnnotatedStr('Node1')
    x.title = 'Title for node 1'

    etc.

    The fact that you subclass it (unless your subclass uses __slots__)
    will give it a dict.
     
    Patrick Maupin, Oct 28, 2011
    #10
  11. On Fri, 28 Oct 2011 01:36:41 +0200, candide wrote:

    > Le 28/10/2011 00:57, Hrvoje Niksic a écrit :
    >
    >> was used at class definition time to suppress it. Built-in and
    >> extension types can choose whether to implement __dict__.
    >>
    >>

    > Is it possible in the CPython implementation to write something like
    > this :
    >
    > "foo".bar = 42
    >
    > without raising an attribute error ?


    No, because built-in strings don't have a __dict__ and so you cannot add
    new attributes that didn't already exist.

    But you can subclass str and do whatever you like.

    class Str(str):
    pass

    Str("foo").bar = 42



    --
    Steven
     
    Steven D'Aprano, Oct 28, 2011
    #11
  12. On Fri, 28 Oct 2011 00:52:40 +0200, candide wrote:

    > Le 28/10/2011 00:19, Steven D'Aprano a écrit :
    >>
    >> What, you think it goes against the laws of physics that nobody thought
    >> to mention it in the docs?<wink>

    >
    >
    > No but I'm expecting from Python documentation to mention the laws of
    > Python ...


    You seem to have missed my point. You said "I can't imagine" that the
    Python docs fail to mention that built-ins don't allow the addition of
    new attributes. I can, easily. The people writing the documentation are
    only human, and if they failed to mention it, oh well, perhaps they
    didn't think of it. This is hardly a surprise. Wanting to add arbitrary
    attributes to built-ins is not exactly an everyday occurrence.


    >>> But beside this, how to recognise classes whose object doesn't have a
    >>> __dict__ attribute ?

    >>
    >> The same way as you would test for any other attribute.
    >>
    >>>>> hasattr(42, '__dict__')

    >> False

    >
    > OK but I'm talking about classes, not instances : 42 has no __dict__
    > attribute but, may be, 43 _has_ such attribute, who knows in advance ?
    > ;)


    True, it is theoretically possible that (say) only odd numbers get a
    __dict__, or primes, or the smallest multiple of seventeen larger than
    the natural logarithm of a googol (10**100). But it's a safe bet that
    nothing so arbitrary will happen.

    Dunder attributes ("Double UNDERscore") like __dict__ are reserved for
    use by Python, and __dict__ has known semantics. You can safely assume
    that either *all* instances of a type will have a __dict__, or *no*
    instances will have one. If some class violates that, oh well, your code
    can't be expected to support every badly-designed stupid class in the
    world.

    Also, keep in mind the difference between a *class* __dict__ and an
    *instance* __dict__.

    >>> hasattr(int, '__dict__') # Does the int class/type have a __dict__?

    True
    >>> hasattr(42, '__dict__') # Does the int instance have a __dict__?

    False



    --
    Steven
     
    Steven D'Aprano, Oct 28, 2011
    #12
  13. candide <> writes:

    > Le 28/10/2011 00:57, Hrvoje Niksic a écrit :
    >
    >> was used at class definition time to suppress it. Built-in and
    >> extension types can choose whether to implement __dict__.
    >>

    >
    > Is it possible in the CPython implementation to write something like this :
    >
    > "foo".bar = 42
    >
    > without raising an attribute error ?


    No, and for good reason. Strings are immutable, so that you needn't
    care which particular instance of "foo" you're looking at, they're all
    equivalent. The interpreter uses that fact to cache instances of short
    strings such as Python identifiers, so that most places that look at a
    string like "foo" are in fact dealing with the same instance. If one
    could change an attribute of a particular instance of "foo", it would no
    longer be allowed for the interpreter to transparently cache them. The
    same goes for integers and other immutable built-in objects.

    If you really need to attach state to strings, subclass them as Steven
    explained. All code that accepts strings (including all built-ins) will
    work just fine, transparent caching will not happen, and attributes are
    writable.
     
    Hrvoje Niksic, Oct 28, 2011
    #13
  14. candide

    candide Guest

    Le 28/10/2011 10:01, Steven D'Aprano a écrit :

    > didn't think of it. This is hardly a surprise. Wanting to add arbitrary
    > attributes to built-ins is not exactly an everyday occurrence.
    >




    Depends. Experimented programmers don't even think of it. But less
    advanced programmers can consider of it. It's is not uncommun to use a
    Python class like a C structure, for instance :

    class C:pass

    C.member1=foo
    C.member2=bar


    Why not with a built-in type instead of a custom class?




    > the natural logarithm of a googol (10**100). But it's a safe bet that
    > nothing so arbitrary will happen.


    betting when programming ? How curious! ;)


    > Also, keep in mind the difference between a *class* __dict__ and an
    > *instance* __dict__.
    >


    You mean this distinction

    >>> hasattr('', '__dict__')

    False
    >>> hasattr(''.__class__, '__dict__')

    True
    >>>



    ?
     
    candide, Oct 28, 2011
    #14
  15. candide

    candide Guest

    Le 28/10/2011 05:02, Patrick Maupin a écrit :

    > You can easily do that by subclassing a string:
    >
    > class AnnotatedStr(str):
    > pass
    >
    > x = AnnotatedStr('Node1')
    > x.title = 'Title for node 1'
    >



    Less or more what I did. But requires to transport the string graph
    structure to the AnnotatedStr one.
     
    candide, Oct 28, 2011
    #15
  16. candide

    candide Guest

    Le 28/10/2011 11:08, Hrvoje Niksic a écrit :

    > longer be allowed for the interpreter to transparently cache them. The
    > same goes for integers and other immutable built-in objects.



    On the other hand, immutability and optimization don't explain the whole
    thing because you can't do something like [].bar = 42.



    >
    > If you really need to attach state to strings, subclass them as Steven
    > explained. All code that accepts strings (including all built-ins) will
    > work just fine, transparent caching will not happen, and attributes are
    > writable.


    OK, thanks, I'll consider it seriously. Actually I have a directed graph
    whose nodes are given as string but it's oversimplistic to identify node
    and string (for nodes requiring specific methods).
     
    candide, Oct 28, 2011
    #16
  17. On Fri, 28 Oct 2011 00:52:40 +0200, candide <> wrote:
    [snip]
    >>>>> hasattr(42, '__dict__')

    >> False

    [snip]
    >
    > Let'have a try :
    >
    > >>> hasattr(43, '__dict__')

    > False
    > >>>

    >
    > so we have proved by induction that no integer instance has a
    > dictionnary attribute ;)


    You left out an important step in this proof by induction. Observe:

    >>> n = 0
    >>> hasattr(n, "__dict__")

    False
    >>> if hasattr(n, "__dict__") is False:

    .... hasattr(n+1, "__dict__") is False
    ....
    True

    There, now it's proven by induction.

    --
    To email me, substitute nowhere->spamcop, invalid->net.
     
    Peter Pearson, Oct 28, 2011
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ed Young
    Replies:
    4
    Views:
    340
    Ed Young
    Aug 10, 2003
  2. Replies:
    1
    Views:
    359
    Alex Martelli
    Nov 6, 2003
  3. Derek Fountain

    When is a __dict__ not a __dict__?

    Derek Fountain, Apr 21, 2004, in forum: Python
    Replies:
    1
    Views:
    350
    John Roth
    Apr 21, 2004
  4. Jean Brouwers

    __slots__ vs __dict__

    Jean Brouwers, May 12, 2004, in forum: Python
    Replies:
    5
    Views:
    527
    Marcus von Appen
    May 13, 2004
  5. Alona
    Replies:
    2
    Views:
    669
Loading...

Share This Page