Python3, __slots__ and serialization

Discussion in 'Python' started by Eric Jacoboni, Feb 8, 2014.

  1. Hi,

    Say i want create a class with a __slots__ tuple in order to prevent
    creation of new attributes from outside the class.

    Say i want to serialize instances of this class... With pickle, all is
    ok : i can dump an object to a file, then reload it.

    With PyYAML, i can dump an object to a file, but trying to reload it fails.

    If i comment out the __slots__ tuple, all is fine but, then, the class
    is "open"... Is there some magic to have both a __slots__ tuple and a
    working yaml.load() ?
     
    Eric Jacoboni, Feb 8, 2014
    #1
    1. Advertising

  2. On 2/8/14 1:06 PM, Eric Jacoboni wrote:
    > Hi,
    >
    > Say i want create a class with a __slots__ tuple in order to prevent
    > creation of new attributes from outside the class.
    >
    > Say i want to serialize instances of this class... With pickle, all is
    > ok : i can dump an object to a file, then reload it.
    >
    > With PyYAML, i can dump an object to a file, but trying to reload it fails.
    >
    > If i comment out the __slots__ tuple, all is fine but, then, the class
    > is "open"... Is there some magic to have both a __slots__ tuple and a
    > working yaml.load() ?
    >


    Preventing attribute creation runs counter to Python's philosophy, so it
    doesn't surprise me that it causes problems. There might be a way to
    make it all work together, but I don't know.

    __slots__ wasn't meant to be a way to prevent attribute creation, it was
    meant as a way to save space by avoiding the need for an attributes
    dictionary.

    Why do you need to prevent attribute creation? If someone uses your
    class and wants to use it in ways you didn't anticipate, why should you
    try to stop them?

    --
    Ned Batchelder, http://nedbatchelder.com
     
    Ned Batchelder, Feb 8, 2014
    #2
    1. Advertising

  3. On 2/8/14 1:29 PM, Ned Batchelder wrote:
    > On 2/8/14 1:06 PM, Eric Jacoboni wrote:
    >> Hi,
    >>
    >> Say i want create a class with a __slots__ tuple in order to prevent
    >> creation of new attributes from outside the class.
    >>
    >> Say i want to serialize instances of this class... With pickle, all is
    >> ok : i can dump an object to a file, then reload it.
    >>
    >> With PyYAML, i can dump an object to a file, but trying to reload it
    >> fails.
    >>
    >> If i comment out the __slots__ tuple, all is fine but, then, the class
    >> is "open"... Is there some magic to have both a __slots__ tuple and a
    >> working yaml.load() ?
    >>

    >
    > Preventing attribute creation runs counter to Python's philosophy, so it
    > doesn't surprise me that it causes problems. There might be a way to
    > make it all work together, but I don't know.
    >
    > __slots__ wasn't meant to be a way to prevent attribute creation, it was
    > meant as a way to save space by avoiding the need for an attributes
    > dictionary.
    >
    > Why do you need to prevent attribute creation? If someone uses your
    > class and wants to use it in ways you didn't anticipate, why should you
    > try to stop them?
    >


    I also looked at why it fails. I get this error:


    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File
    "/usr/local/virtualenvs/lab/lib/python2.7/site-packages/yaml/__init__.py",
    line 71, in load
    return loader.get_single_data()
    File
    "/usr/local/virtualenvs/lab/lib/python2.7/site-packages/yaml/constructor.py",
    line 39, in get_single_data
    return self.construct_document(node)
    File
    "/usr/local/virtualenvs/lab/lib/python2.7/site-packages/yaml/constructor.py",
    line 43, in construct_document
    data = self.construct_object(node)
    File
    "/usr/local/virtualenvs/lab/lib/python2.7/site-packages/yaml/constructor.py",
    line 90, in construct_object
    data = constructor(self, tag_suffix, node)
    File
    "/usr/local/virtualenvs/lab/lib/python2.7/site-packages/yaml/constructor.py",
    line 610, in construct_python_object_new
    return self.construct_python_object_apply(suffix, node, newobj=True)
    File
    "/usr/local/virtualenvs/lab/lib/python2.7/site-packages/yaml/constructor.py",
    line 601, in construct_python_object_apply
    self.set_python_instance_state(instance, state)
    File
    "/usr/local/virtualenvs/lab/lib/python2.7/site-packages/yaml/constructor.py",
    line 563, in set_python_instance_state
    setattr(object, key, value)
    TypeError: can't set attributes of built-in/extension type 'object'

    This is set_python_instance_state, the failing code:

    def set_python_instance_state(self, instance, state):
    if hasattr(instance, '__setstate__'):
    instance.__setstate__(state)
    else:
    slotstate = {}
    if isinstance(state, tuple) and len(state) == 2:
    state, slotstate = state
    if hasattr(instance, '__dict__'):
    instance.__dict__.update(state)
    elif state:
    slotstate.update(state)
    for key, value in slotstate.items():
    setattr(object, key, value)

    Two things seem clear to me: 1) you can define __setstate__ to do what
    you need to do, and 2) this code is simply wrong. The last line should
    use "instance", not "object", and it might be that the last two lines
    should really be indented withing the elif.

    All that said, skip __slots__: it isn't helping you with any actual
    problem, and it's forcing you to jump through hoops, or run code that
    most Python programs don't run.

    --
    Ned Batchelder, http://nedbatchelder.com
     
    Ned Batchelder, Feb 8, 2014
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    8
    Views:
    2,291
    deadsea
    Jan 2, 2005
  2. Replies:
    3
    Views:
    1,068
  3. Dimitri Ognibene
    Replies:
    4
    Views:
    807
    Dimitri Ognibene
    Sep 2, 2006
  4. Ramunas Urbonas
    Replies:
    1
    Views:
    420
    Dino Chiesa [Microsoft]
    Jul 27, 2004
  5. Andrew Berg
    Replies:
    0
    Views:
    353
    Andrew Berg
    Jun 16, 2012
Loading...

Share This Page