dict subclass and pickle bug (?)

Discussion in 'Python' started by James Stroud, Dec 5, 2008.

  1. James Stroud

    James Stroud Guest

    Hello All,

    I subclassed dict and overrode __setitem__. When instances are
    unpickled, the __setstate__ is not called before the keys are assigned
    via __setitem__ in the unpickling protocol.

    I googled a bit and found that this a bug filed in 2003:

    http://bugs.python.org/issue826897

    It is still "open" with "normal" priority.

    Am I missing something? Is there a workaround for this bug that makes
    fixing it pointless or has it just fallen through the cracks for the
    last 5 years?

    Here is an example:

    class DictPlus(dict):
    def __init__(self, *args, **kwargs):
    self.extra_thing = ExtraThingClass()
    dict.__init__(self, *args, **kwargs)
    def __setitem__(self, k, v):
    do_something_with(self.extra_thing, k, v)
    dict.__setitem__(self, k, v)

    Upon unpickling, the error would be:

    AttributeError: 'DictPlus' object has no attribute 'extra_thing'


    I'm still using python 2.5.1.

    James

    --
    James Stroud
    UCLA-DOE Institute for Genomics and Proteomics
    Box 951570
    Los Angeles, CA 90095

    http://www.jamesstroud.com
    James Stroud, Dec 5, 2008
    #1
    1. Advertising

  2. James Stroud

    James Stroud Guest

    James Stroud wrote:
    > Hello All,
    >
    > I subclassed dict and overrode __setitem__. When instances are
    > unpickled, the __setstate__ is not called before the keys are assigned
    > via __setitem__ in the unpickling protocol.
    >
    > I googled a bit and found that this a bug filed in 2003:
    >
    > http://bugs.python.org/issue826897
    >
    > It is still "open" with "normal" priority.


    Here is the ugly "fix" I'm basically going to have to live with, it seems:

    class DictPlus(dict):
    def __init__(self, *args, **kwargs):
    self.extra_thing = ExtraThingClass()
    dict.__init__(self, *args, **kwargs)
    def __setitem__(self, k, v):
    try:
    do_something_with(self.extra_thing, k, v)
    except AttributeError:
    self.extra_thing = ExtraThingClass()
    do_something_with(self.extra_thing, k, v)
    dict.__setitem__(self, k, v)
    def __setstate__(self, adict):
    pass


    This violates this:

    Beautiful is better than ugly.


    I can't imagine this bug has survived but I also can't imagine any
    better way to do this without specifying a different protocol, which
    would probably break other pickling I'm doing. I don't feel like finding
    out right now.

    Maybe repeal pep 307 ;o)


    James


    --
    James Stroud
    UCLA-DOE Institute for Genomics and Proteomics
    Box 951570
    Los Angeles, CA 90095

    http://www.jamesstroud.com
    James Stroud, Dec 5, 2008
    #2
    1. Advertising

  3. James Stroud

    Terry Reedy Guest

    James Stroud wrote:
    > James Stroud wrote:
    >> Hello All,
    >>
    >> I subclassed dict and overrode __setitem__. When instances are
    >> unpickled, the __setstate__ is not called before the keys are assigned
    >> via __setitem__ in the unpickling protocol.
    >>
    >> I googled a bit and found that this a bug filed in 2003:


    It is an 'issue' reporting a possibly unexpected side-effect of protocol
    working as designed and documented. Possibly a design flaw, but not a
    bug in the narrow sense (in spite of the url of the issue tracker).

    >> http://bugs.python.org/issue826897
    >>
    >> It is still "open" with "normal" priority.

    >
    > Here is the ugly "fix" I'm basically going to have to live with, it seems:
    >
    > class DictPlus(dict):
    > def __init__(self, *args, **kwargs):
    > self.extra_thing = ExtraThingClass()
    > dict.__init__(self, *args, **kwargs)
    > def __setitem__(self, k, v):
    > try:
    > do_something_with(self.extra_thing, k, v)
    > except AttributeError:
    > self.extra_thing = ExtraThingClass()
    > do_something_with(self.extra_thing, k, v)
    > dict.__setitem__(self, k, v)
    > def __setstate__(self, adict):
    > pass


    I took the liberty of adding this to the issue.

    > I can't imagine this bug has survived


    because there is no bug to fix. I have suggesting closing.

    Terry Jan Reedy
    Terry Reedy, Dec 5, 2008
    #3
  4. James Stroud

    James Stroud Guest

    Terry Reedy wrote:
    > because there is no bug to fix. I have suggesting closing.


    May I suggest to add something to this effect within the issue itself so
    others won't spend time trying to figure out why the "bug" is still
    open? If this is a more general feature of issues, then perhaps it would
    be helpful to provide a footnote at the bottom of all issue pages via
    the page template that explains why they are not bugs and suggests a
    general course of action for the programmer.

    James
    James Stroud, Dec 5, 2008
    #4
  5. James Stroud

    James Stroud Guest

    James Stroud wrote:
    > Terry Reedy wrote:
    >> because there is no bug to fix. I have suggesting closing.

    >
    > May I suggest to add something to this effect within the issue itself so
    > others won't spend time trying to figure out why the "bug" is still
    > open?


    Sorry, you did that.

    James
    James Stroud, Dec 5, 2008
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jimmy Retzlaff
    Replies:
    0
    Views:
    378
    Jimmy Retzlaff
    Oct 16, 2003
  2. Christos TZOTZIOY Georgiou

    How to pickle a subclass of tuple?

    Christos TZOTZIOY Georgiou, Jun 3, 2004, in forum: Python
    Replies:
    4
    Views:
    364
    Christos TZOTZIOY Georgiou
    Jun 6, 2004
  3. a pickle's pickle

    , Aug 2, 2005, in forum: Python
    Replies:
    4
    Views:
    383
  4. Michele Simionato
    Replies:
    2
    Views:
    1,872
    Michele Simionato
    May 23, 2008
  5. S.Volkov
    Replies:
    2
    Views:
    213
    S.Volkov
    Mar 12, 2006
Loading...

Share This Page