__reduce__(1) vs __reduce__(2)

Discussion in 'Python' started by Kirill Simonov, Apr 18, 2006.

  1. Could someone explain why __reduce__(2) works for files while
    __reduce__(1) doesn't?
    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "/usr/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
    TypeError: can't pickle file objects(<function __newobj__ at 0xb7e6317c>, (<type 'file'>,), None, None, None)

    What is a correct procedure of getting state and restoring Python objects?
     
    Kirill Simonov, Apr 18, 2006
    #1
    1. Advertisements

  2. I think it is a bug. Both should raise an error.
    __reduce__ and __reduce_ex__ are part of the pickle protocol.
    Files are not meant to be pickable, since they are already
    persistent. With protocol 0 and 1, an error is raised when
    you try to pickle a builtin object that doesn't have a custom
    __reduce__() method. However, protocol 2 changed that, and
    now almost all objects can be pickled, but not necessarily
    unpickled to their original state. Files are one such example:
    you can pickle them, but the unpickled object won't be useful;
    you will get uninitialised file object. Basically, the result
    is equivalent to the following:
    unpickled = file.__new__(file)

    [snipped]
    You should never call __reduce__() with the protocol argument.
    It works only for objects that don't override the default
    __reduce__() method, and is possible only because
    object.__reduce_ex__ and object.__reduce__ represent the same
    underlying function. For example, sets override this method,
    and don't expect the protocol argument:
    Traceback (most recent call last):
    ...
    TypeError: __reduce__() takes no arguments (1 given)

    The correct way is to try to call obj.__reduce_ex__(protocol)
    first, and fall back to obj.__reduce__() if that method is not
    available. Example:

    if hasattr(obj, '__reduce_ex__'):
    state_tuple = obj.__reduce_ex__(2)
    else:
    state_tuple = obj.__reduce__()

    For more details check the pickle protocol documentation:
    http://docs.python.org/lib/pickle-protocol.html

    as well as Extensions to the pickle protocol PEP:
    http://www.python.org/dev/peps/pep-0307/

    Ziga
     
    Ziga Seilnacht, Apr 20, 2006
    #2
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.