Re: Is there anything that pickle + copy_reg cannot serialize?

Discussion in 'Python' started by Jean-Paul Calderone, Dec 8, 2005.

  1. On Fri, 09 Dec 2005 02:17:10 +0800, Maurice LING <> wrote:
    >
    >> Since copy_reg lets you specify arbitrary code to serialize arbitrary
    >> objects, you shouldn't run into any single object that you cannot
    >> serialize to a pickle.

    >
    > [snip - example of pickling code objects]
    >
    >
    >I cannot understand 2 things, which I seek assistance for:
    >1. Is code object the only thing can cannot be pickled (less facing
    >recursion limits)?


    No. There are lots of objects that cannot be pickled by default. Any
    extension type which does not explicitly support it cannot be pickled.
    Generators cannot be pickled. Method descriptors can't be pickled. Et
    cetera.

    >2. In the above example, how copy_reg works with pickle?


    Any time pickle thinks it has found something it cannot pickle, it asks
    the copy_reg module for some help. The above example basically teaches
    the copy_reg module how to give the pickle module the help it needs for
    code objects.

    Jean-Paul
     
    Jean-Paul Calderone, Dec 8, 2005
    #1
    1. Advertising

  2. Jean-Paul Calderone

    Maurice LING Guest


    >>> Since copy_reg lets you specify arbitrary code to serialize arbitrary
    >>> objects, you shouldn't run into any single object that you cannot
    >>> serialize to a pickle.

    >>
    >>
    >> [snip - example of pickling code objects]
    >>
    >>
    >> I cannot understand 2 things, which I seek assistance for:
    >> 1. Is code object the only thing can cannot be pickled (less facing
    >> recursion limits)?

    >
    >
    > No. There are lots of objects that cannot be pickled by default. Any
    > extension type which does not explicitly support it cannot be pickled.
    > Generators cannot be pickled. Method descriptors can't be pickled. Et
    > cetera.


    Thank Jean-Paul.

    Sorry for not specifying clearly enough. Given that copy_reg lets you
    specify arbitrary code to serialize arbitrary objects, of which some are
    taken care of by pickle, in the set of possible Python types,

    >>> import types
    >>> dir(types)

    ['BooleanType', 'BufferType', 'BuiltinFunctionType',
    'BuiltinMethodType', 'ClassType', 'CodeType', 'ComplexType',
    'DictProxyType', 'DictType', 'DictionaryType', 'EllipsisType',
    'FileType', 'FloatType', 'FrameType', 'FunctionType', 'GeneratorType',
    'InstanceType', 'IntType', 'LambdaType', 'ListType', 'LongType',
    'MethodType', 'ModuleType', 'NoneType', 'NotImplementedType',
    'ObjectType', 'SliceType', 'StringType', 'StringTypes', 'TracebackType',
    'TupleType', 'TypeType', 'UnboundMethodType', 'UnicodeType',
    'XRangeType', '__builtins__', '__doc__', '__file__', '__name__']

    What types cannot be serialized by pickle (besides CodeType) and
    requires handling using copy_reg?

    Thanks
    Maurice


    >
    >> 2. In the above example, how copy_reg works with pickle?

    >
    >
    > Any time pickle thinks it has found something it cannot pickle, it asks
    > the copy_reg module for some help. The above example basically teaches
    > the copy_reg module how to give the pickle module the help it needs for
    > code objects.
    >
    > Jean-Paul
     
    Maurice LING, Dec 9, 2005
    #2
    1. Advertising

  3. Maurice LING wrote:

    > Sorry for not specifying clearly enough. Given that copy_reg lets you
    > specify arbitrary code to serialize arbitrary objects, of which some are
    > taken care of by pickle, in the set of possible Python types,


    the types module contains a selection of type objects; the set of possible
    types that you may have in a Python program is unbounded.

    > What types cannot be serialized by pickle (besides CodeType) and
    > requires handling using copy_reg?


    import types, pickle, sys

    for t in dir(types):
    try:
    if t == "BufferType":
    o = buffer("hello")
    elif t == "EllipsisType":
    o = Ellipsis
    elif t == "FileType":
    o = file("hello.txt", "w")
    elif t == "NoneType":
    o = None
    elif t == "SliceType":
    o = slice(0,0,0)
    elif t == "XRangeType":
    o = xrange(0,0,1)
    else:
    o = getattr(types, t)()
    except:
    print t, "FAILED:", sys.exc_value
    else:
    try:
    s = pickle.dumps(o)
    except:
    print t, "FAILED:", str(sys.exc_value).upper()
    else:
    print t, "OK"

    should give you an idea (feel free to add more object factories)

    </F>
     
    Fredrik Lundh, Dec 9, 2005
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gordz
    Replies:
    3
    Views:
    3,960
    John Oakes
    Jun 7, 2004
  2. a pickle's pickle

    , Aug 2, 2005, in forum: Python
    Replies:
    4
    Views:
    394
  3. Maurice LING
    Replies:
    0
    Views:
    283
    Maurice LING
    Dec 8, 2005
  4. Jean-Paul Calderone
    Replies:
    1
    Views:
    370
    Maurice LING
    Dec 8, 2005
  5. Michele Simionato
    Replies:
    2
    Views:
    1,914
    Michele Simionato
    May 23, 2008
Loading...

Share This Page