Pickling an extension type subclasses problems

Discussion in 'Python' started by Gaetan de Menten, Feb 5, 2010.

  1. Hi all,

    I am trying to write an (optional!) C extension for SQLAlchemy, and I
    am struggling to make an extension type picklable *and* using the same
    format as the pure Python version (so that computers with the C
    extension can unpickle pickles from computers without it and
    vice-versa). Also the extension type (MyBase in the example code
    below) only serves as a base class for the type that get pickled
    (MyPythonType). MyPythonType also has further subclasses and those
    should be picklable too.

    My setup is as follow:

    On the Python side:
    ================

    try:
    from cextension import MyBase
    except ImportError:
    class MyBase(object):
    __slots__ = ('a', 'b', 'c')
    def __init__(self, a, b, c):
    self.a, self.b, self.c = a, b, c

    def __getstate__(self):
    ...

    def __setstate__(self, state):
    ...
    ...


    class MyPythonType(MyBase):
    ...

    On the .c side:
    ===========
    I implemented a reduce method, which returns

    Py_BuildValue("(O(s)N)", Py_TYPE(self), "__state__", state);

    and in my BaseRowProxy_init method, I check the number of arguments,
    whether the "__state__" marker is present and unpack the state if it
    is. This makes the type nicely pickleable, but this is obviously not
    the same "pickle format" as the Python version.

    What would you recommend in this kind of situation? I'm a bit tired of
    trial and error... My last hope is to define __reduce__ on the Python
    side too, and make it use a module-level "reconstructor" function as
    follow:

    def mypythontype_reconstructor(cls, state):
    obj = object.__new__(cls, state):
    ....
    return obj

    Will this work? Any other idea?

    Thanks in advance,
    --
    Gaƫtan de Menten
     
    Gaetan de Menten, Feb 5, 2010
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. danny
    Replies:
    2
    Views:
    293
    Greg Chapman
    Apr 23, 2004
  2. Edward Loper

    pickling subclasses of dict/list

    Edward Loper, Jul 3, 2004, in forum: Python
    Replies:
    1
    Views:
    406
    Greg Chapman
    Jul 4, 2004
  3. Replies:
    0
    Views:
    341
  4. harold fellermann

    pickling extension class

    harold fellermann, Jan 18, 2005, in forum: Python
    Replies:
    4
    Views:
    429
    David M. Cooke
    Jan 18, 2005
  5. Rolando Abarca

    c extension, classes and subclasses

    Rolando Abarca, Jul 9, 2009, in forum: Ruby
    Replies:
    2
    Views:
    97
    KUBO Takehiro
    Jul 12, 2009
Loading...

Share This Page