Pickling an extension type subclasses problems

  • Thread starter Gaetan de Menten
  • Start date
G

Gaetan de Menten

Hi all,

I am trying to write an (optional!) C extension for SQLAlchemy, and I
am struggling to make an extension type picklable *and* using the same
format as the pure Python version (so that computers with the C
extension can unpickle pickles from computers without it and
vice-versa). Also the extension type (MyBase in the example code
below) only serves as a base class for the type that get pickled
(MyPythonType). MyPythonType also has further subclasses and those
should be picklable too.

My setup is as follow:

On the Python side:
================

try:
from cextension import MyBase
except ImportError:
class MyBase(object):
__slots__ = ('a', 'b', 'c')
def __init__(self, a, b, c):
self.a, self.b, self.c = a, b, c

def __getstate__(self):
...

def __setstate__(self, state):
...
...


class MyPythonType(MyBase):
...

On the .c side:
===========
I implemented a reduce method, which returns

Py_BuildValue("(O(s)N)", Py_TYPE(self), "__state__", state);

and in my BaseRowProxy_init method, I check the number of arguments,
whether the "__state__" marker is present and unpack the state if it
is. This makes the type nicely pickleable, but this is obviously not
the same "pickle format" as the Python version.

What would you recommend in this kind of situation? I'm a bit tired of
trial and error... My last hope is to define __reduce__ on the Python
side too, and make it use a module-level "reconstructor" function as
follow:

def mypythontype_reconstructor(cls, state):
obj = object.__new__(cls, state):
....
return obj

Will this work? Any other idea?

Thanks in advance,
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,756
Messages
2,569,531
Members
45,004
Latest member
KetoBoostGummies

Latest Threads

Top