[c-api]Transmutation of an extension object into a read-only bufferadding an integer in-place.

[Giacomo Alzetta]

I'm trying to implement a c-extension which defines a new class(ModPolynomial on the python side, ModPoly on the C-side).
At the moment I'm writing the in-place addition, but I get a *really* strange behaviour.

Here's the code for the in-place addition:

#define ModPoly_Check(v) (PyObject_TypeCheck(v, &ModPolyType))
[...]
static PyObject *
ModPoly_InPlaceAdd(PyObject *self, PyObject *other)
{

if (!ModPoly_Check(self)) {
// This should never occur for in-place addition, am I correct?
if (!ModPoly_Check(other)) {
PyErr_SetString(PyExc_TypeError, "Neither argument is a ModPolynomial.");
return NULL;
}
return ModPoly_InPlaceAdd(other, self);
} else {
if (!PyInt_Check(other) && !PyLong_Check(other)) {
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}
}

ModPoly *Tself = (ModPoly *)self;
PyObject *tmp, *tmp2;
tmp = PyNumber_Add(Tself->ob_item[0], other);
tmp2 = PyNumber_Remainder(tmp, Tself->n_modulus);

Py_DECREF(tmp);
tmp = Tself->ob_item[0];
Tself->ob_item[0] = tmp2;
Py_DECREF(tmp);

printf("%d\n", (int)ModPoly_Check(self));
return self;

}

And here's an example usage:

Now, how come my ModPolynomial suddenly becomes a read-only buffer, even though that last printf tells us that the object returned is of the correct type?
If I raise an exception instead of returning self, the ModPolynomial gets incremented correctly. If I use the Py_RETURN_NONE macro, the ModPolynomial is correctly replaced by None.

 

[Hans Mulder]

I'm trying to implement a c-extension which defines a new class(ModPolynomial on the python side, ModPoly on the C-side).
At the moment I'm writing the in-place addition, but I get a *really* strange behaviour.

Here's the code for the in-place addition:

#define ModPoly_Check(v) (PyObject_TypeCheck(v, &ModPolyType))
[...]
static PyObject *
ModPoly_InPlaceAdd(PyObject *self, PyObject *other)
{

if (!ModPoly_Check(self)) {
// This should never occur for in-place addition, am I correct?
if (!ModPoly_Check(other)) {
PyErr_SetString(PyExc_TypeError, "Neither argument is a ModPolynomial.");
return NULL;
}
return ModPoly_InPlaceAdd(other, self);
} else {
if (!PyInt_Check(other) && !PyLong_Check(other)) {
Py_INCREF(Py_NotImplemented);
return Py_NotImplemented;
}
}

ModPoly *Tself = (ModPoly *)self;
PyObject *tmp, *tmp2;
tmp = PyNumber_Add(Tself->ob_item[0], other);
tmp2 = PyNumber_Remainder(tmp, Tself->n_modulus);

Py_DECREF(tmp);
tmp = Tself->ob_item[0];
Tself->ob_item[0] = tmp2;
Py_DECREF(tmp);

printf("%d\n", (int)ModPoly_Check(self));
return self;

}

I have no experience writing extensions in C, but as I see it,
you're returning a new reference to self, so you'd need:

Py_INCREF(self);

If you don't, then a Py_DECREF inside the assignment operator
causes your polynomial to be garbage collected. Its heap slot
is later used for the unrelated buffer object you're seeing.

Hope this helps,

-- HansM

 

[Giacomo Alzetta]

Il giorno venerdì 10 agosto 2012 11:22:13 UTC+2, Hans Mulder ha scritto:

I'm trying to implement a c-extension which defines a new class(ModPolynomial on the python side, ModPoly on the C-side).

At the moment I'm writing the in-place addition, but I get a *really* strange behaviour.
Here's the code for the in-place addition:
#define ModPoly_Check(v) (PyObject_TypeCheck(v, &ModPolyType))

[...]

static PyObject *

ModPoly_InPlaceAdd(PyObject *self, PyObject *other)


if (!ModPoly_Check(self)) {

// This should never occur for in-place addition, am I correct?

if (!ModPoly_Check(other)) {

PyErr_SetString(PyExc_TypeError, "Neither argument is a ModPolynomial.");

return NULL;

return ModPoly_InPlaceAdd(other, self);

} else {

if (!PyInt_Check(other) && !PyLong_Check(other)) {

return Py_NotImplemented;



ModPoly *Tself = (ModPoly *)self;

PyObject *tmp, *tmp2;

tmp = PyNumber_Add(Tself->ob_item[0], other);

tmp2 = PyNumber_Remainder(tmp, Tself->n_modulus);

Py_DECREF(tmp);

tmp = Tself->ob_item[0];

Tself->ob_item[0] = tmp2;

Py_DECREF(tmp);

printf("%d\n", (int)ModPoly_Check(self));

return self;

}

I have no experience writing extensions in C, but as I see it,

you're returning a new reference to self, so you'd need:



Py_INCREF(self);



If you don't, then a Py_DECREF inside the assignment operator

causes your polynomial to be garbage collected. Its heap slot

is later used for the unrelated buffer object you're seeing.



Hope this helps,



-- HansM

Yes, you're right. I didn't thought the combined operator would do a Py_DECREF if the iadd operation was implemented, but it obviosuly makes sense.

 

[Hans Mulder]

Il giorno venerdì 10 agosto 2012 11:22:13 UTC+2, Hans Mulder ha scritto: [...]
Yes, you're right. I didn't thought the combined operator would do a Py_DECREF
if the iadd operation was implemented, but it obviosuly makes sense.

The += operator cannot know if the iadd returns self or a newly created
object. Mutable types usually do the former; non-mutable types must do
the latter.

Come to think of it: why are your polynomials mutable?

As a mathematician, I would think of polynomials as elements of
some kind of ring, and I'd expect them to be non-mutable.

-- HansM

 

[Giacomo Alzetta]

Il giorno venerdì 10 agosto 2012 14:21:50 UTC+2, Hans Mulder ha scritto:

Il giorno venerdì 10 agosto 2012 11:22:13 UTC+2, Hans Mulder ha scritto:
[...]

Yes, you're right. I didn't thought the combined operator would do a Py_DECREF

if the iadd operation was implemented, but it obviosuly makes sense.

The += operator cannot know if the iadd returns self or a newly created

object. Mutable types usually do the former; non-mutable types must do

the latter.



Come to think of it: why are your polynomials mutable?



As a mathematician, I would think of polynomials as elements of

some kind of ring, and I'd expect them to be non-mutable.



-- HansM

Usually non-mutable types simply do not implement the iadd operation, and the interpreter tries the simple add after(see intobject.c, longobject.c etc..).

I've decided to make my polynomials mutable for efficiency.
I want to implement the AKS primality tests, and if I want to use it with big numbers the number of coefficients of a polynomial can easily go up to 1k-10k-100k and using non-mutable polynomials would mean to allocate and free that much memory for almost every operation.
Like this I have to allocate/free less frequently.

[even though I must admit that this is a premature optimization :s, since I've not profile anything, but untill I do not implement them I wont be ableto see how much time I gain.]

 

[Stefan Behnel]

Giacomo Alzetta, 10.08.2012 10:20:

I'm trying to implement a c-extension which defines a new class(ModPolynomial on the python side, ModPoly on the C-side).
At the moment I'm writing the in-place addition, but I get a *really* strange behaviour.

You should take a look at Cython. It makes these things way easier and
safer than with manually written C code. It will save you a lot of code,
debugging and general hassle.

Stefan

 

[Giacomo Alzetta]

Il giorno venerdì 10 agosto 2012 20:50:08 UTC+2, Stefan Behnel ha scritto:

Giacomo Alzetta, 10.08.2012 10:20:





You should take a look at Cython. It makes these things way easier and

safer than with manually written C code. It will save you a lot of code,

debugging and general hassle.



Stefan

I already know Cython, but I hope to learn a bit how python works from the C-side writing this extension.

Also this work is going to be included in a research work I'm doing, so I'dprefer to stick to Python and C, without having to put cython sources or cython-generated c modules(which I know are almost completely unreadable from a human point of view. Or at least the ones I saw).

Anyway thank you for the suggestion.

 

[Giacomo Alzetta]

Il giorno venerdì 10 agosto 2012 20:50:08 UTC+2, Stefan Behnel ha scritto:

Giacomo Alzetta, 10.08.2012 10:20:





You should take a look at Cython. It makes these things way easier and

safer than with manually written C code. It will save you a lot of code,

debugging and general hassle.



Stefan

I already know Cython, but I hope to learn a bit how python works from the C-side writing this extension.

Also this work is going to be included in a research work I'm doing, so I'dprefer to stick to Python and C, without having to put cython sources or cython-generated c modules(which I know are almost completely unreadable from a human point of view. Or at least the ones I saw).

Anyway thank you for the suggestion.

 

[Stefan Behnel]

Giacomo Alzetta, 11.08.2012 08:21:

I'd prefer to stick to Python and C, without having to put cython
sources or cython-generated c modules (which I know are almost
completely unreadable from a human point of view. Or at least the ones I
saw).

And the cool thing is: you don't have to read them.

Stefan

 

[Giacomo Alzetta]

Il giorno sabato 11 agosto 2012 08:40:18 UTC+2, Stefan Behnel ha scritto:

Giacomo Alzetta, 11.08.2012 08:21:







And the cool thing is: you don't have to read them.



Stefan

Yes, but since all this code will end-up in the hands of some examiner, he'll have to read them.

 

[Giacomo Alzetta]

Il giorno sabato 11 agosto 2012 08:40:18 UTC+2, Stefan Behnel ha scritto:

Giacomo Alzetta, 11.08.2012 08:21:







And the cool thing is: you don't have to read them.



Stefan

Yes, but since all this code will end-up in the hands of some examiner, he'll have to read them.

 

[Stefan Behnel]

Giacomo Alzetta, 11.08.2012 10:55:

Il giorno sabato 11 agosto 2012 08:40:18 UTC+2, Stefan Behnel ha scritto:

Yes, but since all this code will end-up in the hands of some examiner, he'll have to read them.

I'd just ask him what he prefers: beautiful Python code with a couple of
static type declarations in it, or verbose C code with lots of C-isms and
C-API-isms all over the place. If he's smart enough, he'll force you into
writing the code in Cython.

Stefan