C-API, tp_dictoffset vs tp_members

U

Ulrich Eckhardt

Hi!

When would I use PyObject_SetAttrString/tp_dictoffset instead of tp_members?

I have a predefined set of members, some of which are optional. The problem
I had with an embedded dictionary was that I can't see its elements using
"dir()". Now I just converted to using tp_members, and it still seems to
work correctly for the cases I tested. Even better, I can declare the fields
as read-only then and add doc-strings. So, I wonder, what made the original
author[2] use tp_dictoffset instead?

In case you wonder, I'm looking at the Python bindings for GDB[1], file
gdb/python/python-type.c, struct field_object.


Thanks!

Uli


[1] http://sourceware.org/gdb/wiki/PythonGdb
[2] Yes I could ask them, but I'd like to get an understanding of the pros
and cons.
 
M

Martin v. Löwis

When would I use PyObject_SetAttrString/tp_dictoffset instead of tp_members?

When I have a variable list of attributes, and cannot statically know
what those attributes might be.
I have a predefined set of members, some of which are optional.

Having optional fields is also a good reason.
The problem
I had with an embedded dictionary was that I can't see its elements using
"dir()".

How so? That should work fine.
Now I just converted to using tp_members, and it still seems to
work correctly for the cases I tested.

So how do you do optional fields now? In particular, how would you do
optional integers?
Even better, I can declare the fields
as read-only then and add doc-strings. So, I wonder, what made the original
author[2] use tp_dictoffset instead?

Most likely, it was the simplest approach. This code only wants to set
the attributes, and never read them. It's easier to maintain: if you
want to add a field with tp_members, you have to change multiple places,
and you have to consider garbage collection (assuming you have embedded
objects). With tp_dictoffset, adding another attribute is easy.

Regards,
Martin
 
U

Ulrich Eckhardt

Martin v. Löwis said:
Having optional fields is also a good reason.

What is the use of T_OBJECT_EX vs T_OBJECT in PyMemberDef then? I would
have though that the former describes an optional field, because the
behaviour of accessing it when it is NULL is the same as accessing a
nonexistent field. However, I see that it still appears in the dir()
output even if it is NULL, so it seems I'm misunderstanding this.
How so? That should work fine.
['__class__', '__delattr__', '__doc__', '__getattribute__',
'__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__str__']'example value'

The 'example_attribute' can be accessed, but it is not visible in the
dir() output. The code for this is below.

So how do you do optional fields now? In particular, how would you do
optional integers?

See above, I would expect T_OBJECT_EX to do that.

Thanks!

Uli



/* example.c

gcc -Wall example.c -shared -I /usr/include/python2.5 -o example.so
*/

#include <Python.h>
#include <structmember.h>

typedef struct {
PyObject_HEAD
PyObject* dict;
} Example;

static void
Example_dealloc(PyObject* self)
{
Example* ex = (Example*)self;
Py_XDECREF(ex->dict);
self->ob_type->tp_free(self);
}

static PyObject *
Example_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Example* self;
PyObject* attr;

self = (Example*)type->tp_alloc(type, 0);
if(self==NULL)
return NULL;

self->dict = PyDict_New();
if(self->dict==NULL) {
Py_DECREF(self);
return NULL;
}

attr = PyString_FromString("example value");
if(PyObject_SetAttrString((PyObject*)self, "example_attribute", attr)<0) {
Py_DECREF(self);
return NULL;
}

return (PyObject*)self;
}

static PyTypeObject ExampleType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"example.Example", /*tp_name*/
sizeof(Example), /*tp_basicsize*/
0, /*tp_itemsize*/
Example_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT, /*tp_flags*/
0, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
offsetof(Example, dict), /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
Example_new, /* tp_new */
};

void
initexample(void)
{
PyObject* m;

if (PyType_Ready(&ExampleType) < 0)
return;

m = Py_InitModule3("example", 0,
"Example module that creates an extension type.");

if (m == NULL)
return;

Py_INCREF(&ExampleType);
PyModule_AddObject(m, "Example", (PyObject*)&ExampleType);
}
 
M

Martin v. Löwis

I have a predefined set of members, some of which are optional.
What is the use of T_OBJECT_EX vs T_OBJECT in PyMemberDef then?

Right - this works for optional objects. However, it can't possibly
work for any of the other fields.
I would
have though that the former describes an optional field, because the
behaviour of accessing it when it is NULL is the same as accessing a
nonexistent field. However, I see that it still appears in the dir()
output even if it is NULL, so it seems I'm misunderstanding this.

I suppose that's because there will still be a descriptor for the field
in the class.
The 'example_attribute' can be accessed, but it is not visible in the
dir() output. The code for this is below.

I see. So you do need a tp_members list:

static PyMemberDef example_members[] = {
{"__dict__", T_OBJECT, offsetof(Example, dict), READONLY},
{0}
};

Perhaps dir() should know about tp_dictoffset, but alas, it doesn't.

Regards,
Martin
 
U

Ulrich Eckhardt

Martin v. Löwis said:
Right - this works for optional objects. However, it can't possibly
work for any of the other fields.

I have two members, one T_OBJECT and one T_OBJECT_EX. Both are NULL and both
still appear in the dir() output. For one, accessing it returns 'None', for
the other it raises an exception. I would have expected an element that is
not accessible, not even 'None', to also not be visible in dir().
I suppose that's because there will still be a descriptor for the
field in the class.

So is that intentional or is it dir() that could be improved there? Should I
file a bugreport?


Thanks for the tip with __dict__ in tp_members, that does the job!

Uli
 
M

Martin v. Löwis

I have a predefined set of members, some of which are optional.
I have two members, one T_OBJECT and one T_OBJECT_EX. Both are NULL and both
still appear in the dir() output. For one, accessing it returns 'None', for
the other it raises an exception. I would have expected an element that is
not accessible, not even 'None', to also not be visible in dir().

I understood your expectation already. I maintain my theory that it
shows up because there is a descriptor on the class.
So is that intentional or is it dir() that could be improved there?

It's probably both.
Should I file a bugreport?

Only if you can provide a patch also. My guess is that when you have
the patch completed, you might realize that it is not an improvement,
despite achieving what you wanted to achieve.

Regards,
Martin
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,481
Members
44,900
Latest member
Nell636132

Latest Threads

Top