Extending Python - Understanding Flags...

[Redefined Horizons]

I'm trying to understand the argument flags that are used in the
method table of an extension module written in C.

First let me ask this question about the method table. Is it an C
array named "PyMethodDef"?

Now, onto my questions about the arguments:

[1] I see that even when the Python function we are supplying takes no
arguments, (the argument flag is METH_NOARGS), that we still pass the
C function a reference to "self". Is this "self" actually a pointer to
the section of memory that represents the object in Python that is
calling the method?

[2] What is the difference between "positional" arguments indicated by
the METH_VARARGS argument flag, and "keyword" arguments that are
indicated by the METH_KEYWORDS argument flag? Does one determine the
identity of an argument based on the order or position in which the
arguments are passed to the C function, while the other uses keys in a
dictionary to identify the arguments?

That should be the start of my customization questions... :]

If it is any consolation I hope to take what I learn and put together
a tutorial on expanding Python for those that only have a little
experience with C programming, like myself. I plan to explain
everything like I was teaching a 6-year old. :]

I was just going to put this on a wiki, but is there a way to include
with other Python documentation on an "official" site?

Thanks for the help.

Scott Huey

 

[John Machin]

I'm trying to understand the argument flags that are used in the
method table of an extension module written in C.

First let me ask this question about the method table. Is it an C
array named "PyMethodDef"?

Its *type* is PyMethodDef (there is a typedef for that in python.h). You
can name it what you like, but in a module called "foo", calling this
thing "foo_methods" might be a reasonable idea.

Please read section 1.4 of the Extending and Embedding Manual.

Now, onto my questions about the arguments:

[1] I see that even when the Python function we are supplying takes no
arguments, (the argument flag is METH_NOARGS), that we still pass the
C function a reference to "self". Is this "self" actually a pointer to
the section of memory that represents the object in Python that is
calling the method?

This is independent of whether it is NOARGS or VARARGS. Irrespective of
whether the C function is intended to appear to the Python caller as a
method of a type/class, or as module-level function, its first arg is
self. In the latter case, it should be unused by the called function. If
you were to print its contents (using the %p format in printf, if
available), you would find that it is NULL.

Please read section 1.1 of the manual.

[2] What is the difference between "positional" arguments indicated by
the METH_VARARGS argument flag, and "keyword" arguments that are
indicated by the METH_KEYWORDS argument flag? Does one determine the
identity of an argument based on the order or position in which the
arguments are passed to the C function, while the other uses keys in a
dictionary to identify the arguments?

The difference is the same as if the called function is a Python
function. In one sense, there is no difference -- even if you declare
keywords, a call can still use positional arguments. Note that
METH_KEYWORDS is not a substitute for METH_VARARGS; it's an add-on -- or
a bitwise-or-on

def afunction(foo, bar, zot):
# positional args, caller's only option is afunction(42, 1, 666)
def bfunction(foo=0, bar="plugh", zot=None):
# keyword args, caller can do:
bfunction(zot=10**100) # using a keyword: args foo & bar get default values
bfunction(10, "xyzzy") # positional: foo = 10; bar = "xyzzy"; zot = 10**100

In a Python function, nothing extra needs to be done.
In a C-extension function, a char * [] is required to supply the names
of the arguments. The default value caper is handled by the caller
initialising the C variables that will receive the argument values.

Manual: please read section 1.4 and section 1.8.

That should be the start of my customization questions... :]

If it is any consolation I hope to take what I learn and put together
a tutorial on expanding Python for those that only have a little
experience with C programming, like myself. I plan to explain
everything like I was teaching a 6-year old. :]

I was just going to put this on a wiki, but is there a way to include
with other Python documentation on an "official" site?

The manual is in fact written as a tutorial. You may prefer to propose
changes to the manual, rather than to publish yet another document.

A good idea is to download the Python source and go looking in the
Modules directory. If module foo is implemented in C, its source will be
there as foomodule.c. For a simple module that provides only functions,
look at e.g. binascii. For something that defines types, try datetime.
They and others should provide you with good examples.

HTH,
John

 

[John Machin]

def bfunction(foo=0, bar="plugh", zot=None):
# keyword args, caller can do:
bfunction(zot=10**100) # using a keyword: args foo & bar get default values
bfunction(10, "xyzzy") # positional: foo = 10; bar = "xyzzy"; zot = 10**100

The last line above should end with
zot = None
instead of
zot = 10**100

 

[Alex Martelli]

If it is any consolation I hope to take what I learn and put together
a tutorial on expanding Python for those that only have a little
experience with C programming, like myself. I plan to explain
everything like I was teaching a 6-year old. :]

I was just going to put this on a wiki, but is there a way to include
with other Python documentation on an "official" site?

The manual is in fact written as a tutorial. You may prefer to propose
changes to the manual, rather than to publish yet another document.

I don't think the manual even TRIES to address "those that only have a
little experience with C programming", nor should it -- there's a vast
niche for a truly tutorial document based on learn-by-doing, which takes
somebody with a "C 101" course as their only C experience, and a good
knowledge of Python, and turns them into writers of C extension modules
for Python.

I suggest to the OP that he starts this effort as a wiki, publicize it
widely but with good taste to try and attract willing helpers, and think
about possibly get it included on Python's official site only if and
when it becomes more mature and well-accepted. Best of luck: I think
it's a worthwhile project and I hope I can be of help (I have published
a number of shorter documents on this subject, and I'd be happy to allow
them to be shared in as much as I retain copyright on most of them --
alas, can't do that for the chapter of Python in a Nutshell that deals
with writing Python extensions in C, since that's (C) O'Reilly!-).


Alex