C Module question

[boblatest]

Hello,

I'm trying to write a Python extension module in C for the first time.
I have two questions:

1. How can I pass a file-like object into the C part? The PyArg_*
functions can convert objects to all sort of types, but not FILE*.

2. How can I preserve information needed in the C part between calls
to my module functions? The way I've thought of is:
- put this information in a dynamically allocated struct
- pass it out of and into the C module as CObject into/from an
intermediate Python wrapper module where it gets stored in a Python
object

Is this the way to do it?

Thanks.
robert

 

[Marc 'BlackJack' Rintsch]

1. How can I pass a file-like object into the C part? The PyArg_*
functions can convert objects to all sort of types, but not FILE*.
http://docs.python.org/c-api/file.html#PyFile_AsFile

2. How can I preserve information needed in the C part between calls to
my module functions? The way I've thought of is: - put this information
in a dynamically allocated struct - pass it out of and into the C module
as CObject into/from an intermediate Python wrapper module where it gets
stored in a Python object

Why passing it in and out? Simply use the C module level to store the
information.

Ciao,
Marc 'BlackJack' Rintsch

 

[Floris Bruynooghe]

Hi

1. How can I pass a file-like object into the C part? The PyArg_*
functions can convert objects to all sort of types, but not FILE*.

Parse it as a generic PyObject object (format string of "O" in
PyArg_*), check the type and cast it. Or use "O!" as format string
and the typechecking can be done for you, only thing left is casting
it.

See http://docs.python.org/c-api/arg.html and http://docs.python.org/c-api/file.html
for exact details.

(2 is answered already...)

Regards
Floris

 

[Floris Bruynooghe]

Parse it as a generic PyObject object (format string of "O" in
PyArg_*), check the type and cast it.  Or use "O!" as format string
and the typechecking can be done for you, only thing left is casting
it.

Seehttp://docs.python.org/c-api/arg.htmlandhttp://docs.python.org/c-api/file.html
for exact details.

Sorry, I probably should have mentioned you want to cast the object to
PyFileObject and then use the PyFile_AsFile() function to get the
FILE* handle.


Floris

 

[boblatest]

http://docs.python.org/c-api/file.html#PyFile_AsFile

Yes, got it. At first I thought I had to use the "Parse" functions for
my args, but in fact I can of course just access the args as a tuple
(and then do the type checking myself).

Why passing it in and out?  Simply use the C module level to store the
information.

But if I create several instances of a class (in the wrapper module)
my C methods won't know which object they were called on.

robert

 

[boblatest]

Sorry, I probably should have mentioned you want to cast the object to
PyFileObject and then use the PyFile_AsFile() function to get the
FILE* handle.

Yes, I figured that out by now. Sadly this doesn't work on "file-like"
objects like those that are created by opening bz2 files (using the
bz2 lib). Not that I have any idea on how that should work anyway.

I still can resolve this issue in my wrapper module and let the C part
deal with the raw buffer contents.

All in all I must say that implementing a C extension is a piece of
cake. Had I known that it was this straightforward I wouldn't have
asked my questions in the first place. Making the whole thing more
robust will be a bit more difficult, and I need to find out how to
deal with ressources that are dynamically allocated on the C side.

But that should be easy, and I'll just keep all the Python OO and
Exceptions stuff in the wrapper and call my C stuff from there in a
more or less straightforward C manner.

Thanks to everybody for helping out,

robert
(I'll be back for sure)

 

[Marc 'BlackJack' Rintsch]

Yes, got it. At first I thought I had to use the "Parse" functions for
my args, but in fact I can of course just access the args as a tuple
(and then do the type checking myself).


But if I create several instances of a class (in the wrapper module) my
C methods won't know which object they were called on.

Well you talked about functions in the module not about methods on
objects.

Ciao,
Marc 'BlackJack' Rintsch

 

[Marc 'BlackJack' Rintsch]

All in all I must say that implementing a C extension is a piece of
cake. Had I known that it was this straightforward I wouldn't have asked
my questions in the first place. Making the whole thing more robust will
be a bit more difficult, and I need to find out how to deal with
ressources that are dynamically allocated on the C side.

But that should be easy, and I'll just keep all the Python OO and
Exceptions stuff in the wrapper and call my C stuff from there in a more
or less straightforward C manner.

Then you might considering the `ctypes` module to call your C stuff.
This way it is easier to build the extension and it is also independent
from the Python version.

Ciao,
Marc 'BlackJack' Rintsch

 

[John Machin]

Forget "more robust". You need "quite robust". Seriously consider
abandoning your "piece of cake" euphoria and use Cython or ctypes
instead. At the very least check out the C code that Cython generates
for a simple task and consider whether you really want to hand-code
that.

 

[greg]

Sadly this doesn't work on "file-like"
objects like those that are created by opening bz2 files (using the
bz2 lib).

If the C code you're calling requires a FILE *, then you're
out of luck. There's no way of getting a FILE * from an object
that's not based on an actual PyFile object, since there
simply isn't one to be had.

There is an exception to that -- if the object is one that
has a file descriptor underlying it somewhere (such as a
pipe or socket) you could get that and wrap it using fdopen().

 

[Gabriel Genellina]

En Mon, 10 Nov 2008 11:44:44 -0200, (e-mail address removed)

Yes, I figured that out by now. Sadly this doesn't work on "file-like"
objects like those that are created by opening bz2 files (using the
bz2 lib). Not that I have any idea on how that should work anyway.

Call the file-like methods as you would with any other object method. That
is, if you got an object `fobj` from Python and you want to write a C
string:

char* str="some text\n";
PyObject_CallMethod(fobj, "write", "s", str)

See also PyObject_CallObject and the other variants:
http://docs.python.org/c-api/object.html#PyObject_Call