Need help with C extension module

Discussion in 'Python' started by chris, Sep 7, 2005.

  1. chris

    chris Guest

    This is my first attempt at undertaking a C extension module. I want
    to wrap an existing C library so I can call the functions from Python.
    There are only two functions I'm interested in calling. I did mess
    with Pyrex a bit and Swig, to no avail, so I turned to doing it by
    hand. Using the example in Programming Python, I did get the easier of
    the two functions working--only takes a string parameter. I'm stuck
    now on the other function and not sure how to wrap it, because it
    involves some structs. Here's a simplified version of the C:

    struct In
    {
    int x;
    char* s;
    ... (only primitive data types)
    };

    struct Out
    {
    int y;
    char* s;
    ... (only primitive data types)
    };

    Out* func(In* x, Out* y);

    So the function takes pointers to the two structs, and fills out the
    output struct and also returns the pointer to it. I would envision the
    Python looking like

    in = In()
    in.y = 1
    in.s = "abc"
    ....

    out = func(in)

    maybe? Just no idea how to deal with the structs in the C extension
    module code.

    Any tips appreciated.

    Thanks,
    Chris
    chris, Sep 7, 2005
    #1
    1. Advertising

  2. chris

    Robert Kern Guest

    chris wrote:
    > This is my first attempt at undertaking a C extension module. I want
    > to wrap an existing C library so I can call the functions from Python.
    > There are only two functions I'm interested in calling. I did mess
    > with Pyrex a bit and Swig, to no avail, so I turned to doing it by
    > hand. Using the example in Programming Python, I did get the easier of
    > the two functions working--only takes a string parameter. I'm stuck
    > now on the other function and not sure how to wrap it, because it
    > involves some structs. Here's a simplified version of the C:
    >
    > struct In
    > {
    > int x;
    > char* s;
    > ... (only primitive data types)
    > };
    >
    > struct Out
    > {
    > int y;
    > char* s;
    > ... (only primitive data types)
    > };
    >
    > Out* func(In* x, Out* y);
    >
    > So the function takes pointers to the two structs, and fills out the
    > output struct and also returns the pointer to it. I would envision the
    > Python looking like
    >
    > in = In()
    > in.y = 1
    > in.s = "abc"
    > ...
    >
    > out = func(in)
    >
    > maybe? Just no idea how to deal with the structs in the C extension
    > module code.


    It's *not* a straightforward task. You have to define an extension type. See

    http://docs.python.org/ext/defining-new-types.html

    However, Pyrex really does make defining extension types much, much
    easier. I highly suggest trying to debug your Pyrex code before learning
    all of the boilerplate necessary to create extension types in pure C.

    --
    Robert Kern


    "In the fields of hell where the grass grows high
    Are the graves of dreams allowed to die."
    -- Richard Harter
    Robert Kern, Sep 7, 2005
    #2
    1. Advertising

  3. chris

    chris Guest

    Any tips on what the pyrex should look like for my example?
    chris, Sep 8, 2005
    #3
  4. chris

    Robert Kern Guest

    chris wrote:
    > Any tips on what the pyrex should look like for my example?


    # Untested and off the top of my head; please read the Pyrex
    # documentation and correct my mistakes before using any of this!
    # Notably, I'm pretty sure the __init__ and __dealloc__ bits are
    # slightly wrong.

    cdef extern from "Python.h":
    void *PyMem_Malloc(unsigned int size)
    void PyMem_Free(void *buf)

    ctypedef struct In:
    int x
    char *s
    # ...

    ctypedef struct Out:
    int y
    char *s
    # ...

    cdef Out *func(Int *x, Out *y)

    cdef class Output:
    cdef Out *output

    def __init__(self):
    self.output = <Out*>PyMem_Malloc(sizeof(Out))

    def __dealloc__(self):
    PyMem_Free(self.output)

    property y:
    def __get__(self):
    return self.output.y

    # ...

    cdef class Input:
    cdef In *input

    def __init__(self):
    self.input = <In*>PyMem_Malloc(sizeof(In))

    def __dealloc__(self):
    PyMem_Free(self.input)

    property x:
    def __get__(self):
    return self.input.x

    def __set__(self, value):
    self.input.x = value

    # ...

    def func(self):
    output = Output()
    func(self.input, output.output)
    return output

    --
    Robert Kern


    "In the fields of hell where the grass grows high
    Are the graves of dreams allowed to die."
    -- Richard Harter
    Robert Kern, Sep 8, 2005
    #4
  5. chris

    chris Guest

    chris, Sep 8, 2005
    #5
  6. chris

    chris Guest

    Ok, I found further examples on the Internet and got something working
    (it seems), but I have a question about the memory management. The
    example I found did not include any of the PyMem_... functions.

    Here's roughly what I have working:

    cdef extern from "my.h":
    cdef struct inputs:
    char *x

    cdef struct outputs:
    int y

    outputs *func(inputs *x, outputs *y)
    int init(char* fname)

    class Analyzer:
    def __init__(self, fname):
    init(fname)

    # inp is my python "Inputs" object.
    def myfunc(self, inp):
    cdef inputs* i
    i.x= inp.x

    cdef outputs* o
    o = func(i, o)
    return o.y

    class Inputs:
    def __init__(self):
    self.x = ""

    So there is no explicit memory management going on there as in Robert's
    example. Is this ok?

    Thanks,
    Chris
    chris, Sep 8, 2005
    #6
  7. chris

    Ken Seehart Guest

    chris wrote:
    > This is my first attempt at undertaking a C extension module. I want
    > to wrap an existing C library so I can call the functions from Python.
    > There are only two functions I'm interested in calling. I did mess
    > with Pyrex a bit and Swig, to no avail, so I turned to doing it by
    > hand. Using the example in Programming Python, I did get the easier of
    > the two functions working--only takes a string parameter. I'm stuck
    > now on the other function and not sure how to wrap it, because it
    > involves some structs. Here's a simplified version of the C:
    >
    > struct In
    > {
    > int x;
    > char* s;
    > ... (only primitive data types)
    > };
    >
    > struct Out
    > {
    > int y;
    > char* s;
    > ... (only primitive data types)
    > };
    >
    > Out* func(In* x, Out* y);
    >
    > So the function takes pointers to the two structs, and fills out the
    > output struct and also returns the pointer to it. I would envision the
    > Python looking like
    >
    > in = In()
    > in.y = 1
    > in.s = "abc"
    > ...
    >
    > out = func(in)
    >
    > maybe? Just no idea how to deal with the structs in the C extension
    > module code.
    >
    > Any tips appreciated.
    >
    > Thanks,
    > Chris
    >


    Since others have responded about Pyrex, I'll just add my two cents,
    which may or not apply to you.

    The idea of filling in a struct, passing it to a function, then
    returning a struct, is a typical c-ism. If the only purpose of the
    structs is to pass data to the function and receive the result, read on.
    On the other hand, if the structures represent "things" in the
    object-oriented sense, just ignore what I am saying.

    I would wrap the function such that the python code simply takes a set
    of parameters and returns a tuple. This is much more pythonic, and much
    easier to code.

    y,s,... = func(x,s,...)

    My untested pseudocode would look something like this:

    static PyObject *map_sweep(map__object *self, PyObject *args)
    {
    In input;
    Out output;

    if (PyArg_ParseTuple(args, "is...", &(In.x), &(In.s), ...))
    {
    return NULL;
    }

    func(&In, &Out);

    return Py_BuildValue("is...", Out.y, Out.s, ...);
    }
    Ken Seehart, Sep 13, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Anand
    Replies:
    3
    Views:
    878
    Tim Daneliuk
    Nov 8, 2003
  2. Christian Seberino
    Replies:
    3
    Views:
    1,169
    Christian Seberino
    Feb 5, 2004
  3. Ilariu Raducan
    Replies:
    2
    Views:
    341
    Ilariu Raducan
    Jul 14, 2004
  4. Steve
    Replies:
    3
    Views:
    26,949
    Steve
    Aug 25, 2006
  5. Floris Bruynooghe
    Replies:
    1
    Views:
    332
    Floris Bruynooghe
    Dec 24, 2008
Loading...

Share This Page