Help with implementing callback functions using ctypes

Discussion in 'Python' started by jamadagni, May 8, 2013.

  1. jamadagni

    jamadagni Guest

    Hello. I am running Kubuntu Raring with Python 3.3.1. (I mostly don't myself use Py2.)

    I have the below C program spiro.c (obviously a simplified testcase) which I compile to a sharedlib using clang -fPIC -shared -o libspiro.so spiro.c, sudo cp to /usr/lib and am trying to call from a Python script spiro.py using ctypes. However it would seem that the first call to the callback results in a segfault. (The line before the callback is executed.) I tried debugging using gdb but perhaps because of my insufficient knowledge I couldn't find out what is causing the problem.

    I hope some of the kind knowledgeable souls here can help me out:

    -- spiro.c

    # include <stdbool.h>
    # include <stdio.h>

    typedef struct _bezctx bezctx ;
    struct _bezctx {
    void ( * moveto ) ( bezctx * bc, double x, double y, bool starts_open_path ) ;
    void ( * lineto ) ( bezctx * bc, double x, double y ) ;
    void ( * curveto ) ( bezctx * bc, double c1x, double c1y, double c2x, double c2y, double x, double y ) ;
    void ( * track_seg ) ( bezctx * bc, unsigned int start_node_idx ) ;
    } ;

    typedef struct {
    double x, y ;
    char ty ;
    } spiro_cp ;

    bool spiro_to_bezier_strict ( const spiro_cp src [], int src_len, bezctx * bc ) {
    for ( int i = 0 ; i < src_len ; ++i ) {
    const spiro_cp * p = &(src) ;
    printf ( "%g %g %c\n", p->x, p->y, p->ty ) ;
    if ( i % 2 == 0 )
    bc -> moveto ( bc, p->x, p->y, i % 3 ) ;
    else
    bc -> lineto ( bc, p->x, p->y ) ;
    bc -> curveto ( bc, 233, 150, 267, 150, 300, 100 ) ;
    bc -> track_seg ( bc, i ) ;
    }
    return src_len % 2 == 0 ;
    }

    -- spiro.py

    from ctypes import *

    # incomplete struct definition needed for pointer below
    class bezctx ( Structure ) : pass

    # pointer to function types for callback
    MoveToFnType = CFUNCTYPE ( None, POINTER ( bezctx ), c_double, c_double, c_bool )
    LineToFnType = CFUNCTYPE ( None, POINTER ( bezctx ), c_double, c_double )
    CurveToFnType = CFUNCTYPE ( None, POINTER ( bezctx ), c_double, c_double, c_double, c_double, c_double, c_double )
    TrackSegmentFnType = CFUNCTYPE ( None, POINTER ( bezctx ), c_uint )

    # completing the struct definition
    bezctx._fields_ = [ ( "moveto" , MoveToFnType ),
    ( "lineto" , LineToFnType ),
    ( "curveto" , CurveToFnType ),
    ( "track_seg", TrackSegmentFnType ) ]

    # the python callbacks
    def moveTo ( bc, x, y, startsOpenPath ) :
    print ( "moveTo", bc, x, y, startsOpenPath )
    def lineTo ( bc, x, y ) :
    print ( "lineTo", bc, x, y )
    def curveTo ( bc, c1x, c1y, c2x, c2y, x, y ) :
    print ( "curveTo", bc, c1x, c1y, c2x, c2y, x, y )
    def trackSegment ( bc, start_node_idx ) :
    print ( "trackSegment", bc, start_node_idx )

    # instance of struct
    bc = bezctx ()
    bc . moveto = MoveToFnType ( moveTo )
    bc . lineto = LineToFnType ( lineTo )
    bc . curveto = CurveToFnType ( curveTo )
    bc . track_seg = TrackSegmentFnType ( trackSegment )

    # another struct
    class spiro_cp ( Structure ) :
    _fields_ = [ ( "x", c_double ), ( "y", c_double ), ( "ty", c_char ) ]

    # array of struct instances
    srclist = [ spiro_cp ( 147, 215, b'o' ),
    spiro_cp ( 168, 136, b'o' ),
    spiro_cp ( 294, 184, b'[' ),
    spiro_cp ( 381, 136, b']' ),
    spiro_cp ( 445, 204, b'c' ),
    spiro_cp ( 364, 235, b'c' ),
    spiro_cp ( 266, 285, b'v' ) ]
    src = ( spiro_cp * len ( srclist ) ) ( *srclist )

    # open the sharedlib
    libspiro = CDLL ( "libspiro.so" )
    spiro_to_bezier_strict = libspiro.spiro_to_bezier_strict

    # call the C function
    spiro_to_bezier_strict ( src, len ( src ), bc )

    -- bash session

    $ python3 spiro-ctypes.py
    147 215 o
    Segmentation fault (core dumped)
     
    jamadagni, May 8, 2013
    #1
    1. Advertising

  2. jamadagni

    dieter Guest

    jamadagni <> writes:
    > ...

    I cannot help you with "ctypes". But, if you might be able to use
    "cython", then calling callbacks is not too difficult
    (you can find an example in e.g. my "dm.xmlsec.binding").

    Note, however, that properly handling the GIL ("Global Interpreter Lock")
    may be of great importance when the Python/C boundary is crossed --
    at least when you intend to use your code in a multi thread environment.
     
    dieter, May 9, 2013
    #2
    1. Advertising

  3. dieter, 09.05.2013 07:54:
    > jamadagni writes:
    >> ...

    > I cannot help you with "ctypes". But, if you might be able to use
    > "cython", then calling callbacks is not too difficult


    +1 for using Cython. It also has (multi-)source level gdb support, which
    greatly helps in debugging crashes like this one.

    http://docs.cython.org/src/userguide/debugging.html


    > (you can find an example in e.g. my "dm.xmlsec.binding").


    An "official" example is here:

    https://github.com/cython/cython/tree/master/Demos/callback


    > Note, however, that properly handling the GIL ("Global Interpreter Lock")
    > may be of great importance when the Python/C boundary is crossed --
    > at least when you intend to use your code in a multi thread environment.


    Cython makes this really easy. You can use the "with" statement to release
    the GIL when you don't need it, and the compiler will produce errors when
    you try to do things that require the GIL while you don't own it.

    Stefan
     
    Stefan Behnel, May 9, 2013
    #3
  4. jamadagni

    Nobody Guest

    On Wed, 08 May 2013 04:19:07 -0700, jamadagni wrote:

    > I have the below C program spiro.c (obviously a simplified testcase)
    > which I compile to a sharedlib using clang -fPIC -shared -o libspiro.so
    > spiro.c, sudo cp to /usr/lib and am trying to call from a Python script
    > spiro.py using ctypes. However it would seem that the first call to the
    > callback results in a segfault.


    > # call the C function
    > spiro_to_bezier_strict ( src, len ( src ), bc )


    This line should be:

    spiro_to_bezier_strict ( src, len ( src ), byref(bc) )

    Without the byref(...), it will try to pass a copy of the structure rather
    than passing a pointer to it.
     
    Nobody, May 9, 2013
    #4
  5. jamadagni

    jamadagni Guest

    On Friday, May 10, 2013 12:02:08 AM UTC+5:30, Nobody wrote:
    > This line should be:
    > spiro_to_bezier_strict ( src, len ( src ), byref(bc) )
    > Without the byref(...), it will try to pass a copy of the structure rather
    > than passing a pointer to it.


    Wow silly silly mistake of mine, passing an object where a pointer is expected. Alternatively, if I specify:

    spro_to_bezier_strict.argtypes = [ POINTER(spiro_cp), c_int, POINTER(bezctx) ]

    it works equally good and apparently automatically converts the object specified to a pointer to it (which is good and intuitive, I guess, though against C practice).

    I guess if one wants to be safe when interfacing to C functions, one shouldalways specify argtypes (even if not restype).

    But really, a segfault? I would expect a more meaningful error or an exception to be raised. Or is the black magic occurring at a level lower than where Python's control goes?

    If I had seen this post I might not have even spent time on Cython. (For some reason Google Groups doesn't send me email updates even if I ask it to.)Using CTYpes I was able to implement a callback with less coding (for thisparticular requirement) than Cython which required an extra wrapper layer on top of the library to pass through a void* representing the Python callback (or a state variable would have to be used which may mean no thread-safety).

    But I guess Cython would be faster for those who want that. Even though there is an extra function call every time (for the wrapper) it would be faster since it's at the C level rather than at the Python level (or something like that).

    Thanks to all who helped on this, and also the people over at the cython-users list who helped me on the Cython wrapper too.
     
    jamadagni, May 24, 2013
    #5
  6. On Wed, May 8, 2013 at 10:54 PM, dieter <> wrote:

    > jamadagni <> writes:
    > > ...

    > I cannot help you with "ctypes". But, if you might be able to use
    > "cython", then calling callbacks is not too difficult
    > (you can find an example in e.g. my "dm.xmlsec.binding").
    >
    > Note, however, that properly handling the GIL ("Global Interpreter Lock")
    > may be of great importance when the Python/C boundary is crossed --
    > at least when you intend to use your code in a multi thread environment.
    >


    Cython is good.

    So is the new cffi, which might be thought of as a safer (API-level)
    version of ctypes (which is ABI-level).
     
    Dan Stromberg, May 24, 2013
    #6
  7. On Friday, May 24, 2013 8:56:28 AM UTC+5:30, Dan Stromberg wrote:
    > Cython is good. So is the new cffi, which might be thought of as a
    > safer (API-level) version of ctypes (which is ABI-level).


    Hi -- can you clarify what is this new CFFI and where I can get it? In the Python 3 library reference I seem to see only CTypes.
     
    Shriramana Sharma, May 26, 2013
    #7
  8. https://cffi.readthedocs.org/en/release-0.6/

    ----------------------------------------
    > Date: Sun, 26 May 2013 09:12:10 -0700
    > Subject: Re: Help with implementing callback functions using ctypes
    > From:
    > To:
    >
    > On Friday, May 24, 2013 8:56:28 AM UTC+5:30, Dan Stromberg wrote:
    >> Cython is good. So is the new cffi, which might be thought of as a
    >> safer (API-level) version of ctypes (which is ABI-level).

    >
    > Hi -- can you clarify what is this new CFFI and where I can get it? In the Python 3 library reference I seem to see only CTypes.
    > --
    > http://mail.python.org/mailman/listinfo/python-list
     
    Carlos Nepomuceno, May 26, 2013
    #8
  9. On Sun, May 26, 2013 at 9:12 AM, Shriramana Sharma <>wrote:

    > On Friday, May 24, 2013 8:56:28 AM UTC+5:30, Dan Stromberg wrote:
    > > Cython is good. So is the new cffi, which might be thought of as a
    > > safer (API-level) version of ctypes (which is ABI-level).

    >
    > Hi -- can you clarify what is this new CFFI and where I can get it? In the
    > Python 3 library reference I seem to see only CTypes.
    >


    CFFI is like CTypes, but it allows you to do things like see C preprocessor
    symbols.

    https://cffi.readthedocs.org/en/release-0.6/

    I don't think it comes with CPython, at least not yet. But I believe it
    comes with the latest Pypy.
     
    Dan Stromberg, May 26, 2013
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Henk Punt
    Replies:
    0
    Views:
    408
    Henk Punt
    Jul 23, 2004
  2. James Hu

    callback for ctypes

    James Hu, Nov 1, 2005, in forum: Python
    Replies:
    3
    Views:
    400
    Peter Hansen
    Nov 2, 2005
  3. Replies:
    1
    Views:
    383
  4. Replies:
    0
    Views:
    514
  5. OJ
    Replies:
    2
    Views:
    495
Loading...

Share This Page