Re: C API: array of floats/ints from python to C and back

Discussion in 'Python' started by Daniel Fetchinson, Dec 27, 2008.

  1. >> I'm trying to write an extension module in C which contains a single
    >> function with the following prototype:
    >> void func( int N, int * arg1, int * arg2, int * ret );
    >> Here arg1 and arg2 are length N arrays, and the function computes ret
    >> which is also an N length array. From python I'd like to call this
    >> function as
    >> ret = func( [ 1, 2, 3 ], [ 2, 3, 4] )

    >
    > This requirement pretty much dictates the slow answer you have.


    I'm not sure I understand what you mean.

    > > Does this mean that I can only pass the arrays from python to C as
    > > generic python objects and in a later operation I need to get the
    > > elements from this generic python object, construct a C array and pass
    > > that to the C function? ... What's the simplest way of doing this?

    >
    > Either use ctypes, look into array.array, or look into numpy.array.


    I have considered using ctypes but for my needs using the C API
    directly seems more reasonable. array.array and numpy.array doesn't
    fit my needs since I need to do long and complicated operations on the
    two (pretty large) integer arrays that would be too slow using
    array.array and numpy.array (I've verified this claim by benchmarking
    a numpy.array based solution).

    > I'd just use numpy, myself:
    > import numpy
    > total = numpy.array([1, 2, 3]) + numpy.array([2, 3, 4])


    What makes you think I want to add two arrays?

    Cheers,
    Daniel


    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 27, 2008
    #1
    1. Advertising

  2. Daniel Fetchinson

    Robert Kern Guest

    Daniel Fetchinson wrote:

    > I agree that array.array is more efficient than a list but the input
    > for my function will come from PIL and PIL returns a list. So I have a
    > list to begin with which will be passed to the C function.


    With recent versions of PIL, numpy can create an array from an Image very
    quickly, possibly without any copying of memory.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Dec 28, 2008
    #2
    1. Advertising

  3. Daniel Fetchinson

    Aaron Brady Guest

    On Dec 27, 6:06 pm, Scott David Daniels <> wrote:
    > Daniel Fetchinson wrote:
    > >.... I have a list to begin with which will be passed to the C function.

    >
    >  > I assume converting the list to an array.array and passing that to the C
    >
    > > function doesn't make any difference in terms of speed since the
    > > operation itself will be done in the C function anyway.

    >
    > Right, but why bother to do the conversion in C where you'll have to
    > fiddle with refcounts and error propogation?  convert in python, and
    > go to the underlying data in C.


    I think you could use ctypes.c_int* 20, which is an awesome data
    type-- a 20-element array of integers. You could also use this, which
    I just discovered, and is actually moderately disturbing.

    #WARNING unsafe code
    >>> from ctypes import *
    >>> i= c_int(42)
    >>> pi= pointer(i)
    >>> pi[0]

    42
    >>> pi[1]

    0
    >>> pi[2]

    0
    >>> pi[1]= 20
    >>> pi

    <__main__.LP_c_long object at 0x00A97D00>
    >>> pi[1]

    20

    What memory is it overwriting?
    Aaron Brady, Dec 28, 2008
    #3
  4. >> I agree that array.array is more efficient than a list but the input
    >> for my function will come from PIL and PIL returns a list. So I have a
    >> list to begin with which will be passed to the C function.

    >
    > With recent versions of PIL, numpy can create an array from an Image very
    > quickly, possibly without any copying of memory.


    What exactly do you mean? (1) PIL creates a list which can be easily
    converted by numpy to a numpy.array or (2) with the help of numpy one
    can create a numpy.array from an image directly?

    Since I will have to pass the list or numy.array to C anyway I don't
    see any advantage to (1) although (2) can be useful.

    Daniel

    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 28, 2008
    #4
  5. On 12/27/08, Robert Kern <> wrote:
    > Daniel Fetchinson wrote:
    >
    >> I agree that array.array is more efficient than a list but the input
    >> for my function will come from PIL and PIL returns a list. So I have a
    >> list to begin with which will be passed to the C function.

    >
    > With recent versions of PIL, numpy can create an array from an Image very
    > quickly, possibly without any copying of memory.
    >
    > --
    > Robert Kern
    >
    > "I have come to believe that the whole world is an enigma, a harmless enigma
    > that is made terrible by our own mad attempt to interpret it as though it
    > had
    > an underlying truth."
    > -- Umberto Eco
    >
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >


    Hi Robert, now that we are at it let me ask something related. Now I
    managed to get my lists to C, operate on them in C and convert the
    result back to a python tuple. Since this is my first C program that
    uses the python C API I'm sure there are refcounting issues that I'm
    not considering, maybe also memory management too.

    This is the function I have, the corresponding python function will
    take two equal length lists of integers and the C function will
    compute their sum and return the result as a python tuple.


    static PyObject *func( PyObject * self, PyObject * args )
    {
    int j, N;
    int * src1, * src2;
    PyObject *list1, *list2;

    list1 = PyTuple_GetItem( args, 0 );
    N = PyList_Size( list1 );
    src1 = ( int * ) malloc( N * sizeof( int ) );
    for( j = 0; j < N; j++ )
    {
    src1[j] = (int)PyInt_AsLong( PyList_GetItem( list1, j ) );
    }

    list2 = PyTuple_GetItem( args, 1 );
    N = PyList_Size( list2 );
    src2 = ( int * ) malloc( N * sizeof( int ) );
    for( j = 0; j < N; j++ )
    {
    src2[j] = (int)PyInt_AsLong( PyList_GetItem( list2, j ) );
    }

    PyObject * tuple;
    tuple = PyTuple_New( N );
    for( j = 0; j < N; j++ )
    {
    PyTuple_SetItem( tuple, j, PyInt_FromLong( (long)( src1[j] +
    src2[j] ) ) );
    }

    free( src1 );
    free( src2 );

    return tuple;
    }

    Do I have to free the memory occupied by the python objects list1 and
    list2? Do I have to do any refcounting for list1, list2, tuple?

    Any comment on the above code will be very appreciated! If I'm pushed
    in the right direction I'm a fast learner but the beginning steps are
    always hard :)

    Cheers,
    Daniel



    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 28, 2008
    #5
  6. En Sat, 27 Dec 2008 22:54:52 -0200, Daniel Fetchinson
    <> escribió:

    > This is the function I have, the corresponding python function will
    > take two equal length lists of integers and the C function will
    > compute their sum and return the result as a python tuple.
    >
    >
    > static PyObject *func( PyObject * self, PyObject * args )
    > {
    > int j, N;
    > int * src1, * src2;
    > PyObject *list1, *list2;
    >
    > list1 = PyTuple_GetItem( args, 0 );
    > N = PyList_Size( list1 );
    > src1 = ( int * ) malloc( N * sizeof( int ) );
    > for( j = 0; j < N; j++ )
    > {
    > src1[j] = (int)PyInt_AsLong( PyList_GetItem( list1, j ) );
    > }
    >
    > list2 = PyTuple_GetItem( args, 1 );
    > N = PyList_Size( list2 );
    > src2 = ( int * ) malloc( N * sizeof( int ) );
    > for( j = 0; j < N; j++ )
    > {
    > src2[j] = (int)PyInt_AsLong( PyList_GetItem( list2, j ) );
    > }
    >
    > PyObject * tuple;
    > tuple = PyTuple_New( N );
    > for( j = 0; j < N; j++ )
    > {
    > PyTuple_SetItem( tuple, j, PyInt_FromLong( (long)( src1[j] +
    > src2[j] ) ) );
    > }
    >
    > free( src1 );
    > free( src2 );
    >
    > return tuple;
    > }


    As others already said, using a Numpy array or an array.array object would
    be more efficient (and even easier - the C code gets a pointer to an array
    of integers, as usual).

    > Do I have to free the memory occupied by the python objects list1 and
    > list2?


    No. Usually you don't do that for any Python object - just
    increment/decrement its reference count (using Py_INCREF/Py_DECREF).

    > Do I have to do any refcounting for list1, list2, tuple?


    In this case list1 and list2 come from PyTuple_GetItem; the docs say it
    returns a "borrowed reference" (that is, the function doesn't increment
    the refcount itself). So you don't have to decrement it yourself (and it
    isn't necesary to increment it in the first place, because the "args"
    tuple holds a reference, so the object can't disappear until the function
    exits, at least)

    > Any comment on the above code will be very appreciated! If I'm pushed
    > in the right direction I'm a fast learner but the beginning steps are
    > always hard :)


    You MUST check EVERY function call for errors!
    And check the argument's type (how do you know it is a list?). Once you
    are sure the first parameter *is* a list, you may use the "macro" version
    of several functions, like PyList_GET_SIZE and PyList_GET_ITEM.
    You should check that both lists have the same length too.
    And you should check that elements are integers, or convertible to
    integers (in case of error, PyInt_AsLong returns -1 and PyErr_Occurred()
    is true)
    To fill the resulting tuple, use PyTuple_SET_ITEM instead. BTW, why return
    a tuple and not a list?

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 28, 2008
    #6
  7. Daniel Fetchinson

    Robert Kern Guest

    Daniel Fetchinson wrote:
    >>> I agree that array.array is more efficient than a list but the input
    >>> for my function will come from PIL and PIL returns a list. So I have a
    >>> list to begin with which will be passed to the C function.

    >> With recent versions of PIL, numpy can create an array from an Image very
    >> quickly, possibly without any copying of memory.

    >
    > What exactly do you mean? (1) PIL creates a list which can be easily
    > converted by numpy to a numpy.array or (2) with the help of numpy one
    > can create a numpy.array from an image directly?


    (2) a = numpy.asarray(img)

    > Since I will have to pass the list or numy.array to C anyway I don't
    > see any advantage to (1) although (2) can be useful.


    (1) is still somewhat useful, if you're willing to bear the cost of the numpy
    dependency. The conversion code from any Python sequence to a numpy array of the
    desired type and dimensionality is a 2-liner (function call and error check).
    The memory is managed by numpy, so all you have to do is manage the refcount of
    the array object like any other Python object.

    Okay, 5-liner given C's verbosity:

    intx = PyArray_FROM_OTF(pyintx, PyArray_INT, NPY_IN_ARRAY);
    if (!intx) {
    PyErr_SetString(PyExc_ValueError, "intx must be an array of ints");
    goto fail;
    }

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Dec 28, 2008
    #7
  8. >> This is the function I have, the corresponding python function will
    >> take two equal length lists of integers and the C function will
    >> compute their sum and return the result as a python tuple.
    >>
    >>
    >> static PyObject *func( PyObject * self, PyObject * args )
    >> {
    >> int j, N;
    >> int * src1, * src2;
    >> PyObject *list1, *list2;
    >>
    >> list1 = PyTuple_GetItem( args, 0 );
    >> N = PyList_Size( list1 );
    >> src1 = ( int * ) malloc( N * sizeof( int ) );
    >> for( j = 0; j < N; j++ )
    >> {
    >> src1[j] = (int)PyInt_AsLong( PyList_GetItem( list1, j ) );
    >> }
    >>
    >> list2 = PyTuple_GetItem( args, 1 );
    >> N = PyList_Size( list2 );
    >> src2 = ( int * ) malloc( N * sizeof( int ) );
    >> for( j = 0; j < N; j++ )
    >> {
    >> src2[j] = (int)PyInt_AsLong( PyList_GetItem( list2, j ) );
    >> }
    >>
    >> PyObject * tuple;
    >> tuple = PyTuple_New( N );
    >> for( j = 0; j < N; j++ )
    >> {
    >> PyTuple_SetItem( tuple, j, PyInt_FromLong( (long)( src1[j] +
    >> src2[j] ) ) );
    >> }
    >>
    >> free( src1 );
    >> free( src2 );
    >>
    >> return tuple;
    >> }

    >
    > As others already said, using a Numpy array or an array.array object would
    > be more efficient (and even easier - the C code gets a pointer to an array
    > of integers, as usual).


    Thanks, I didn't know that an array.array can be passed to C code as a
    C pointer to an array. So I'll use an array.array because this is
    really convenient.

    >> Do I have to free the memory occupied by the python objects list1 and
    >> list2?

    >
    > No. Usually you don't do that for any Python object - just
    > increment/decrement its reference count (using Py_INCREF/Py_DECREF).


    Great, one headache less :)

    >> Do I have to do any refcounting for list1, list2, tuple?

    >
    > In this case list1 and list2 come from PyTuple_GetItem; the docs say it
    > returns a "borrowed reference" (that is, the function doesn't increment
    > the refcount itself). So you don't have to decrement it yourself (and it
    > isn't necesary to increment it in the first place, because the "args"
    > tuple holds a reference, so the object can't disappear until the function
    > exits, at least)
    >
    >> Any comment on the above code will be very appreciated! If I'm pushed
    >> in the right direction I'm a fast learner but the beginning steps are
    >> always hard :)

    >
    > You MUST check EVERY function call for errors!


    Yes, I know :)

    > And check the argument's type (how do you know it is a list?). Once you
    > are sure the first parameter *is* a list, you may use the "macro" version
    > of several functions, like PyList_GET_SIZE and PyList_GET_ITEM.


    The macro versions are preferable because they are faster?

    > You should check that both lists have the same length too.
    > And you should check that elements are integers, or convertible to
    > integers (in case of error, PyInt_AsLong returns -1 and PyErr_Occurred()
    > is true)
    > To fill the resulting tuple, use PyTuple_SET_ITEM instead. BTW, why return
    > a tuple and not a list?


    No particular reason, other than the fact that I won't need to modify
    these lists/tuples from python so whenever something will not change,
    I use a tuple because it's immutable. Or this is not a very good
    practice? There is no difference between lists and tuples in terms of
    speed I suppose (getitem, setitem, etc).

    Thanks a lot,
    Daniel


    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 28, 2008
    #8
  9. >>>> I agree that array.array is more efficient than a list but the input
    >>>> for my function will come from PIL and PIL returns a list. So I have a
    >>>> list to begin with which will be passed to the C function.
    >>> With recent versions of PIL, numpy can create an array from an Image very
    >>> quickly, possibly without any copying of memory.

    >>
    >> What exactly do you mean? (1) PIL creates a list which can be easily
    >> converted by numpy to a numpy.array or (2) with the help of numpy one
    >> can create a numpy.array from an image directly?

    >
    > (2) a = numpy.asarray(img)


    Thanks, I didn't know about this, maybe it will be useful.

    >> Since I will have to pass the list or numy.array to C anyway I don't
    >> see any advantage to (1) although (2) can be useful.

    >
    > (1) is still somewhat useful, if you're willing to bear the cost of the
    > numpy
    > dependency. The conversion code from any Python sequence to a numpy array of
    > the
    > desired type and dimensionality is a 2-liner (function call and error
    > check).
    > The memory is managed by numpy, so all you have to do is manage the refcount
    > of
    > the array object like any other Python object.
    >
    > Okay, 5-liner given C's verbosity:
    >
    > intx = PyArray_FROM_OTF(pyintx, PyArray_INT, NPY_IN_ARRAY);
    > if (!intx) {
    > PyErr_SetString(PyExc_ValueError, "intx must be an array of ints");
    > goto fail;
    > }


    Yes, the dependency on numpy is my main concern. If it will help with
    my problem I don't mind the dependency actually, so I'll do more
    detailed benchmarks first.

    Thank you,
    Daniel



    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 28, 2008
    #9
  10. En Sun, 28 Dec 2008 00:40:52 -0200, Daniel Fetchinson
    <> escribió:

    >> You MUST check EVERY function call for errors!

    >
    > Yes, I know :)
    >


    Believe me, if you don't, there is a risk of crashing the program. And
    they're a lot harder to find and fix.

    >> And check the argument's type (how do you know it is a list?). Once you
    >> are sure the first parameter *is* a list, you may use the "macro"
    >> version
    >> of several functions, like PyList_GET_SIZE and PyList_GET_ITEM.

    >
    > The macro versions are preferable because they are faster?


    Yes, because they don't do any additional checking. PyList_GET_ITEM just
    retrieves the item; PyList_GetItem checks whether it is called on a list
    object, then checks if index is out of bounds, and only then goes to
    retrieve the item.

    >> You should check that both lists have the same length too.
    >> And you should check that elements are integers, or convertible to
    >> integers (in case of error, PyInt_AsLong returns -1 and PyErr_Occurred()
    >> is true)
    >> To fill the resulting tuple, use PyTuple_SET_ITEM instead. BTW, why
    >> return
    >> a tuple and not a list?

    >
    > No particular reason, other than the fact that I won't need to modify
    > these lists/tuples from python so whenever something will not change,
    > I use a tuple because it's immutable. Or this is not a very good
    > practice? There is no difference between lists and tuples in terms of
    > speed I suppose (getitem, setitem, etc).


    Usually lists represent an ordered collection of similar items, where
    position is not relevant (the third item is treated more or less the same
    as the tenth); by example, a list of attendants to certain event. It makes
    sense to process the whole list doing the same thing to each element, and
    it makes sense to ask "is this item in the list?."

    Tuples represent an ordered collection of dissimilar items, where position
    is relevant (because it identifies each item); by example, a row from a
    database table (name, address, age, phone number), or a point (x,y,z) in
    space. It isn't common to process the whole tuple doing the same thing for
    each element, and usually it doesn't make sense to look for certain item
    in the tuple. But it does make sense to refer to individual items like
    t[2], or t.age (namedtuple, 2.6 and up)

    From this point of view, lists and tuples are conceptually different -
    tuples aren't "frozen" lists. But this is just common usage, or maybe
    historical intent; of course you are free to use whatever structure you
    feel adequate.

    Once the list/tuple is created and filled, there is no speed difference
    accessing the individual items. Creating an empty list that grows one
    element at a time is slow for large lists (the memory block has to be
    re-allocated and copied over evry time it gets full) but this doesn't
    happen if you provide the final size when creating the list.

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 28, 2008
    #10
  11. >> This is the function I have, the corresponding python function will
    >> take two equal length lists of integers and the C function will
    >> compute their sum and return the result as a python tuple.
    >>
    >>
    >> static PyObject *func( PyObject * self, PyObject * args )
    >> {
    >> int j, N;
    >> int * src1, * src2;
    >> PyObject *list1, *list2;
    >>
    >> list1 = PyTuple_GetItem( args, 0 );
    >> N = PyList_Size( list1 );
    >> src1 = ( int * ) malloc( N * sizeof( int ) );
    >> for( j = 0; j < N; j++ )
    >> {
    >> src1[j] = (int)PyInt_AsLong( PyList_GetItem( list1, j ) );
    >> }
    >>
    >> list2 = PyTuple_GetItem( args, 1 );
    >> N = PyList_Size( list2 );
    >> src2 = ( int * ) malloc( N * sizeof( int ) );
    >> for( j = 0; j < N; j++ )
    >> {
    >> src2[j] = (int)PyInt_AsLong( PyList_GetItem( list2, j ) );
    >> }
    >>
    >> PyObject * tuple;
    >> tuple = PyTuple_New( N );
    >> for( j = 0; j < N; j++ )
    >> {
    >> PyTuple_SetItem( tuple, j, PyInt_FromLong( (long)( src1[j] +
    >> src2[j] ) ) );
    >> }
    >>
    >> free( src1 );
    >> free( src2 );
    >>
    >> return tuple;
    >> }

    >
    > As others already said, using a Numpy array or an array.array object would
    > be more efficient (and even easier - the C code gets a pointer to an array
    > of integers, as usual).


    I looked for this in the C API docs but couldn't find anything on how
    to make an array.array python object appear as a pointer to integers
    (or floats, etc) in C code. On

    http://docs.python.org/c-api/concrete.html#sequence-objects

    There is only list and tuple or maybe you mean byte array? That has
    only been introduced in python 2.6 and I'm working on 2.5.

    Daniel

    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 28, 2008
    #11
  12. >>> You MUST check EVERY function call for errors!
    >>
    >> Yes, I know :)
    >>

    >
    > Believe me, if you don't, there is a risk of crashing the program. And
    > they're a lot harder to find and fix.


    Sure, what I meant by the smiley is just that it was a quick and dirty
    example, not real code. In a real code I do check these sorts of
    things.

    >>> And check the argument's type (how do you know it is a list?). Once you
    >>> are sure the first parameter *is* a list, you may use the "macro"
    >>> version
    >>> of several functions, like PyList_GET_SIZE and PyList_GET_ITEM.

    >>
    >> The macro versions are preferable because they are faster?

    >
    > Yes, because they don't do any additional checking. PyList_GET_ITEM just
    > retrieves the item; PyList_GetItem checks whether it is called on a list
    > object, then checks if index is out of bounds, and only then goes to
    > retrieve the item.


    Thanks, I was wondering what the difference between these versions
    are, now it's clear.

    >>> You should check that both lists have the same length too.
    >>> And you should check that elements are integers, or convertible to
    >>> integers (in case of error, PyInt_AsLong returns -1 and PyErr_Occurred()
    >>> is true)
    >>> To fill the resulting tuple, use PyTuple_SET_ITEM instead. BTW, why
    >>> return
    >>> a tuple and not a list?

    >>
    >> No particular reason, other than the fact that I won't need to modify
    >> these lists/tuples from python so whenever something will not change,
    >> I use a tuple because it's immutable. Or this is not a very good
    >> practice? There is no difference between lists and tuples in terms of
    >> speed I suppose (getitem, setitem, etc).

    >
    > Usually lists represent an ordered collection of similar items, where
    > position is not relevant (the third item is treated more or less the same
    > as the tenth); by example, a list of attendants to certain event. It makes
    > sense to process the whole list doing the same thing to each element, and
    > it makes sense to ask "is this item in the list?."
    >
    > Tuples represent an ordered collection of dissimilar items, where position
    > is relevant (because it identifies each item); by example, a row from a
    > database table (name, address, age, phone number), or a point (x,y,z) in
    > space. It isn't common to process the whole tuple doing the same thing for
    > each element, and usually it doesn't make sense to look for certain item
    > in the tuple. But it does make sense to refer to individual items like
    > t[2], or t.age (namedtuple, 2.6 and up)
    >
    > From this point of view, lists and tuples are conceptually different -
    > tuples aren't "frozen" lists. But this is just common usage, or maybe
    > historical intent; of course you are free to use whatever structure you
    > feel adequate.
    >
    > Once the list/tuple is created and filled, there is no speed difference
    > accessing the individual items. Creating an empty list that grows one
    > element at a time is slow for large lists (the memory block has to be
    > re-allocated and copied over evry time it gets full) but this doesn't
    > happen if you provide the final size when creating the list.


    All right, this is clear then too, I'll probably use a list.

    Cheers,
    Daniel


    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 28, 2008
    #12
  13. En Sun, 28 Dec 2008 01:47:08 -0200, Daniel Fetchinson
    <> escribió:

    >> As others already said, using a Numpy array or an array.array object
    >> would
    >> be more efficient (and even easier - the C code gets a pointer to an
    >> array
    >> of integers, as usual).

    >
    > I looked for this in the C API docs but couldn't find anything on how
    > to make an array.array python object appear as a pointer to integers
    > (or floats, etc) in C code. On
    >
    > http://docs.python.org/c-api/concrete.html#sequence-objects
    >
    > There is only list and tuple or maybe you mean byte array? That has
    > only been introduced in python 2.6 and I'm working on 2.5.


    array is a library module, and isn't really part of the API. You're
    looking for the buffer protocol:
    PyObject_AsReadBuffer/PyObject_AsWriteBuffer; see
    http://docs.python.org/c-api/objbuffer.html

    Given an array.array('l') (containing C long integers):

    int do_something(PyObject* obj)
    {
    long *vec;
    Py_ssize_t nbytes, nitems, i;

    if (PyObject_AsReadBuffer(obj, (const void **)&vec, &nbytes) != 0)
    return NULL;
    nitems = nbytes/sizeof(long);
    for (i=0; i<nitems; i++) {
    /* do something with vec */
    }
    return ret;
    }

    From Python you can get "vec" and "nitems" using the buffer_info() method
    of array objects.

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 28, 2008
    #13
  14. >>> As others already said, using a Numpy array or an array.array object
    >>> would
    >>> be more efficient (and even easier - the C code gets a pointer to an
    >>> array
    >>> of integers, as usual).

    >>
    >> I looked for this in the C API docs but couldn't find anything on how
    >> to make an array.array python object appear as a pointer to integers
    >> (or floats, etc) in C code. On
    >>
    >> http://docs.python.org/c-api/concrete.html#sequence-objects
    >>
    >> There is only list and tuple or maybe you mean byte array? That has
    >> only been introduced in python 2.6 and I'm working on 2.5.

    >
    > array is a library module, and isn't really part of the API. You're
    > looking for the buffer protocol:
    > PyObject_AsReadBuffer/PyObject_AsWriteBuffer; see
    > http://docs.python.org/c-api/objbuffer.html
    >
    > Given an array.array('l') (containing C long integers):
    >
    > int do_something(PyObject* obj)
    > {
    > long *vec;
    > Py_ssize_t nbytes, nitems, i;
    >
    > if (PyObject_AsReadBuffer(obj, (const void **)&vec, &nbytes) != 0)
    > return NULL;
    > nitems = nbytes/sizeof(long);
    > for (i=0; i<nitems; i++) {
    > /* do something with vec */
    > }
    > return ret;
    > }
    >
    > From Python you can get "vec" and "nitems" using the buffer_info() method
    > of array objects.


    Thanks very much, this was very helpful!

    Cheers,
    Daniel

    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
    Daniel Fetchinson, Dec 28, 2008
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    547
    Mark P
    Apr 3, 2005
  2. Skybuck Flying

    ints ints ints and ints

    Skybuck Flying, Jul 8, 2004, in forum: C Programming
    Replies:
    24
    Views:
    809
    Jack Klein
    Jul 10, 2004
  3. Replies:
    6
    Views:
    264
    Dan Bishop
    Apr 6, 2008
  4. Laszlo Nagy
    Replies:
    2
    Views:
    253
    Matthew Woodcraft
    Aug 9, 2008
  5. Laszlo Nagy
    Replies:
    0
    Views:
    276
    Laszlo Nagy
    Aug 8, 2008
Loading...

Share This Page