void * C array to a Numpy array using Swig

Discussion in 'Python' started by Krish, Jan 12, 2006.

  1. Krish

    Krish Guest

    Hello People

    I hope I am On Topic. Anyways, here is my problem. Any insights would
    be really appreciated.

    I have wrapped a C IO module using SWIG -> Python Module. Suppose the
    name of the module is "imageio" and the reader function from the file
    is image_read() which returns an object ( "filled" C structure) with a
    lot of members.

    Let

    a = imageio.image_read("testfile.image")

    Now a is the object which contains pointer to data of the image. ( void
    * pdata). ( a also contains the datatype which I need to typecast to
    get the data pointed by pdata). My question is how to access the data
    pointed by pdata in *Python*.

    Ideally I want the data to be converted into a Scipy(Numpy) Array
    object so that I can perform further operations in Python. Any pointers
    will be appreciated.

    I think this has to do with PyArrayObjects, TypeMaps etc. but I
    couldn't consolidate the whole information. If anyone has done this
    kinda stuff before or if anyone can point me to a correct/better way,
    it would be really great.

    Regards
    Krish Subramaniam
    Krish, Jan 12, 2006
    #1
    1. Advertising

  2. Krish

    Jon Guest

    Krish,

    In case you find a good solution, I am also looking for one!

    For now I essentially use helper functions on the c side which wrap in
    SWIG to return the data as a string in python. That string can then be
    converted to a numpy array using the fromstring function. This is
    inefficient as it does an unnecessary copy but avoids dependence on
    numeric versus numarray etc. It uses the cstring thing in SWIG (see the
    manual). The library I am wrapping does not have an image struct, but
    returns the data into memory that the user has to malloc.

    In the swig file I have something like this, which I've simplified to
    try to get to the point. It assumes you have two c functions which take
    a pointer to your struct as argument, the first returns the size of the
    data (what to malloc), the second copies the data into your memory
    where a pointer to the memory location was second arg.

    Doubtless I've introduced typos below, but hopefully you get the idea?

    Good luck,

    Jon
    ---
    typedef struct
    {
    stuff /* I don't know or care what is in here */
    } imagefilestruct;

    %extend imagefilestruct {

    [... snip constructor destructor other functions etc]

    %cstring_output_allocate_size( char ** s, int *slen, free(*$1))
    get_data ;

    void get_data(char **s, int *slen){
    void * array;
    size_t size;
    size = libraryfunction_get_size(self);
    array=malloc(size));
    libraryfunc_get_data(self, array);
    *slen = size;
    *s = (char *) array;
    }
    }
    Jon, Jan 12, 2006
    #2
    1. Advertising

  3. Krish

    Krish Guest

    Thanks Jon Much

    I will implement your method for the time being. I am sure there is
    some method which doesn't copy the data but uses the existing data.

    I mean essentially we should build a PyArrayObject with the data intact
    and other parameters filled. If someone sheds some light, would be
    awesome. I am gonna try it this weekend.

    Appreciated
    Krish
    Krish, Jan 12, 2006
    #3
  4. Krish wrote:
    > Hello People
    >
    > I hope I am On Topic. Anyways, here is my problem. Any insights would
    > be really appreciated.


    Posting to the list would help
    generate more responses, I think.

    >
    > I have wrapped a C IO module using SWIG -> Python Module. Suppose the
    > name of the module is "imageio" and the reader function from the file
    > is image_read() which returns an object ( "filled" C structure) with a
    > lot of members.
    >
    > Let
    >
    > a = imageio.image_read("testfile.image")
    >
    > Now a is the object which contains pointer to data of the image. ( void
    > * pdata). ( a also contains the datatype which I need to typecast to
    > get the data pointed by pdata). My question is how to access the data
    > pointed by pdata in *Python*.


    Yes, you are right that you need to use typemaps. It's been awhile
    since I did this kind of thing, but here are some pointers.

    1) Look at Michael Sanner's typemaps at

    http://www.scripps.edu/mb/olson/people/sanner/html/Python/NumericTypemaps/

    Except for the CHAR version, these should work for NumPy by replacing
    Numeric/arrayobject.h with numpy/arrayobject.h

    2) In full scipy there are typemaps for numpy arrays in
    cluster/src/swig_num.i

    Look here...

    http://projects.scipy.org/scipy/scipy/file/trunk/Lib/cluster/src/swig_num.i

    This should help you get started with some examples. Typemaps can be a
    little confusing at first, but they do make your interface a bit nicer.

    -Travis
    Travis E. Oliphant, Jan 12, 2006
    #4
  5. "Travis E. Oliphant" <> writes:

    > Krish wrote:


    > Yes, you are right that you need to use typemaps. It's been awhile
    > since I did this kind of thing, but here are some pointers.


    Also, there's http://geosci.uchicago.edu/csc/numptr
    Philip Austin, Jan 13, 2006
    #5
  6. Philip Austin wrote:
    > "Travis E. Oliphant" <> writes:
    >
    >
    >>Krish wrote:

    >
    >
    >>Yes, you are right that you need to use typemaps. It's been awhile
    >>since I did this kind of thing, but here are some pointers.

    >
    >
    > Also, there's http://geosci.uchicago.edu/csc/numptr
    >


    This is interesting.

    I've noticed his getpointerX functions seem to be implemented already by

    PyArray_AsCArray in the new NumPy.

    -Travis
    Travis E. Oliphant, Jan 13, 2006
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ollej Reemt
    Replies:
    7
    Views:
    527
    Jack Klein
    Apr 22, 2005
  2. Stig Brautaset

    `void **' revisited: void *pop(void **root)

    Stig Brautaset, Oct 25, 2003, in forum: C Programming
    Replies:
    15
    Views:
    788
    The Real OS/2 Guy
    Oct 28, 2003
  3. Replies:
    5
    Views:
    833
    S.Tobias
    Jul 22, 2005
  4. Replies:
    1
    Views:
    406
    Victor Bazarov
    May 23, 2007
  5. Replies:
    2
    Views:
    486
    Robert Kern
    Nov 13, 2007
Loading...

Share This Page