void * C array to a Numpy array using Swig

[Krish]

Hello People

I hope I am On Topic. Anyways, here is my problem. Any insights would
be really appreciated.

I have wrapped a C IO module using SWIG -> Python Module. Suppose the
name of the module is "imageio" and the reader function from the file
is image_read() which returns an object ( "filled" C structure) with a
lot of members.

Let

a = imageio.image_read("testfile.image")

Now a is the object which contains pointer to data of the image. ( void
* pdata). ( a also contains the datatype which I need to typecast to
get the data pointed by pdata). My question is how to access the data
pointed by pdata in *Python*.

Ideally I want the data to be converted into a Scipy(Numpy) Array
object so that I can perform further operations in Python. Any pointers
will be appreciated.

I think this has to do with PyArrayObjects, TypeMaps etc. but I
couldn't consolidate the whole information. If anyone has done this
kinda stuff before or if anyone can point me to a correct/better way,
it would be really great.

Regards
Krish Subramaniam

 

[Jon]

Krish,

In case you find a good solution, I am also looking for one!

For now I essentially use helper functions on the c side which wrap in
SWIG to return the data as a string in python. That string can then be
converted to a numpy array using the fromstring function. This is
inefficient as it does an unnecessary copy but avoids dependence on
numeric versus numarray etc. It uses the cstring thing in SWIG (see the
manual). The library I am wrapping does not have an image struct, but
returns the data into memory that the user has to malloc.

In the swig file I have something like this, which I've simplified to
try to get to the point. It assumes you have two c functions which take
a pointer to your struct as argument, the first returns the size of the
data (what to malloc), the second copies the data into your memory
where a pointer to the memory location was second arg.

Doubtless I've introduced typos below, but hopefully you get the idea?

Good luck,

Jon
---
typedef struct
{
stuff /* I don't know or care what is in here */
} imagefilestruct;

%extend imagefilestruct {

[... snip constructor destructor other functions etc]

%cstring_output_allocate_size( char ** s, int *slen, free(*$1))
get_data ;

void get_data(char **s, int *slen){
void * array;
size_t size;
size = libraryfunction_get_size(self);
array=malloc(size));
libraryfunc_get_data(self, array);
*slen = size;
*s = (char *) array;
}
}

 

[Krish]

Thanks Jon Much

I will implement your method for the time being. I am sure there is
some method which doesn't copy the data but uses the existing data.

I mean essentially we should build a PyArrayObject with the data intact
and other parameters filled. If someone sheds some light, would be
awesome. I am gonna try it this weekend.

Appreciated
Krish

 

[Travis E. Oliphant]

Hello People

I hope I am On Topic. Anyways, here is my problem. Any insights would
be really appreciated.

Posting to the (e-mail address removed) list would help
generate more responses, I think.

I have wrapped a C IO module using SWIG -> Python Module. Suppose the
name of the module is "imageio" and the reader function from the file
is image_read() which returns an object ( "filled" C structure) with a
lot of members.

Let

a = imageio.image_read("testfile.image")

Now a is the object which contains pointer to data of the image. ( void
* pdata). ( a also contains the datatype which I need to typecast to
get the data pointed by pdata). My question is how to access the data
pointed by pdata in *Python*.

Yes, you are right that you need to use typemaps. It's been awhile
since I did this kind of thing, but here are some pointers.

1) Look at Michael Sanner's typemaps at

http://www.scripps.edu/mb/olson/people/sanner/html/Python/NumericTypemaps/

Except for the CHAR version, these should work for NumPy by replacing
Numeric/arrayobject.h with numpy/arrayobject.h

2) In full scipy there are typemaps for numpy arrays in
cluster/src/swig_num.i

Look here...

http://projects.scipy.org/scipy/scipy/file/trunk/Lib/cluster/src/swig_num.i

This should help you get started with some examples. Typemaps can be a
little confusing at first, but they do make your interface a bit nicer.

-Travis

 

[Philip Austin]

Krish wrote:

Yes, you are right that you need to use typemaps. It's been awhile
since I did this kind of thing, but here are some pointers.

Also, there's http://geosci.uchicago.edu/csc/numptr

 

[Travis E. Oliphant]

Also, there's http://geosci.uchicago.edu/csc/numptr

This is interesting.

I've noticed his getpointerX functions seem to be implemented already by

PyArray_AsCArray in the new NumPy.

-Travis