Hacking NArray

[Brian Schröder]

Hello group,

I will be busy doing volume data processing the next six months, and I have decided to use narray for this purpose.

I'm will shurely have need to implement some functions in c, because otherwise it will take just too long, but I'd really love to make the basic structures in ruby.

So I will have to add custom functions to narray (e.g. right now I need an inplace abs! operator). This will be a lot of inplace operations, because when working with volume data even some gigs of memory are eaten up fast. And I think I will have to extend the fftw code, to for example include
- inplace dfts,
- saving of plans

So my question is, if anybody has advice on hacking narray. The code does not seem too complicated, but it is completely undocumented and I don't want to duplicate efforts. The other question is, if someone else has already achieved some of my goals, such that I could reuse components.

Thanks a lot,

Brian

 

[Brian Schröder]

So my question is, if anybody has advice on hacking narray. The code

does not seem too complicated, but it is completely undocumented and I
don't want to duplicate efforts. The other question is, if someone
else has already achieved some of my goals, such that I could reuse
components.

Well, in fact the code seems to be generated by ruby-scripts that are
totally undocumented, so two layers of undocumented code makes it harder
than I thought. I would really appreciate an example on how to extend
narray. That would be:

- How to create a function that works on one or more arrays (in
place|with copy), one element at a time
- How to create a function that works on one or more arrays with random
access to the entries.

 

[Dave Baldwin]

Hi Brian,

I cannot offer any advice on narray, but I have been investigating some
image processing algorithms in Ruby. I looked at narray but decided it
wasn't worth the effort to understand and holding a 2D image as an
array of arrays didn't seem that natural a fit. Also many of the
member functions weren't that relevant to what I was going to be doing.

My first cut was to implement a simple image class where the image was
held in an array and I provided an [x, y] operator to access the data
and build on this to add image fill, convolution, stats, thinning,
etc.. Even though I was operating on small images (fingerprints)
typically 256x256 the performance was not great so I replaced a few key
routines such as convolution with c extensions. This improved things
somewhat, but as the processing got even more sophisticated performance
became an issue again. Most of the image data ended up being held as
ruby floats (so took 3 words) - the memory footprint was large as a
consequence and a lot of time in my c routines was spent converting
between ruby floats and c floats.

Finally I rewrote the ruby image class mainly in c with the native
pixel data being held in a c array as c floats. Performance and memory
footprint are now adequate for my needs.

I would advise you to implement your volume class storage and common
member functions in c and extend the class with ruby for the high level
algorithm control. Processing time grows rapidly when you go into 2D
image and even more so for 3D volumes. Try a few experiments in just
looping through the volume setting a voxel at a time and you will be
convinced!

Dave.

 

[Yoshiki Tsunesada]

than I thought. I would really appreciate an example on how to extend

narray. That would be:
- How to create a function that works on one or more arrays (in
place|with copy), one element at a time
- How to create a function that works on one or more arrays with random
access to the entries.

Could you show Ruby code you expect to work with it ?

Yoshiki

 

[Brian Schröder]

Could you show Ruby code you expect to work with it ?

Yoshiki

Well:

a.abs!

for example.

And something like:

define_inplace_operator 'abs!', {[[:float, :float], [:double, :double], [:int, :int]] => '*p = *p > 0 ? p : -p;'}

in the mkop.rb skript to create those functions automatically in the c source.

That would be a nice dream...

But at the moment I'll try to go with the other advice and implement my own purely double based volume class. (And with code generation maybe also float, int etc, but I have to do some exploration first, as I've never written a c extension.

Regards,

Brian

 

[Yoshiki Tsunesada]

Brian,

Adding methods to NArray object is rather simple:
If an object is of double float,

static VALUE my_narray_abs_in_place(VALUE self)
{
struct NARRAY *na;
double *ptr;
int i, n;
GetNArray(self, na);
n = na->total;
ptr = (double*) na->ptr;
for (i = 0; i < n; i++) ptr = fabs(ptr);
return self;
}

and,

rb_define_method(cNArray, "abs!", my_narray_abs_in_place, 0);

Could you show Ruby code you expect to work with it ?

Yoshiki

Well:

a.abs!

for example.

And something like:

define_inplace_operator 'abs!', {[[:float, :float], [:double,
:double], [:int, :int]] => '*p = *p > 0 ? p : -p;'}

in the mkop.rb skript to create those functions automatically in the
c source.

That would be a nice dream...

But at the moment I'll try to go with the other advice and implement
my own purely double based volume class. (And with code generation
maybe also float, int etc, but I have to do some exploration first,
as I've never written a c extension.

Regards,

Brian

 

[Brian Schröder]

Brian,

Adding methods to NArray object is rather simple:
If an object is of double float,

static VALUE my_narray_abs_in_place(VALUE self)
{
struct NARRAY *na;
double *ptr;
int i, n;
GetNArray(self, na);
n = na->total;
ptr = (double*) na->ptr;
for (i = 0; i < n; i++) ptr = fabs(ptr);
return self;
}

and,

rb_define_method(cNArray, "abs!", my_narray_abs_in_place, 0);

Thank you for this information. It is very helpfull. Maybe it should be included onto the website?

best regards,

Brian Schröder

than I thought. I would really appreciate an example on how to extend
narray. That would be:
- How to create a function that works on one or more arrays (in
place|with copy), one element at a time
- How to create a function that works on one or more arrays with random
access to the entries.
Could you show Ruby code you expect to work with it ?

Yoshiki

Well:

a.abs!

for example.

And something like:

define_inplace_operator 'abs!', {[[:float, :float], [:double,
:double], [:int, :int]] => '*p = *p > 0 ? p : -p;'}

in the mkop.rb skript to create those functions automatically in the
c source.

That would be a nice dream...

But at the moment I'll try to go with the other advice and implement
my own purely double based volume class. (And with code generation
maybe also float, int etc, but I have to do some exploration first,
as I've never written a c extension.

Regards,

Brian

 

[Bertram Scharpf]

Hi Brian,

your header says

From: Brian =?ISO-8859-15?Q?Schr=C3=B6der?= <[email protected]>

I suppose you meant something like

From: Brian =?UTF-8?Q?Schr=C3=B6der?= <[email protected]>

Bertram