Questions about Inline::C

[January Weiner]

Dear all,

I had to revert to C again. Since the thought of parsing all my files in C
was dreadful, I decided to use Inline::C to only do the job that was really
computationally intensive. Currently, my program seems to run quite well,
I am very happy with Inline::C, and a little coding in C made me bless
Larry again and again.

There are, however, few questions that remain open.

1. In the Inline::C cookbook, it says

I would urge you to stay away from "malloc"ing your own buffer. Just
use Perl's built in memory management. In other words, just create a
new Perl string scalar. The function "newSVpv" does just that. And
"newSVpvf" includes "sprintf" functionality.

Why? I mean, is there is any danger with malloc() from within Inline::C
that is not present in regular C programs, or is it because using malloc
is just generally tricky? I much prefer to do the allocation myself,
as I know exactly how large my matrix is and as I want to allocate the
whole matrix ( say, double 2000 x 2000 ) in one go myself. The reason is
that I fear that using perl guts for my calculations (accessing the
matrix, calculating values in it etc.) will be not much slower than a
simple C implementation. Maybe I am wrong, I haven't tested it.

Furthermore, I feel more comfortable using things that I know well, even
thought they might turn dangerous.

2. When using Inline_Stack to return a list to Perl code, I found that the
following throws a segfault:

Inline_Stack_Reset ;
Inline_Stack_Push( sv2_mortal( newSVpv( 'score' ) ) ) ;
Inline_Stack_Done ;

...while the following does not:

Inline_Stack_Reset ;
Inline_Stack_Push( sv2_mortal( newSVpv( "score" ) ) ) ;
Inline_Stack_Done ;

Should I be worried, or is it normal behaviour? (I tried to look up the
definition of Inline_Stack_Push, but I have trouble finding it).

Cheers,

January

--

 

[Mirco Wahab]

Thus spoke January Weiner (on 2006-09-28 09:58):

I had to revert to C again. Since the thought of parsing all my files in C
was dreadful, I decided to use Inline::C to only do the job that was really
computationally intensive. Currently, my program seems to run quite well,
I am very happy with Inline::C, and a little coding in C made me bless
Larry again and again.

One question in advance: Why use Inline::C if
your C-Program is what counts. Why don't you
simply link the 'perl' to your program and
call it with the stuff you need. This is,
in large projects (imho) *much* simpler than
vice versa.

Use a _static_ PerlInterpreter*, so your
perl will keep state during whole program run.

I much prefer to do the allocation myself,
as I know exactly how large my matrix is and as I want to allocate the
whole matrix ( say, double 2000 x 2000 ) in one go myself. The reason is
that I fear that using perl guts for my calculations (accessing the
matrix, calculating values in it etc.) will be not much slower than a
simple C implementation. Maybe I am wrong, I haven't tested it.

Perl will be much slower if used trivially so (I can tell
you this). Just use your matrix in C as usual and make
some Into-Perl-Calls if necessary, eg. by pack(...)-ing
your variables into Perl-scalars and unpacking
them again in the perl.

Regards

Mirco

 

[Sisyphus]

..
..

I mean, is there is any danger with malloc() from within Inline::C
that is not present in regular C programs

Yes, I believe so - not sure of the details but I think it has something to
do with the possibility that perl has been compiled with its own malloc()
function.
Instead of malloc(), use New(). (See 'perldoc perlapi'.)

There are some *other* C functions that it's best to avoid, too - in favour
of their perl API equivalents. See 'perldoc perlclib'

2. When using Inline_Stack to return a list to Perl code, I found that the
following throws a segfault:

Inline_Stack_Reset ;
Inline_Stack_Push( sv2_mortal( newSVpv( 'score' ) ) ) ;
Inline_Stack_Done ;

...while the following does not:

Inline_Stack_Reset ;
Inline_Stack_Push( sv2_mortal( newSVpv( "score" ) ) ) ;
Inline_Stack_Done ;

Should I be worried, or is it normal behaviour? (I tried to look up the
definition of Inline_Stack_Push, but I have trouble finding it).

In perl you can quote string literals inside either single or double
quotes - but in C you have to use double quotes. Using single quotes will
inevitably result in some sort of error. (I get a segfault, too.)

newSVpv() has to take 2 arguments - see (again) 'perldoc perlapi'.

Cheers,
Rob

 

[January Weiner]

Yes, I believe so - not sure of the details but I think it has something to
do with the possibility that perl has been compiled with its own malloc()
function.
Instead of malloc(), use New(). (See 'perldoc perlapi'.)

O, thanks, that's cool, many thanks.

In perl you can quote string literals inside either single or double
quotes - but in C you have to use double quotes. Using single quotes will

....oh my. That is what years of writing Perl does to you Sure, I have
completly forgotten. Single quotes are for characters. Of course. Silly me

newSVpv() has to take 2 arguments - see (again) 'perldoc perlapi'.

Typo when writing the Usenet post. I'm using newSVpvf.

Cheers,
January

--

 

[January Weiner]

One question in advance: Why use Inline::C if
your C-Program is what counts. Why don't you

Not really. The C program does not do anything new (Needleman-Wunsch +
Smith-Waterman algorithms with one or two important modifications that you
will not find elsewhere). The Perl part is, apart from the parsing and
output formatting, Real Science (TM) It does not take as much time to
calculate, but its tricky.

simply link the 'perl' to your program and
call it with the stuff you need. This is,

You mean, call Perl from C and not C from Perl? Hmm, that could be a
solution as well. However, I have much more stuff (and also much more
stuff that matters) in Perl than I do in C -- for my project, I have now
only two small functions in C that do a simple job quickly. For everything
else, I have my Perl.

I am wondering, however, how complicated it is to link an existing C
library to Perl. I am sure there are dozens of examples for that, if you
know of a good one, I would love to see how it works...

in large projects (imho) *much* simpler than
vice versa.

Because I am lazy and because it is not a large project.

Perl will be much slower if used trivially so (I can tell
you this). Just use your matrix in C as usual and make
some Into-Perl-Calls if necessary, eg. by pack(...)-ing
your variables into Perl-scalars and unpacking
them again in the perl.

Thanks,

January

--

 

[Mirco Wahab]

Thus spoke January Weiner (on 2006-09-28 12:25):

I am wondering, however, how complicated it is to link an existing C
library to Perl. I am sure there are dozens of examples for that, if you
know of a good one, I would love to see how it works...

OK, I see your point now. You could give
a link option afaik by :

use Inline C => Config => LIBS => '-L/your/path -lyourlib';

.... according to the "Inline" docs

Did you think about PDL?

==>
http://pdl.sourceforge.net/PDLdocs/API.html#creating_a_piddle_in_c

Regards

Mirco

 

[Sisyphus]

..
..

(I tried to look up the
definition of Inline_Stack_Push, but I have trouble finding it).

I meant to address this in my earlier post (but forgot).
Inline_Stack_Vars and friends are defined in INLINE.h which is a header file
auto-generated by Inline::C. It gets written in the build directory - and
then cleaned up once the compilation has successfully completed (unless you
run with the Config option 'CLEAN_AFTER_BUILD => 0').

Here's what INLINE.h contains (copied from C.pm):

#define Inline_Stack_Vars dXSARGS
#define Inline_Stack_Items items
#define Inline_Stack_Item(x) ST(x)
#define Inline_Stack_Reset sp = mark
#define Inline_Stack_Push(x) XPUSHs(x)
#define Inline_Stack_Done PUTBACK
#define Inline_Stack_Return(x) XSRETURN(x)
#define Inline_Stack_Void XSRETURN(0)

#define INLINE_STACK_VARS Inline_Stack_Vars
#define INLINE_STACK_ITEMS Inline_Stack_Items
#define INLINE_STACK_ITEM(x) Inline_Stack_Item(x)
#define INLINE_STACK_RESET Inline_Stack_Reset
#define INLINE_STACK_PUSH(x) Inline_Stack_Push(x)
#define INLINE_STACK_DONE Inline_Stack_Done
#define INLINE_STACK_RETURN(x) Inline_Stack_Return(x)
#define INLINE_STACK_VOID Inline_Stack_Void

#define inline_stack_vars Inline_Stack_Vars
#define inline_stack_items Inline_Stack_Items
#define inline_stack_item(x) Inline_Stack_Item(x)
#define inline_stack_reset Inline_Stack_Reset
#define inline_stack_push(x) Inline_Stack_Push(x)
#define inline_stack_done Inline_Stack_Done
#define inline_stack_return(x) Inline_Stack_Return(x)
#define inline_stack_void Inline_Stack_Void

Cheers,
Rob

 

[January Weiner]

Inline_Stack_Vars and friends are defined in INLINE.h which is a header file
auto-generated by Inline::C. It gets written in the build directory - and
then cleaned up once the compilation has successfully completed (unless you
run with the Config option 'CLEAN_AFTER_BUILD => 0').

Oh, that I do know, I have found that It's just that it didn't tell me
much, because it just redefines the macros. (define inline_stack_push as
Inline_Stack_Push, and earlier Inline_Stack_Push as XPUSHs... but I still
do not know what XPUSHs is).

Anyway, this doesn't matter, as the (stupid! stupid!) error was completly
on my side.

j.

--

 

[January Weiner]

OK, I see your point now. You could give
a link option afaik by :

use Inline C => Config => LIBS => '-L/your/path -lyourlib';

Ah, that is nice (I should have RTFM...).

Did you think about PDL?

==>
http://pdl.sourceforge.net/PDLdocs/API.html#creating_a_piddle_in_c

And this is really nice. Many thanks! (and my program works beautifully)

January

--

 

[Ben Morrow]

Oh, that I do know, I have found that It's just that it didn't tell me
much, because it just redefines the macros. (define inline_stack_push as
Inline_Stack_Push, and earlier Inline_Stack_Push as XPUSHs... but I still
do not know what XPUSHs is).

See perlapi. IMHO those Inline_Foo macros are pretty stupid: just
renaming the XS macros for the hell of it.

Ben

 

[Sisyphus]

..
..

but I still
do not know what XPUSHs is).

perldoc perlapi

Cheers,
Rob

 

[xhoster]

Dear all,

I had to revert to C again. Since the thought of parsing all my files in
C was dreadful, I decided to use Inline::C to only do the job that was
really computationally intensive. Currently, my program seems to run
quite well, I am very happy with Inline::C, and a little coding in C made
me bless Larry again and again.

There are, however, few questions that remain open.

1. In the Inline::C cookbook, it says


Why? I mean, is there is any danger with malloc() from within
Inline::C that is not present in regular C programs, or is it because
using malloc is just generally tricky?

I think that it is some of both. Someone else pointed out New() vs. malloc
(which I've never been burned by myself) so I would just say that if you
malloc memory, you have to figure out when to free it, while Perl does the
refcounting for you (but I can never get it to work the first time on
complicated Inline code. Either it isn't intuitive or I haven't used it
enough.)

I much prefer to do the
allocation myself, as I know exactly how large my matrix is and as I
want to allocate the whole matrix ( say, double 2000 x 2000 ) in one
go myself.

Will that fit as an automatic (I think that is the term) variable?
double x[2000][2000];

If this works, and it has the appropriate scope for what you need (i.e.
only needed inside one sub, no passing data back and forth from C to Perl)
this is the way I would do it.

The reason is that I fear that using perl guts for my
calculations (accessing the matrix, calculating values in it etc.)
will be not much slower than a simple C implementation. Maybe I am
wrong, I haven't tested it.

You don't need to use perl guts for each manipulation. You can get the
char* to the start of the Perl sv string memory, and then cast that into
whatever you want. Store that casted pointer, and use it just like there
were no Perl involved from then on.

my $x= pack "d*", @list_of_numbers;
foo($x,scalar @list_of_numbers);

__C__

void foo (unsigned char* s, int num_doubles) {
double * x = (double*) s;
//now use x as any other double *, being sure not to go out of bounds.
}


Xho

 

[January Weiner]

Will that fit as an automatic (I think that is the term) variable?
double x[2000][2000];

Unfortunately, no, as I do not know how large the matrix is going to be.

You don't need to use perl guts for each manipulation. You can get the
char* to the start of the Perl sv string memory, and then cast that into
whatever you want. Store that casted pointer, and use it just like there
were no Perl involved from then on.

my $x= pack "d*", @list_of_numbers;
foo($x,scalar @list_of_numbers);

void foo (unsigned char* s, int num_doubles) {
double * x = (double*) s;
//now use x as any other double *, being sure not to go out of bounds.
}

This is nice, thanks!
j.

--

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to

You don't need to use perl guts for each manipulation. You can get the
char* to the start of the Perl sv string memory, and then cast that into
whatever you want. Store that casted pointer, and use it just like there
were no Perl involved from then on.

my $x= pack "d*", @list_of_numbers;
foo($x,scalar @list_of_numbers);

__C__

void foo (unsigned char* s, int num_doubles) {
double * x = (double*) s;
//now use x as any other double *, being sure not to go out of bounds.
}

If this works, then by coincidence only. Memory alignment of SvPV()
and of double* is absolutely different.

Hope this helps,
Ilya

P.S. In *many* cases SvPV() will point at the start of malloc()ed
region of size

1 + sizeof(double) * scalar @list_of_numbers

Since this is not divisible by sizeof(double), some m_alloc()ers
may align it differently than requests for sizeof(double) * n.

Moreover, there is no guarantie that SvPV() points at start of
malloc()ed region.