Questions about Inline::C

Discussion in 'Perl Misc' started by January Weiner, Sep 28, 2006.

  1. Dear all,

    I had to revert to C again. Since the thought of parsing all my files in C
    was dreadful, I decided to use Inline::C to only do the job that was really
    computationally intensive. Currently, my program seems to run quite well,
    I am very happy with Inline::C, and a little coding in C made me bless
    Larry again and again.

    There are, however, few questions that remain open.

    1. In the Inline::C cookbook, it says

    > I would urge you to stay away from "malloc"ing your own buffer. Just
    > use Perl's built in memory management. In other words, just create a
    > new Perl string scalar. The function "newSVpv" does just that. And
    > "newSVpvf" includes "sprintf" functionality.


    Why? I mean, is there is any danger with malloc() from within Inline::C
    that is not present in regular C programs, or is it because using malloc
    is just generally tricky? I much prefer to do the allocation myself,
    as I know exactly how large my matrix is and as I want to allocate the
    whole matrix ( say, double 2000 x 2000 ) in one go myself. The reason is
    that I fear that using perl guts for my calculations (accessing the
    matrix, calculating values in it etc.) will be not much slower than a
    simple C implementation. Maybe I am wrong, I haven't tested it.

    Furthermore, I feel more comfortable using things that I know well, even
    thought they might turn dangerous.

    2. When using Inline_Stack to return a list to Perl code, I found that the
    following throws a segfault:

    Inline_Stack_Reset ;
    Inline_Stack_Push( sv2_mortal( newSVpv( 'score' ) ) ) ;
    Inline_Stack_Done ;

    ...while the following does not:

    Inline_Stack_Reset ;
    Inline_Stack_Push( sv2_mortal( newSVpv( "score" ) ) ) ;
    Inline_Stack_Done ;

    Should I be worried, or is it normal behaviour? (I tried to look up the
    definition of Inline_Stack_Push, but I have trouble finding it).

    Cheers,

    January

    --
    January Weiner, Sep 28, 2006
    #1
    1. Advertising

  2. January Weiner

    Mirco Wahab Guest

    Thus spoke January Weiner (on 2006-09-28 09:58):

    > I had to revert to C again. Since the thought of parsing all my files in C
    > was dreadful, I decided to use Inline::C to only do the job that was really
    > computationally intensive. Currently, my program seems to run quite well,
    > I am very happy with Inline::C, and a little coding in C made me bless
    > Larry again and again.


    One question in advance: Why use Inline::C if
    your C-Program is what counts. Why don't you
    simply link the 'perl' to your program and
    call it with the stuff you need. This is,
    in large projects (imho) *much* simpler than
    vice versa.

    Use a _static_ PerlInterpreter*, so your
    perl will keep state during whole program run.

    > I much prefer to do the allocation myself,
    > as I know exactly how large my matrix is and as I want to allocate the
    > whole matrix ( say, double 2000 x 2000 ) in one go myself. The reason is
    > that I fear that using perl guts for my calculations (accessing the
    > matrix, calculating values in it etc.) will be not much slower than a
    > simple C implementation. Maybe I am wrong, I haven't tested it.


    Perl will be much slower if used trivially so (I can tell
    you this). Just use your matrix in C as usual and make
    some Into-Perl-Calls if necessary, eg. by pack(...)-ing
    your variables into Perl-scalars and unpacking
    them again in the perl.

    Regards

    Mirco
    Mirco Wahab, Sep 28, 2006
    #2
    1. Advertising

  3. January Weiner

    Sisyphus Guest

    "January Weiner" <> wrote in message
    ..
    ..
    > I mean, is there is any danger with malloc() from within Inline::C
    > that is not present in regular C programs


    Yes, I believe so - not sure of the details but I think it has something to
    do with the possibility that perl has been compiled with its own malloc()
    function.
    Instead of malloc(), use New(). (See 'perldoc perlapi'.)

    There are some *other* C functions that it's best to avoid, too - in favour
    of their perl API equivalents. See 'perldoc perlclib'

    > 2. When using Inline_Stack to return a list to Perl code, I found that the
    > following throws a segfault:
    >
    > Inline_Stack_Reset ;
    > Inline_Stack_Push( sv2_mortal( newSVpv( 'score' ) ) ) ;
    > Inline_Stack_Done ;
    >
    > ...while the following does not:
    >
    > Inline_Stack_Reset ;
    > Inline_Stack_Push( sv2_mortal( newSVpv( "score" ) ) ) ;
    > Inline_Stack_Done ;
    >
    > Should I be worried, or is it normal behaviour? (I tried to look up the
    > definition of Inline_Stack_Push, but I have trouble finding it).
    >


    In perl you can quote string literals inside either single or double
    quotes - but in C you have to use double quotes. Using single quotes will
    inevitably result in some sort of error. (I get a segfault, too.)

    newSVpv() has to take 2 arguments - see (again) 'perldoc perlapi'.

    Cheers,
    Rob
    Sisyphus, Sep 28, 2006
    #3
  4. Sisyphus <> wrote:
    > Yes, I believe so - not sure of the details but I think it has something to
    > do with the possibility that perl has been compiled with its own malloc()
    > function.
    > Instead of malloc(), use New(). (See 'perldoc perlapi'.)


    O, thanks, that's cool, many thanks.

    > In perl you can quote string literals inside either single or double
    > quotes - but in C you have to use double quotes. Using single quotes will


    ....oh my. That is what years of writing Perl does to you :) Sure, I have
    completly forgotten. Single quotes are for characters. Of course. Silly me
    :)

    > newSVpv() has to take 2 arguments - see (again) 'perldoc perlapi'.


    Typo when writing the Usenet post. I'm using newSVpvf.

    Cheers,
    January

    --
    January Weiner, Sep 28, 2006
    #4
  5. Mirco Wahab <> wrote:
    > One question in advance: Why use Inline::C if
    > your C-Program is what counts. Why don't you


    Not really. The C program does not do anything new (Needleman-Wunsch +
    Smith-Waterman algorithms with one or two important modifications that you
    will not find elsewhere). The Perl part is, apart from the parsing and
    output formatting, Real Science (TM) :) It does not take as much time to
    calculate, but its tricky.

    > simply link the 'perl' to your program and
    > call it with the stuff you need. This is,


    You mean, call Perl from C and not C from Perl? Hmm, that could be a
    solution as well. However, I have much more stuff (and also much more
    stuff that matters) in Perl than I do in C -- for my project, I have now
    only two small functions in C that do a simple job quickly. For everything
    else, I have my Perl.

    I am wondering, however, how complicated it is to link an existing C
    library to Perl. I am sure there are dozens of examples for that, if you
    know of a good one, I would love to see how it works...

    > in large projects (imho) *much* simpler than
    > vice versa.


    Because I am lazy :) and because it is not a large project.

    > Perl will be much slower if used trivially so (I can tell
    > you this). Just use your matrix in C as usual and make
    > some Into-Perl-Calls if necessary, eg. by pack(...)-ing
    > your variables into Perl-scalars and unpacking
    > them again in the perl.


    Thanks,

    January

    --
    January Weiner, Sep 28, 2006
    #5
  6. January Weiner

    Mirco Wahab Guest

    Thus spoke January Weiner (on 2006-09-28 12:25):

    > I am wondering, however, how complicated it is to link an existing C
    > library to Perl. I am sure there are dozens of examples for that, if you
    > know of a good one, I would love to see how it works...


    OK, I see your point now. You could give
    a link option afaik by :

    use Inline C => Config => LIBS => '-L/your/path -lyourlib';

    .... according to the "Inline" docs

    Did you think about PDL?

    ==>
    http://pdl.sourceforge.net/PDLdocs/API.html#creating_a_piddle_in_c

    Regards

    Mirco
    Mirco Wahab, Sep 28, 2006
    #6
  7. January Weiner

    Sisyphus Guest

    "January Weiner" <> wrote in message
    ..
    ..
    > (I tried to look up the
    > definition of Inline_Stack_Push, but I have trouble finding it).
    >


    I meant to address this in my earlier post (but forgot).
    Inline_Stack_Vars and friends are defined in INLINE.h which is a header file
    auto-generated by Inline::C. It gets written in the build directory - and
    then cleaned up once the compilation has successfully completed (unless you
    run with the Config option 'CLEAN_AFTER_BUILD => 0').

    Here's what INLINE.h contains (copied from C.pm):

    #define Inline_Stack_Vars dXSARGS
    #define Inline_Stack_Items items
    #define Inline_Stack_Item(x) ST(x)
    #define Inline_Stack_Reset sp = mark
    #define Inline_Stack_Push(x) XPUSHs(x)
    #define Inline_Stack_Done PUTBACK
    #define Inline_Stack_Return(x) XSRETURN(x)
    #define Inline_Stack_Void XSRETURN(0)

    #define INLINE_STACK_VARS Inline_Stack_Vars
    #define INLINE_STACK_ITEMS Inline_Stack_Items
    #define INLINE_STACK_ITEM(x) Inline_Stack_Item(x)
    #define INLINE_STACK_RESET Inline_Stack_Reset
    #define INLINE_STACK_PUSH(x) Inline_Stack_Push(x)
    #define INLINE_STACK_DONE Inline_Stack_Done
    #define INLINE_STACK_RETURN(x) Inline_Stack_Return(x)
    #define INLINE_STACK_VOID Inline_Stack_Void

    #define inline_stack_vars Inline_Stack_Vars
    #define inline_stack_items Inline_Stack_Items
    #define inline_stack_item(x) Inline_Stack_Item(x)
    #define inline_stack_reset Inline_Stack_Reset
    #define inline_stack_push(x) Inline_Stack_Push(x)
    #define inline_stack_done Inline_Stack_Done
    #define inline_stack_return(x) Inline_Stack_Return(x)
    #define inline_stack_void Inline_Stack_Void

    Cheers,
    Rob
    Sisyphus, Sep 29, 2006
    #7
  8. Sisyphus <> wrote:
    > Inline_Stack_Vars and friends are defined in INLINE.h which is a header file
    > auto-generated by Inline::C. It gets written in the build directory - and
    > then cleaned up once the compilation has successfully completed (unless you
    > run with the Config option 'CLEAN_AFTER_BUILD => 0').


    Oh, that I do know, I have found that :) It's just that it didn't tell me
    much, because it just redefines the macros. (define inline_stack_push as
    Inline_Stack_Push, and earlier Inline_Stack_Push as XPUSHs... but I still
    do not know what XPUSHs is).

    Anyway, this doesn't matter, as the (stupid! stupid!) error was completly
    on my side.

    j.

    --
    January Weiner, Sep 29, 2006
    #8
  9. Mirco Wahab <> wrote:
    > OK, I see your point now. You could give
    > a link option afaik by :


    > use Inline C => Config => LIBS => '-L/your/path -lyourlib';


    Ah, that is nice (I should have RTFM...).

    > Did you think about PDL?


    > ==>
    > http://pdl.sourceforge.net/PDLdocs/API.html#creating_a_piddle_in_c


    And this is really nice. Many thanks! (and my program works beautifully)

    January

    --
    January Weiner, Sep 29, 2006
    #9
  10. January Weiner

    Ben Morrow Guest

    Quoth January Weiner <>:
    > Sisyphus <> wrote:
    > > Inline_Stack_Vars and friends are defined in INLINE.h which is a header file
    > > auto-generated by Inline::C. It gets written in the build directory - and
    > > then cleaned up once the compilation has successfully completed (unless you
    > > run with the Config option 'CLEAN_AFTER_BUILD => 0').

    >
    > Oh, that I do know, I have found that :) It's just that it didn't tell me
    > much, because it just redefines the macros. (define inline_stack_push as
    > Inline_Stack_Push, and earlier Inline_Stack_Push as XPUSHs... but I still
    > do not know what XPUSHs is).


    See perlapi. IMHO those Inline_Foo macros are pretty stupid: just
    renaming the XS macros for the hell of it.

    Ben

    --
    Heracles: Vulture! Here's a titbit for you / A few dried molecules of the gall
    From the liver of a friend of yours. / Excuse the arrow but I have no spoon.
    (Ted Hughes, [ Heracles shoots Vulture with arrow. Vulture bursts into ]
    'Alcestis') [ flame, and falls out of sight. ]
    Ben Morrow, Sep 29, 2006
    #10
  11. January Weiner

    Sisyphus Guest

    "January Weiner" <> wrote in message
    ..
    ..
    > but I still
    > do not know what XPUSHs is).
    >


    perldoc perlapi :)

    Cheers,
    Rob
    Sisyphus, Sep 29, 2006
    #11
  12. January Weiner

    Guest

    January Weiner <> wrote:
    > Dear all,
    >
    > I had to revert to C again. Since the thought of parsing all my files in
    > C was dreadful, I decided to use Inline::C to only do the job that was
    > really computationally intensive. Currently, my program seems to run
    > quite well, I am very happy with Inline::C, and a little coding in C made
    > me bless Larry again and again.
    >
    > There are, however, few questions that remain open.
    >
    > 1. In the Inline::C cookbook, it says
    >
    > > I would urge you to stay away from "malloc"ing your own buffer. Just
    > > use Perl's built in memory management. In other words, just create
    > > a new Perl string scalar. The function "newSVpv" does just that. And
    > > "newSVpvf" includes "sprintf" functionality.

    >
    > Why? I mean, is there is any danger with malloc() from within
    > Inline::C that is not present in regular C programs, or is it because
    > using malloc is just generally tricky?


    I think that it is some of both. Someone else pointed out New() vs. malloc
    (which I've never been burned by myself) so I would just say that if you
    malloc memory, you have to figure out when to free it, while Perl does the
    refcounting for you (but I can never get it to work the first time on
    complicated Inline code. Either it isn't intuitive or I haven't used it
    enough.)


    > I much prefer to do the
    > allocation myself, as I know exactly how large my matrix is and as I
    > want to allocate the whole matrix ( say, double 2000 x 2000 ) in one
    > go myself.


    Will that fit as an automatic (I think that is the term) variable?
    double x[2000][2000];

    If this works, and it has the appropriate scope for what you need (i.e.
    only needed inside one sub, no passing data back and forth from C to Perl)
    this is the way I would do it.


    > The reason is that I fear that using perl guts for my
    > calculations (accessing the matrix, calculating values in it etc.)
    > will be not much slower than a simple C implementation. Maybe I am
    > wrong, I haven't tested it.


    You don't need to use perl guts for each manipulation. You can get the
    char* to the start of the Perl sv string memory, and then cast that into
    whatever you want. Store that casted pointer, and use it just like there
    were no Perl involved from then on.

    my $x= pack "d*", @list_of_numbers;
    foo($x,scalar @list_of_numbers);

    __C__

    void foo (unsigned char* s, int num_doubles) {
    double * x = (double*) s;
    //now use x as any other double *, being sure not to go out of bounds.
    }


    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , Oct 1, 2006
    #12
  13. wrote:
    > Will that fit as an automatic (I think that is the term) variable?
    > double x[2000][2000];


    Unfortunately, no, as I do not know how large the matrix is going to be.

    > You don't need to use perl guts for each manipulation. You can get the
    > char* to the start of the Perl sv string memory, and then cast that into
    > whatever you want. Store that casted pointer, and use it just like there
    > were no Perl involved from then on.


    > my $x= pack "d*", @list_of_numbers;
    > foo($x,scalar @list_of_numbers);


    > __C__


    > void foo (unsigned char* s, int num_doubles) {
    > double * x = (double*) s;
    > //now use x as any other double *, being sure not to go out of bounds.
    > }


    This is nice, thanks!
    j.

    --
    January Weiner, Oct 4, 2006
    #13
  14. [A complimentary Cc of this posting was sent to

    <>], who wrote in article <20061001002747.059$>:
    > You don't need to use perl guts for each manipulation. You can get the
    > char* to the start of the Perl sv string memory, and then cast that into
    > whatever you want. Store that casted pointer, and use it just like there
    > were no Perl involved from then on.
    >
    > my $x= pack "d*", @list_of_numbers;
    > foo($x,scalar @list_of_numbers);
    >
    > __C__
    >
    > void foo (unsigned char* s, int num_doubles) {
    > double * x = (double*) s;
    > //now use x as any other double *, being sure not to go out of bounds.
    > }


    If this works, then by coincidence only. Memory alignment of SvPV()
    and of double* is absolutely different.

    Hope this helps,
    Ilya

    P.S. In *many* cases SvPV() will point at the start of malloc()ed
    region of size

    1 + sizeof(double) * scalar @list_of_numbers

    Since this is not divisible by sizeof(double), some m_alloc()ers
    may align it differently than requests for sizeof(double) * n.

    Moreover, there is no guarantie that SvPV() points at start of
    malloc()ed region.
    Ilya Zakharevich, Oct 4, 2006
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Abhi
    Replies:
    2
    Views:
    734
    E. Robert Tisdale
    Jul 3, 2003
  2. Alvin
    Replies:
    7
    Views:
    473
    E. Robert Tisdale
    May 6, 2005
  3. Replies:
    3
    Views:
    456
  4. Daniel Vallstrom
    Replies:
    2
    Views:
    1,867
    Kevin Bracey
    Nov 21, 2003
  5. Nish
    Replies:
    4
    Views:
    484
    Thomas Stegen
    Oct 8, 2004
Loading...

Share This Page