Need help with perlxs and C strings

Discussion in 'Perl Misc' started by Thomas, Aug 6, 2008.

  1. Thomas

    Thomas Guest

    I've spent several hours now reading documentation, a tutorial and a
    magazine article on perlxs and I can't get a minimal example to run.
    What I want to accomplish:

    * Write some data into a C char array in C code, e.g. like this:
    void set_data(char * s)
    {
    s[0] = 'a';
    s[1] = 'b';
    }

    * Call that C function from a Perl module and use the String from within
    a Perl program. For the moment, just print it to stdout.

    Can anyone provide me with a small running example which I can extend?
    At http://search.cpan.org/~rgarcia/perl-5.10.0/pod/perlxs.pod there are
    many pages of documentation but no concise instructions on what to do.
    Other tutorials start with a call to h2xs and pages of modifications to
    be made just for me to learn that the whole thing doesn't work (example:
    "change line with X to Y", only there is no line X). I'd like to use a
    small working example if you know of such a thing. If not, maybe someone
    can write it up, it shouldn't be more than a couple of lines?!

    Part of my problem is that I don't know which tools to use on which
    files in what order. Documentation is lengthy, but that basic
    information seems to be missing. All solutions seem to work with lengthy
    Makefile constructs which I'd like to avoid for the time being.

    Thanks in advance, any help very much appreciated.
    Thomas, Aug 6, 2008
    #1
    1. Advertising

  2. Thomas

    smallpond Guest

    On Aug 6, 9:12 am, Thomas <> wrote:
    > I've spent several hours now reading documentation, a tutorial and a
    > magazine article on perlxs and I can't get a minimal example to run.
    > What I want to accomplish:
    >
    > * Write some data into a C char array in C code, e.g. like this:
    > void set_data(char * s)
    > {
    > s[0] = 'a';
    > s[1] = 'b';
    >
    > }
    >
    > * Call that C function from a Perl module and use the String from within
    > a Perl program. For the moment, just print it to stdout.
    >
    > Can anyone provide me with a small running example which I can extend?
    > Athttp://search.cpan.org/~rgarcia/perl-5.10.0/pod/perlxs.podthere are
    > many pages of documentation but no concise instructions on what to do.
    > Other tutorials start with a call to h2xs and pages of modifications to
    > be made just for me to learn that the whole thing doesn't work (example:
    > "change line with X to Y", only there is no line X). I'd like to use a
    > small working example if you know of such a thing. If not, maybe someone
    > can write it up, it shouldn't be more than a couple of lines?!
    >
    > Part of my problem is that I don't know which tools to use on which
    > files in what order. Documentation is lengthy, but that basic
    > information seems to be missing. All solutions seem to work with lengthy
    > Makefile constructs which I'd like to avoid for the time being.
    >
    > Thanks in advance, any help very much appreciated.


    Your array is not a legal C string (not null-terminated), and also not
    a legal perl scalar type, which is called SV in perlguts. You would
    use sv_setpvn to convert a C char array into a perl scalar.

    There are plenty of CPAN modules that are partly written in C. Cwd is
    one example that handles strings.

    --S
    smallpond, Aug 6, 2008
    #2
    1. Advertising

  3. On Wed, 06 Aug 2008 15:12:24 +0200, Thomas wrote:

    > Part of my problem is that I don't know which tools to use on which
    > files in what order. Documentation is lengthy, but that basic
    > information seems to be missing.
    >


    I think you will want to read perlguts (and maybe some of perlapi) before
    trying to do that. They cover the basics.

    Regards,

    Leon Timmermans
    Leon Timmermans, Aug 6, 2008
    #3
  4. Thomas

    Guest

    Thomas <> wrote:
    > I've spent several hours now reading documentation, a tutorial and a
    > magazine article on perlxs and I can't get a minimal example to run.


    Writing directly in XS is hard. Try Inline::C for something easier.

    > What I want to accomplish:
    >
    > * Write some data into a C char array in C code, e.g. like this:
    > void set_data(char * s)
    > {
    > s[0] = 'a';
    > s[1] = 'b';
    > }
    >
    > * Call that C function from a Perl module and use the String from within
    > a Perl program. For the moment, just print it to stdout.


    use strict;
    use warnings;
    use Inline 'C' ;
    my $x="xxxxxxx";
    print "$x\n";
    set_data($x);
    print "$x\n";
    __DATA__
    __C__
    void set_data(char * s)
    {
    s[0] = 'a';
    s[1] = 'b';
    }

    output:
    xxxxxx
    abxxxx


    It is easy to get segfaults and other bad things using this method, if
    you aren't careful. For example, the SV passed in ($x, here) better have
    storage already allocated for those two bytes, and already think the string
    is at least 2 bytes long.


    > Can anyone provide me with a small running example which I can extend?


    See above. Also, you can tell Inline not to clean up the build directory
    (CLEAN_AFTER_BUILD => 0), so you can also go searching through the _Inline
    build directory to see what the xs that Inline makes on your behalf looks
    like.


    > At http://search.cpan.org/~rgarcia/perl-5.10.0/pod/perlxs.pod there are
    > many pages of documentation but no concise instructions on what to do.


    The "what to do" is implicit. It is documentation. It is there to be
    read. read it, if you want to understand what XS. If you want to flail
    around using the trial and error, then the docs is not what you want.


    > Other tutorials start with a call to h2xs and pages of modifications to
    > be made just for me to learn that the whole thing doesn't work (example:
    > "change line with X to Y", only there is no line X).


    Before discovering the terrifying simplicity of Inline::C, I followed the
    examples given in, ah, some tutorial or another that involved h2xs. It
    worked to get me a working example. If it didn't work for you, what can I
    say? You haven't given us enough details. If the docs are wrong, we might
    be able to fix them, but not if you hide the errors behind "X" and "Y".

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
    , Aug 6, 2008
    #4
  5. Thomas

    Thomas Guest

    Thanks everyone for your replies so far!

    > The "what to do" is implicit. It is documentation. It is there to be
    > read. read it, if you want to understand what XS. If you want to flail
    > around using the trial and error, then the docs is not what you want.


    I've read it twice and am still unsure about what tools to use in what
    order. That may be my fault, but the problem is not from lack of trying
    on my part.

    > Before discovering the terrifying simplicity of Inline::C, I followed the
    > examples given in, ah, some tutorial or another that involved h2xs. It
    > worked to get me a working example. If it didn't work for you, what can I
    > say? You haven't given us enough details. If the docs are wrong, we might
    > be able to fix them, but not if you hide the errors behind "X" and "Y".


    I want to interface with existing C code which is not supposed to be
    copied into the Perl code. But you're right that I wasn't specific
    enough. Here's what I have so far. I just want to get that small example
    running (the set_data code will do something more complicated, but
    that's not the point here):

    == setdata.h
    void set_data(char * data);
    ==

    == setdata.c
    #include "setdata.h"

    void set_data(char * data)
    {
    data[0] = 'a';
    data[1] = 'c';
    data[2] = '\000';
    }
    ==

    == setdata.xs
    #include "EXTERN.h"
    #include "perl.h"
    #include "XSUB.h"
    #include "setdata.h"

    MODULE = Setdata PACKAGE = Setdata

    void
    set_data(data)
    char * data
    OUTPUT:
    data
    ==

    == Setdata.pm
    package Setdata;

    require Exporter;
    require DynaLoader;
    @ISA = qw(Exporter DynaLoader);
    @EXPORT = qw( set_data );

    bootstrap Setdata;
    1;
    ==

    == typemap
    char* T_PV
    ==

    == runme.pl
    use Setdata;

    my $x="xxxxxxxx";
    set_data($x);
    print "Data from C function:\n$x";
    ==

    My Perl version: "This is perl, v5.8.5 built for
    MSWin32-x86-multi-thread". When running

    xsubpp setdata.xs > setdata.xsc

    I get the following output:
    ==
    Error: No INPUT definition for type 'char *', typekind 'T_PV' found in
    setdata.xs, line 10
    Error: No OUTPUT definition for type 'char *', typekind 'T_PV' found in
    setdata.xs, line 12
    Please specify prototyping behavior for setdata.xs (see perlxs manual)
    ==

    What would I have to change in the .xs file to get it right?

    If it works, what would I have to do with the resulting .xsc file?
    Compile with gcc? Will the resulting object file be sufficient for
    Setdata.pm? Right now, runme.pl simply says

    Can't locate loadable object for module Setdata in @INC ( <Perl includes> ).
    Thomas, Aug 7, 2008
    #5
  6. Thomas

    Guest

    Thomas <> wrote:

    > > Before discovering the terrifying simplicity of Inline::C, I followed
    > > the examples given in, ah, some tutorial or another that involved h2xs.
    > > It worked to get me a working example. If it didn't work for you, what
    > > can I say? You haven't given us enough details. If the docs are
    > > wrong, we might be able to fix them, but not if you hide the errors
    > > behind "X" and "Y".

    >
    > I want to interface with existing C code which is not supposed to be
    > copied into the Perl code.


    With Inline::C, you can #include C files without copying them into the Perl
    code. You can't call functions defined in those #includes directly from
    Perl (as far as I know) but you can write Inline::C stub subs that call
    those external functions. That is the way I do it, easier than learning XS
    when I don't need all the complexities and flexibilities that XS
    introduces.




    > But you're right that I wasn't specific
    > enough. Here's what I have so far. I just want to get that small example
    > running (the set_data code will do something more complicated, but
    > that's not the point here):
    >

    .....
    >
    > == typemap
    > char* T_PV
    > ==
    >

    ....

    > xsubpp setdata.xs > setdata.xsc
    >
    > I get the following output:
    > ==
    > Error: No INPUT definition for type 'char *', typekind 'T_PV' found in
    > setdata.xs, line 10
    > Error: No OUTPUT definition for type 'char *', typekind 'T_PV' found in
    > setdata.xs, line 12


    A general typemap has three section, but you only have one in yours.

    From the typemap, it knows to translate char * to T_PV, but it doesn't
    know how to handle a T_PV. If I take the system typemap, and eliminate
    everything not related to T_PV, I get the below typemap, with which
    xsubpp can run:

    ==typemap
    # basic C types
    char * T_PV

    ##########################################
    INPUT
    T_PV
    $var = ($type)SvPV_nolen($arg)
    ##########################################
    OUTPUT
    T_PV
    sv_setpv((SV*)$arg, $var);
    ==

    But it seems like it would be best to just tell xsubpp to use the system
    typemap in the first place, instead of creating your own which adds nothing
    new to the system one.


    > If it works, what would I have to do with the resulting .xsc file?


    I don't know. Who's instructions were you following to get to this
    point in the first place? perlxstut doesn't refer to xsc files.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
    , Aug 7, 2008
    #6
  7. Thomas

    Ben Morrow Guest

    Quoth :
    > Thomas <> wrote:
    >
    > > I want to interface with existing C code which is not supposed to be
    > > copied into the Perl code.

    >
    > With Inline::C, you can #include C files without copying them into the Perl
    > code. You can't call functions defined in those #includes directly from
    > Perl (as far as I know) but you can write Inline::C stub subs that call
    > those external functions. That is the way I do it, easier than learning XS
    > when I don't need all the complexities and flexibilities that XS
    > introduces.


    See also the AUTOWRAP option. (I really should see if I can't get FFI.pm
    to work properly: it's *such* a better way to connect to C libs.)

    > > xsubpp setdata.xs > setdata.xsc

    <snip>
    > > If it works, what would I have to do with the resulting .xsc file?

    >
    > I don't know. Who's instructions were you following to get to this
    > point in the first place? perlxstut doesn't refer to xsc files.


    The Makefile generated by EU::MM does something like

    xsubpp Foo.xs > Foo.xsc && mv Foo.xsc Foo.c
    cc Foo.c -o Foo.o

    so the correct answer is: don't invoke xsubpp directly, use
    ExtUtils::MakeMaker, or Module::Install, or Module::Build.

    Ben

    --
    Joy and Woe are woven fine,
    A Clothing for the Soul divine William Blake
    Under every grief and pine 'Auguries of Innocence'
    Runs a joy with silken twine.
    Ben Morrow, Aug 7, 2008
    #7
  8. Thomas

    Guest

    Ben Morrow <> wrote:
    > Quoth :
    > > Thomas <> wrote:
    > >
    > > > I want to interface with existing C code which is not supposed to be
    > > > copied into the Perl code.

    > >
    > > With Inline::C, you can #include C files without copying them into the
    > > Perl code. You can't call functions defined in those #includes
    > > directly from Perl (as far as I know) but you can write Inline::C stub
    > > subs that call those external functions. That is the way I do it,
    > > easier than learning XS when I don't need all the complexities and
    > > flexibilities that XS introduces.

    >
    > See also the AUTOWRAP option.


    Yep, I was just exploring that option. In my first attempt I had to copy
    the .h file into the __C__ section. But then I got it to "source" the .h
    directly:

    use Inline 'C' => qq{#include "setdata.c"\n} . `cat setdata.h`
    => ENABLE => 'AUTOWRAP';

    And it works fine.

    Other than the fact that the OP is treating Perl strings as if they were C
    strings. \000 is not a string terminator in Perl, and Perl stores the
    length of its strings rather than computing on the fly. When you pass an
    SV as if it were a char *, you lose the ability to change those things. I
    don't know of any automatic fix for people not knowing what they are doing.
    That's why XS is a pain and Inline::C is terrifying.


    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
    , Aug 7, 2008
    #8
  9. Thomas

    Thomas Guest

    wrote:

    > Yep, I was just exploring that option. In my first attempt I had to copy
    > the .h file into the __C__ section. But then I got it to "source" the .h
    > directly:
    >
    > use Inline 'C' => qq{#include "setdata.c"\n} . `cat setdata.h`
    > => ENABLE => 'AUTOWRAP';
    >
    > And it works fine.


    I have to interface with compiled C code. I got Inline C to work (ran
    the http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Hello,_world
    sample program successfully), but my modified version of runme.pl
    doesn't seem to recognize where the compiled object file of setdata.c is
    located. I followed the example at
    http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Automatic_Function_Wrappers
    and came up with this:

    == runme.pl
    use Inline C => Config =>
    ENABLE => AUTOWRAP =>
    LIBS => "-L. -lsetdata";
    use Inline C => q{ void set_data(char *); };

    my $x="xxxxxxxx";
    set_data($x);
    print "Data from C function:\n$x";
    ==

    The compiled version of setdata.c (setdata.obj) is placed in the same
    directory. However, when running the above script, the linker fails with
    this message:

    runme_pl_5bec.obj : error LNK2019: unresolved external symbol _set_data
    referenced in function _XS_main_set_data

    I have to add that I use nmake and cl as compiler and link as linker
    under Windows. The information from LIBS never seems to get included (I
    only know the -l<lib> syntax from Unix, it may be the problem; I don't
    know the alternative). I'd appreciate any further help on how to get
    this running.
    Thomas, Aug 8, 2008
    #9
  10. Thomas

    Guest

    Thomas <> wrote:

    > == runme.pl
    > use Inline C => Config =>
    > ENABLE => AUTOWRAP =>
    > LIBS => "-L. -lsetdata";
    > use Inline C => q{ void set_data(char *); };
    >

    ....
    >
    > The compiled version of setdata.c (setdata.obj) is placed in the same
    > directory. However, when running the above script, the linker fails with
    > this message:
    >
    > runme_pl_5bec.obj : error LNK2019: unresolved external symbol _set_data
    > referenced in function _XS_main_set_data
    >
    > I have to add that I use nmake and cl as compiler and link as linker
    > under Windows.


    Unfortunately, I know nothing about building things on Windows.

    > The information from LIBS never seems to get included (I
    > only know the -l<lib> syntax from Unix, it may be the problem; I don't
    > know the alternative). I'd appreciate any further help on how to get
    > this running.


    Do you know how to link things under your set-up in Windows directly
    (i.e. when not using Perl)? You can tell Inline to do a noisy build, where
    it will show you every step it is carrying out. If you know what the link
    options is supposed to look like in Windows, maybe you can use the
    information from the noisy build to figure how to tweak the options to make
    that happen.

    Also, is the "cl" compiler the compiler which perl itself was compiled
    on? If not, you might run in problems no matter what.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
    , Aug 8, 2008
    #10
  11. Thomas

    Ben Morrow Guest

    Quoth Thomas <>:
    > wrote:
    >
    > > Yep, I was just exploring that option. In my first attempt I had to copy
    > > the .h file into the __C__ section. But then I got it to "source" the .h
    > > directly:
    > >
    > > use Inline 'C' => qq{#include "setdata.c"\n} . `cat setdata.h`
    > > => ENABLE => 'AUTOWRAP';
    > >
    > > And it works fine.

    >
    > I have to interface with compiled C code. I got Inline C to work (ran
    > the http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Hello,_world
    > sample program successfully), but my modified version of runme.pl
    > doesn't seem to recognize where the compiled object file of setdata.c is
    > located. I followed the example at
    > http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Automatic_Function_Wrappers
    > and came up with this:
    >
    > == runme.pl
    > use Inline C => Config =>
    > ENABLE => AUTOWRAP =>
    > LIBS => "-L. -lsetdata";
    > use Inline C => q{ void set_data(char *); };
    >
    > my $x="xxxxxxxx";
    > set_data($x);
    > print "Data from C function:\n$x";
    > ==
    >
    > The compiled version of setdata.c (setdata.obj) is placed in the same


    -lsetdata is looking for a *library*, not an object file. On Win32 it
    would probably be called setdata.lib, and made from setdata.obj using
    LIB.EXE.

    Alternatively, you could try

    LIBS => "-L. setdata.obj"

    or even

    LIBS => '-L. setdata$(OBJ_EXT)'

    for portability (but I don't know how much of MakeMaker Inline::C uses:
    try it and see).

    > I have to add that I use nmake and cl as compiler and link as linker
    > under Windows. The information from LIBS never seems to get included (I
    > only know the -l<lib> syntax from Unix, it may be the problem; I don't
    > know the alternative). I'd appreciate any further help on how to get
    > this running.


    It is the correct syntax on Win32 as well (fortunately, since Microsoft
    in there great wisdom have renamed -lfoo from libfoo.a to foo.lib), but
    for libraries.

    Note that the information in LIBS is considered 'optional': MM will link
    the libraries if it can find them, and not if it can't.

    Ben

    --
    I've seen things you people wouldn't believe: attack ships on fire off
    the shoulder of Orion; I watched C-beams glitter in the dark near the
    Tannhauser Gate. All these moments will be lost, in time, like tears in rain.
    Time to die.
    Ben Morrow, Aug 8, 2008
    #11
  12. Thomas

    sisyphus Guest

    On Aug 9, 12:49 am, Thomas <> wrote:
    .
    .
    >
    > == runme.pl
    > use Inline C => Config =>
    >                 ENABLE => AUTOWRAP =>
    >                 LIBS => "-L. -lsetdata";


    The "-L." will be a problem as Inline's build directory (which is '.')
    is almost certainly not the location that houses the setdata library.
    So you'll need to replace that "-L." with the fully qualified location
    of the setdata library -ie something like "-LC:/library/location".

    And then, if it's not actually a 'setdata.lib' file, I doubt that the
    '-lsetdata' will find it. Probably simplest to create a setdata.lib
    from the setdata.obj file. With Visual Studio I think it's just a
    matter of:

    LIB /OUT:setdata.lib setdata.obj

    Faik, it may even work if you rename setdata.a to setdata.obj (but I'm
    a bit doubtful of that).

    Cheers,
    Rob
    sisyphus, Aug 9, 2008
    #12
  13. Thomas

    Thomas Guest

    Inline C, char* and Visual Studio (was: Re: Need help with perlxsand C strings)

    sisyphus wrote:

    > The "-L." will be a problem as Inline's build directory (which is '.')
    > is almost certainly not the location that houses the setdata library.
    > So you'll need to replace that "-L." with the fully qualified location
    > of the setdata library -ie something like "-LC:/library/location".

    ....
    > LIB /OUT:setdata.lib setdata.obj
    >
    > Faik, it may even work if you rename setdata.a to setdata.obj (but I'm
    > a bit doubtful of that).


    Creating the right lib/dll file and making it available to the Inline C
    compile process really is the key.

    I changed a few things and got it to run. Part of the problem was that I
    inherited a rather broken Perl installation on the development box, so
    reinstalling Perl was an improvement. So perlxs and SWIG might actually
    be easier to use than I thought. Anyway, I used Inline C.

    Here's what I did for the archives. If somethings looks weird, please
    reply. (About the content of the string and the trailing zero character
    - my array never gets treated as a C string in my real application, so I
    don't care about that. Handle differently if you must.)

    Thanks to everyone who helped!

    == setdata.h
    void __declspec( dllexport ) set_data(char * data);

    == setdata.c
    #include "setdata.h"

    void __declspec( dllexport ) set_data(char * data)
    {
    data[0] = 'a';
    data[1] = 'c';
    }

    == runme.pl
    use Inline C => Config =>
    ENABLE => AUTOWRAP;
    use Inline C => q{ void set_data(char *); };

    my $x="xxxxxxxx";
    set_data($x);
    print "Data from C function:\n$x";

    == Using it

    1) Compile with Visual Studio's command line compiler cl and use the
    right linking switches to create setdata.lib and setdata.dll:

    cl /LD /MT setdata.c

    2) Run the vsvars32.bat file which should be part of your Visual Studio
    installation to set all directories in a way that makes cl happy. Make
    sure that you add the directory where the setdata lib/dll files are
    located or move them to whatever directories already are in the LIB
    environment variable. You might also succeed with the -L switch sisyphus
    mentioned.

    3) Run the Perl script:
    perl runme.pl
    Thomas, Aug 11, 2008
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mike Jones
    Replies:
    6
    Views:
    353
    David Harmon
    Mar 1, 2004
  2. Kurt Krueckeberg
    Replies:
    2
    Views:
    694
    =?ISO-8859-1?Q?Ney_Andr=E9_de_Mello_Zunino?=
    Nov 17, 2004
  3. Ben

    Strings, Strings and Damned Strings

    Ben, Jun 22, 2006, in forum: C Programming
    Replies:
    14
    Views:
    733
    Malcolm
    Jun 24, 2006
  4. Torsten Mohr

    C++ example in perldoc perlxs

    Torsten Mohr, Oct 7, 2003, in forum: Perl Misc
    Replies:
    1
    Views:
    87
  5. Dilbert
    Replies:
    3
    Views:
    132
    Dilbert
    May 15, 2010
Loading...

Share This Page