Need help with perlxs and C strings

[Thomas]

I've spent several hours now reading documentation, a tutorial and a
magazine article on perlxs and I can't get a minimal example to run.
What I want to accomplish:

* Write some data into a C char array in C code, e.g. like this:
void set_data(char * s)
{
s[0] = 'a';
s[1] = 'b';
}

* Call that C function from a Perl module and use the String from within
a Perl program. For the moment, just print it to stdout.

Can anyone provide me with a small running example which I can extend?
At http://search.cpan.org/~rgarcia/perl-5.10.0/pod/perlxs.pod there are
many pages of documentation but no concise instructions on what to do.
Other tutorials start with a call to h2xs and pages of modifications to
be made just for me to learn that the whole thing doesn't work (example:
"change line with X to Y", only there is no line X). I'd like to use a
small working example if you know of such a thing. If not, maybe someone
can write it up, it shouldn't be more than a couple of lines?!

Part of my problem is that I don't know which tools to use on which
files in what order. Documentation is lengthy, but that basic
information seems to be missing. All solutions seem to work with lengthy
Makefile constructs which I'd like to avoid for the time being.

Thanks in advance, any help very much appreciated.

 

[smallpond]

I've spent several hours now reading documentation, a tutorial and a
magazine article on perlxs and I can't get a minimal example to run.
What I want to accomplish:

* Write some data into a C char array in C code, e.g. like this:
void set_data(char * s)
{
s[0] = 'a';
s[1] = 'b';

}

* Call that C function from a Perl module and use the String from within
a Perl program. For the moment, just print it to stdout.

Can anyone provide me with a small running example which I can extend?
Athttp://search.cpan.org/~rgarcia/perl-5.10.0/pod/perlxs.podthere are
many pages of documentation but no concise instructions on what to do.
Other tutorials start with a call to h2xs and pages of modifications to
be made just for me to learn that the whole thing doesn't work (example:
"change line with X to Y", only there is no line X). I'd like to use a
small working example if you know of such a thing. If not, maybe someone
can write it up, it shouldn't be more than a couple of lines?!

Part of my problem is that I don't know which tools to use on which
files in what order. Documentation is lengthy, but that basic
information seems to be missing. All solutions seem to work with lengthy
Makefile constructs which I'd like to avoid for the time being.

Thanks in advance, any help very much appreciated.

Your array is not a legal C string (not null-terminated), and also not
a legal perl scalar type, which is called SV in perlguts. You would
use sv_setpvn to convert a C char array into a perl scalar.

There are plenty of CPAN modules that are partly written in C. Cwd is
one example that handles strings.

--S

 

[Leon Timmermans]

Part of my problem is that I don't know which tools to use on which
files in what order. Documentation is lengthy, but that basic
information seems to be missing.

I think you will want to read perlguts (and maybe some of perlapi) before
trying to do that. They cover the basics.

Regards,

Leon Timmermans

 

[xhoster]

I've spent several hours now reading documentation, a tutorial and a
magazine article on perlxs and I can't get a minimal example to run.

Writing directly in XS is hard. Try Inline::C for something easier.

What I want to accomplish:

* Write some data into a C char array in C code, e.g. like this:
void set_data(char * s)
{
s[0] = 'a';
s[1] = 'b';
}

* Call that C function from a Perl module and use the String from within
a Perl program. For the moment, just print it to stdout.

use strict;
use warnings;
use Inline 'C' ;
my $x="xxxxxxx";
print "$x\n";
set_data($x);
print "$x\n";
__DATA__
__C__
void set_data(char * s)
{
s[0] = 'a';
s[1] = 'b';
}

output:
xxxxxx
abxxxx


It is easy to get segfaults and other bad things using this method, if
you aren't careful. For example, the SV passed in ($x, here) better have
storage already allocated for those two bytes, and already think the string
is at least 2 bytes long.

Can anyone provide me with a small running example which I can extend?

See above. Also, you can tell Inline not to clean up the build directory
(CLEAN_AFTER_BUILD => 0), so you can also go searching through the _Inline
build directory to see what the xs that Inline makes on your behalf looks
like.

At http://search.cpan.org/~rgarcia/perl-5.10.0/pod/perlxs.pod there are
many pages of documentation but no concise instructions on what to do.

The "what to do" is implicit. It is documentation. It is there to be
read. read it, if you want to understand what XS. If you want to flail
around using the trial and error, then the docs is not what you want.

Other tutorials start with a call to h2xs and pages of modifications to
be made just for me to learn that the whole thing doesn't work (example:
"change line with X to Y", only there is no line X).

Before discovering the terrifying simplicity of Inline::C, I followed the
examples given in, ah, some tutorial or another that involved h2xs. It
worked to get me a working example. If it didn't work for you, what can I
say? You haven't given us enough details. If the docs are wrong, we might
be able to fix them, but not if you hide the errors behind "X" and "Y".

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[Thomas]

Thanks everyone for your replies so far!

The "what to do" is implicit. It is documentation. It is there to be
read. read it, if you want to understand what XS. If you want to flail
around using the trial and error, then the docs is not what you want.

I've read it twice and am still unsure about what tools to use in what
order. That may be my fault, but the problem is not from lack of trying
on my part.

Before discovering the terrifying simplicity of Inline::C, I followed the
examples given in, ah, some tutorial or another that involved h2xs. It
worked to get me a working example. If it didn't work for you, what can I
say? You haven't given us enough details. If the docs are wrong, we might
be able to fix them, but not if you hide the errors behind "X" and "Y".

I want to interface with existing C code which is not supposed to be
copied into the Perl code. But you're right that I wasn't specific
enough. Here's what I have so far. I just want to get that small example
running (the set_data code will do something more complicated, but
that's not the point here):

== setdata.h
void set_data(char * data);
==

== setdata.c
#include "setdata.h"

void set_data(char * data)
{
data[0] = 'a';
data[1] = 'c';
data[2] = '\000';
}
==

== setdata.xs
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "setdata.h"

MODULE = Setdata PACKAGE = Setdata

void
set_data(data)
char * data
OUTPUT:
data
==

== Setdata.pm
package Setdata;

require Exporter;
require DynaLoader;
@ISA = qw(Exporter DynaLoader);
@EXPORT = qw( set_data );

bootstrap Setdata;
1;
==

== typemap
char* T_PV
==

== runme.pl
use Setdata;

my $x="xxxxxxxx";
set_data($x);
print "Data from C function:\n$x";
==

My Perl version: "This is perl, v5.8.5 built for
MSWin32-x86-multi-thread". When running

xsubpp setdata.xs > setdata.xsc

I get the following output:
==
Error: No INPUT definition for type 'char *', typekind 'T_PV' found in
setdata.xs, line 10
Error: No OUTPUT definition for type 'char *', typekind 'T_PV' found in
setdata.xs, line 12
Please specify prototyping behavior for setdata.xs (see perlxs manual)
==

What would I have to change in the .xs file to get it right?

If it works, what would I have to do with the resulting .xsc file?
Compile with gcc? Will the resulting object file be sufficient for
Setdata.pm? Right now, runme.pl simply says

Can't locate loadable object for module Setdata in @INC ( <Perl includes> ).

 

[xhoster]

I want to interface with existing C code which is not supposed to be
copied into the Perl code.

With Inline::C, you can #include C files without copying them into the Perl
code. You can't call functions defined in those #includes directly from
Perl (as far as I know) but you can write Inline::C stub subs that call
those external functions. That is the way I do it, easier than learning XS
when I don't need all the complexities and flexibilities that XS
introduces.

But you're right that I wasn't specific
enough. Here's what I have so far. I just want to get that small example
running (the set_data code will do something more complicated, but
that's not the point here):
.....

== typemap
char* T_PV
==
....

xsubpp setdata.xs > setdata.xsc

I get the following output:
==
Error: No INPUT definition for type 'char *', typekind 'T_PV' found in
setdata.xs, line 10
Error: No OUTPUT definition for type 'char *', typekind 'T_PV' found in
setdata.xs, line 12

A general typemap has three section, but you only have one in yours.

From the typemap, it knows to translate char * to T_PV, but it doesn't
know how to handle a T_PV. If I take the system typemap, and eliminate
everything not related to T_PV, I get the below typemap, with which
xsubpp can run:

==typemap
# basic C types
char * T_PV

##########################################
INPUT
T_PV
$var = ($type)SvPV_nolen($arg)
##########################################
OUTPUT
T_PV
sv_setpv((SV*)$arg, $var);
==

But it seems like it would be best to just tell xsubpp to use the system
typemap in the first place, instead of creating your own which adds nothing
new to the system one.

If it works, what would I have to do with the resulting .xsc file?

I don't know. Who's instructions were you following to get to this
point in the first place? perlxstut doesn't refer to xsc files.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[Ben Morrow]

Quoth (e-mail address removed):

With Inline::C, you can #include C files without copying them into the Perl
code. You can't call functions defined in those #includes directly from
Perl (as far as I know) but you can write Inline::C stub subs that call
those external functions. That is the way I do it, easier than learning XS
when I don't need all the complexities and flexibilities that XS
introduces.

See also the AUTOWRAP option. (I really should see if I can't get FFI.pm
to work properly: it's *such* a better way to connect to C libs.)

I don't know. Who's instructions were you following to get to this
point in the first place? perlxstut doesn't refer to xsc files.

The Makefile generated by EU::MM does something like

xsubpp Foo.xs > Foo.xsc && mv Foo.xsc Foo.c
cc Foo.c -o Foo.o

so the correct answer is: don't invoke xsubpp directly, use
ExtUtils::MakeMaker, or Module::Install, or Module::Build.

Ben

 

[xhoster]

Quoth (e-mail address removed):

See also the AUTOWRAP option.

Yep, I was just exploring that option. In my first attempt I had to copy
the .h file into the __C__ section. But then I got it to "source" the .h
directly:

use Inline 'C' => qq{#include "setdata.c"\n} . `cat setdata.h`
=> ENABLE => 'AUTOWRAP';

And it works fine.

Other than the fact that the OP is treating Perl strings as if they were C
strings. \000 is not a string terminator in Perl, and Perl stores the
length of its strings rather than computing on the fly. When you pass an
SV as if it were a char *, you lose the ability to change those things. I
don't know of any automatic fix for people not knowing what they are doing.
That's why XS is a pain and Inline::C is terrifying.


Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[Thomas]

Yep, I was just exploring that option. In my first attempt I had to copy
the .h file into the __C__ section. But then I got it to "source" the .h
directly:

use Inline 'C' => qq{#include "setdata.c"\n} . `cat setdata.h`
=> ENABLE => 'AUTOWRAP';

And it works fine.

I have to interface with compiled C code. I got Inline C to work (ran
the http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Hello,_world
sample program successfully), but my modified version of runme.pl
doesn't seem to recognize where the compiled object file of setdata.c is
located. I followed the example at
http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Automatic_Function_Wrappers
and came up with this:

== runme.pl
use Inline C => Config =>
ENABLE => AUTOWRAP =>
LIBS => "-L. -lsetdata";
use Inline C => q{ void set_data(char *); };

my $x="xxxxxxxx";
set_data($x);
print "Data from C function:\n$x";
==

The compiled version of setdata.c (setdata.obj) is placed in the same
directory. However, when running the above script, the linker fails with
this message:

runme_pl_5bec.obj : error LNK2019: unresolved external symbol _set_data
referenced in function _XS_main_set_data

I have to add that I use nmake and cl as compiler and link as linker
under Windows. The information from LIBS never seems to get included (I
only know the -l<lib> syntax from Unix, it may be the problem; I don't
know the alternative). I'd appreciate any further help on how to get
this running.

 

[xhoster]

== runme.pl
use Inline C => Config =>
ENABLE => AUTOWRAP =>
LIBS => "-L. -lsetdata";
use Inline C => q{ void set_data(char *); };
....

The compiled version of setdata.c (setdata.obj) is placed in the same
directory. However, when running the above script, the linker fails with
this message:

runme_pl_5bec.obj : error LNK2019: unresolved external symbol _set_data
referenced in function _XS_main_set_data

I have to add that I use nmake and cl as compiler and link as linker
under Windows.

Unfortunately, I know nothing about building things on Windows.

The information from LIBS never seems to get included (I
only know the -l<lib> syntax from Unix, it may be the problem; I don't
know the alternative). I'd appreciate any further help on how to get
this running.

Do you know how to link things under your set-up in Windows directly
(i.e. when not using Perl)? You can tell Inline to do a noisy build, where
it will show you every step it is carrying out. If you know what the link
options is supposed to look like in Windows, maybe you can use the
information from the noisy build to figure how to tweak the options to make
that happen.

Also, is the "cl" compiler the compiler which perl itself was compiled
on? If not, you might run in problems no matter what.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[Ben Morrow]

I have to interface with compiled C code. I got Inline C to work (ran
the http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Hello,_world
sample program successfully), but my modified version of runme.pl
doesn't seem to recognize where the compiled object file of setdata.c is
located. I followed the example at
http://search.cpan.org/dist/Inline/C/C-Cookbook.pod#Automatic_Function_Wrappers
and came up with this:

== runme.pl
use Inline C => Config =>
ENABLE => AUTOWRAP =>
LIBS => "-L. -lsetdata";
use Inline C => q{ void set_data(char *); };

my $x="xxxxxxxx";
set_data($x);
print "Data from C function:\n$x";
==

The compiled version of setdata.c (setdata.obj) is placed in the same

-lsetdata is looking for a *library*, not an object file. On Win32 it
would probably be called setdata.lib, and made from setdata.obj using
LIB.EXE.

Alternatively, you could try

LIBS => "-L. setdata.obj"

or even

LIBS => '-L. setdata$(OBJ_EXT)'

for portability (but I don't know how much of MakeMaker Inline::C uses:
try it and see).

I have to add that I use nmake and cl as compiler and link as linker
under Windows. The information from LIBS never seems to get included (I
only know the -l<lib> syntax from Unix, it may be the problem; I don't
know the alternative). I'd appreciate any further help on how to get
this running.

It is the correct syntax on Win32 as well (fortunately, since Microsoft
in there great wisdom have renamed -lfoo from libfoo.a to foo.lib), but
for libraries.

Note that the information in LIBS is considered 'optional': MM will link
the libraries if it can find them, and not if it can't.

Ben

 

[sisyphus]

.
.

== runme.pl
use Inline C => Config =>
                ENABLE => AUTOWRAP =>
                LIBS => "-L. -lsetdata";

The "-L." will be a problem as Inline's build directory (which is '.')
is almost certainly not the location that houses the setdata library.
So you'll need to replace that "-L." with the fully qualified location
of the setdata library -ie something like "-LC:/library/location".

And then, if it's not actually a 'setdata.lib' file, I doubt that the
'-lsetdata' will find it. Probably simplest to create a setdata.lib
from the setdata.obj file. With Visual Studio I think it's just a
matter of:

LIB /OUT:setdata.lib setdata.obj

Faik, it may even work if you rename setdata.a to setdata.obj (but I'm
a bit doubtful of that).

Cheers,
Rob

 

[Thomas]

The "-L." will be a problem as Inline's build directory (which is '.')
is almost certainly not the location that houses the setdata library.
So you'll need to replace that "-L." with the fully qualified location
of the setdata library -ie something like "-LC:/library/location". ....
LIB /OUT:setdata.lib setdata.obj

Faik, it may even work if you rename setdata.a to setdata.obj (but I'm
a bit doubtful of that).

Creating the right lib/dll file and making it available to the Inline C
compile process really is the key.

I changed a few things and got it to run. Part of the problem was that I
inherited a rather broken Perl installation on the development box, so
reinstalling Perl was an improvement. So perlxs and SWIG might actually
be easier to use than I thought. Anyway, I used Inline C.

Here's what I did for the archives. If somethings looks weird, please
reply. (About the content of the string and the trailing zero character
- my array never gets treated as a C string in my real application, so I
don't care about that. Handle differently if you must.)

Thanks to everyone who helped!

== setdata.h
void __declspec( dllexport ) set_data(char * data);

== setdata.c
#include "setdata.h"

void __declspec( dllexport ) set_data(char * data)
{
data[0] = 'a';
data[1] = 'c';
}

== runme.pl
use Inline C => Config =>
ENABLE => AUTOWRAP;
use Inline C => q{ void set_data(char *); };

my $x="xxxxxxxx";
set_data($x);
print "Data from C function:\n$x";

== Using it

1) Compile with Visual Studio's command line compiler cl and use the
right linking switches to create setdata.lib and setdata.dll:

cl /LD /MT setdata.c

2) Run the vsvars32.bat file which should be part of your Visual Studio
installation to set all directories in a way that makes cl happy. Make
sure that you add the directory where the setdata lib/dll files are
located or move them to whatever directories already are in the LIB
environment variable. You might also succeed with the -L switch sisyphus
mentioned.

3) Run the Perl script:
perl runme.pl