Perl Embedding Question

[Steve Titus]

I am a veteran Perl user, but a newbie embedder. I am trying to figure
out the best way to solve a problem I have. Please read along--any
advice is appreciated.

TASK: I am talking to a networked camera via HTTP, and the URLs I need
to send and the requests I get back are a variety of fairly complicated
and not always consistent text strings. THE GOAL: I need to interface
with the camera from a C/C++ program.

APPROACH: the camera came with a hard-copy manual that spelled out all
the camera params and commands, syntax, etc. I slapped all the info into
a config file in my own white-space-delimited column format that is
pretty simple to read and understand. As far as parsing goes, I figure
there are 4 general approaches I could have taken:

1. Hack something together using sscanf, strtok, and their ilk, and not
have it robust, but get it working quickly. NOT AN OPTION.

2. Do it "right" in C using lex/yacc, bison/flex, etc. I have done
parsers in this manner, but generally takes longer than using a Perl
approach, and you have to debug the grammar corner cases, etc.

3. Embed Perl into the C program and use Perl for all the text munging.
Has the added advantage of being flexible, not having to recompile when
config file changes, yada.

4. Do the config file in standardized text format like XML, and it may
not be too readable to humans, but I am sure there are some free
libraries that will parse it easily and hand me my data in C nicely.

COMMENTS WELCOME on which approach is best...but I chose 3.

Now to the Perl question:

PERL EMBEDDING: I read the manual and got ExtUtils::Embed working just
fine. I created 3 simple classes: perl_hash, perl_array, and
perl_scalar, in C++, and wrote a simple wrapper class that contains the
Perl interpreter and has one method -- call(). You can call any perl
function you like, hand it a C++ perl_array (@_) that you've filled
inside C/C++ and get back a perl_array as a result from call() after it
invokes Perl. It works great, and I was very proud of myself for taking
a general embedding approach that would work in mutliple projects! But....

PROBLEM: the problem is that I just let ExtUtils::Embed spew out all the
compile and link flags that I used for building, and when I run the
executable on another machine, it barfs with different messages about
shared libararies (depending on machine on which it is run), with 2
representative error messages being:

cperltest: error while loading shared libraries: libperl.so: cannot open
shared object file: No such file or directory

cperltest: /lib/i686/libc.so.6: version `GLIBC_2.3' not found (required
by cperltest)


QUESTIONS:

I have a basic understanding of what shared libraries are and how they
work, but I am not knowledgable about GLIBC compatibilities, or about
the "best" way to attain the following goal: I want someone to be able
to write a C/C++ program that links against my Cperl library, and as far
as they are concerned, it will just work.

1. Is this possible?

2. Could someone please give me a short primer on the way that GCC build
env, user environment (e.g. LD_LIBRARY_PATH), and Perl environment (how
Perl was built, etc), all influence what the "right" answer is to
solving this problem? All these issues have me confused!

3. What approach should I take? The ground rules are that all the boxes
I am working with will all have basically same O/S -- all Red Hat boxes
that look pretty much like

Linux 2.4.20-19.7smp #1 SMP Tue Jul 15 13:34:04 EDT 2003 i686 unknown

but they may have slightly different Perl versions installed in slightly
different places. SHould I compile everything statically so that my
cperl.a essentially contains a stand-alone Perl? How would I go about
doing this if it's possible?

4. How do you figure out what shared libraries an executable depends on?

5. How do you statically compile an executable (gcc version 3.2.3) so
that it depends on NOTHING?

6. Should I even try to compile statically, or can I somehow keep things
the way they are, and just have everyone add a few things to their
LD_LIBRARY_PATH on my behalf? Any philosophical and pratical help would
be great...


Any and all help is so greatly appreciated! Thanks!


Steve

 

[Peter]

4. How do you figure out what shared libraries an executable depends on?

Run your program with strace.
If you are using the default Perl installation in redhat9, all your perl*
dependencies will be in /usr/lib/perl5.
The remaining nonperl-specific stuff you can easily track with strace.

Hope this helps!


Peter

 

[Ben Morrow]

PROBLEM: the problem is that I just let ExtUtils::Embed spew out all the
compile and link flags that I used for building, and when I run the
executable on another machine,

Bad plan. In general, and especially when dealing with perl, you should
build an executable on the machine it is going to run on, using the
appropriate ExtUtils::Embed output on that machine. This does apply much
more generally than just embedding perl: I'm surprised you've not been
bitten by this before. The one exception is binary distros, which work
very hard to make machines clones of each other, at least insofar as
library locations/version/etc are concerned. If you want to go down that
route, you would definitely be best off making a rpm or whatever, and
finding out how to properly interface with the package management
system; this will be a big job, though.

it barfs with different messages about
shared libararies (depending on machine on which it is run), with 2
representative error messages being:

cperltest: error while loading shared libraries: libperl.so: cannot open
shared object file: No such file or directory

libperl.so is essential... this means that *either* this machine hasn't
got one (it is possible, and common, to build perl without libperl.so:
everything that would have been in the lib is linked directly into
/usr/bin/perl) in which case you'll either have to rebuild perl with one
or you'll need the appropriate perl build tree for that machine, so it
can be statically linked; *or* that it is somewhere other than where the
program is looking for it (programs (can) have a search path for runtime
libraries built into them in addition to those in /etc/ld.so.conf).

cperltest: /lib/i686/libc.so.6: version `GLIBC_2.3' not found (required
by cperltest)

This is basically an insurmountable problem: the binary you have is
simply not compatible with that version of glibc.

QUESTIONS:

I have a basic understanding of what shared libraries are and how they
work, but I am not knowledgable about GLIBC compatibilities, or about
the "best" way to attain the following goal: I want someone to be able
to write a C/C++ program that links against my Cperl library, and as far
as they are concerned, it will just work.

1. Is this possible?

Yes. Your lib (.a or .so, it doesn't matter) will have to be either
compatible with the installed libperl.so, or statically linked to its
own copy of libperl.a.

2. Could someone please give me a short primer on the way that GCC build
env, user environment (e.g. LD_LIBRARY_PATH), and Perl environment (how
Perl was built, etc), all influence what the "right" answer is to
solving this problem? All these issues have me confused!

If your perl is statically linked (no libperl.so) then so will your
libcperl.so need to be. This is wasteful (the libperl.so code is
duplicated), but can be easier. OTOH, there may then be issues with
@INC, depending on the differences between the installed perl and the
one built into you lib. You may be able to deal with these by giving
your library (or your program) its own set of perl modules, and setting
@INC inside your program.

Basically, it all depends on how perl was built: your library and any
programs that use it all have to be built in the same way as the copy of
perl your library uses (which may well not be the same as /usr/bin/perl
on that machine).

3. What approach should I take? The ground rules are that all the boxes
I am working with will all have basically same O/S -- all Red Hat boxes
that look pretty much like

Linux 2.4.20-19.7smp #1 SMP Tue Jul 15 13:34:04 EDT 2003 i686 unknown

but they may have slightly different Perl versions installed in slightly
different places. SHould I compile everything statically so that my
cperl.a essentially contains a stand-alone Perl? How would I go about
doing this if it's possible?

4. How do you figure out what shared libraries an executable depends on?

Use ldd: you can do the same for shared libraries. This will tell you
which libs it needs and where the runtime linker will actually resolve
them to (if it can). You can use readelf -d to find out if an executable
specifies its own path for resolving shared libraries (entries of type
RPATH).

5. How do you statically compile an executable (gcc version 3.2.3) so
that it depends on NOTHING?

-static. In your case, you would first need to build a static perl (pass
the appropriate option to Configure). Then build your library against
that copy of perl, and it will statically link in libperl.a.

6. Should I even try to compile statically, or can I somehow keep things
the way they are, and just have everyone add a few things to their
LD_LIBRARY_PATH on my behalf? Any philosophical and pratical help would
be great...

I would be inclined not to compile things statically where you can avoid
it. By far the best answer, as I said, is to build the thing on the
machine it will run on. Otherwise you might try building a libcperl.so
with a statically linked perl; but you will still have problems with
differing libc versions.

---

Have you considered turning your problem inside-out, and writing the
main program in Perl, with XS or Inline modules written in C or C++
where you need to? You may have more luck that way.

Ben

 

[Steve Titus]

Ben,

Thanks very much for your help. Your post helped me out a bunch. I decided
to go with the following approach: I downloaded and built a version of Perl
(5.8.4) statically for use with my project, so that my library would always
be using a consistent version of Perl. When I built the executable, I linked
against this perl. This solves all issues except the GLIBC issue; for that,
I will just build two different versions of executable (for use on the 2
common glibc versions on boxes I am using). The static linking is
regrettable, but allows use of a consistent Perl version, and use of app in
cases when Perl is not installed on box, which is nice.



Steve