Embeded Perl, Dynaloader, relocation error: undefined symbol: Perl_Tstack_sp_ptr

K

kyle.burton

Summary: error using dynamically loaded modules from an embedded Perl
Interpreter.

Sorry for the lengthy set up, I'm just trying to communicate all of
the things that we've explored so far. If you want to ignore the
background jump forward to the example Java code.


Background:

We have embarked on a project to integrate a large Perl application
that we've developed in house [1] with some Java applications so they
can take advantage of what Perl and the application do best.

For these Java applications, Java has to be in the driver's seat, so
we're looking at techniques that involve embedding Perl into the JVM.
Toward this end I've looked (briefly) at PLJava [2], JPL [3] and
Inline::Java [4]. Each of these solutions comes close to what we want
and if we can't get exactly what we want we might fall back to one of
them.

Based on the achievements those projects have made and some
experimentation of our own it is apparent that embedding at least a
single static Interpreter (one for the entire JVM process) into the
JVM is possible. Ideally we'd like to be able to have multiple
independent Perl Interpreters within the JVM. Our distributed Perl
application supports multiple concurrent connections and distributes
requests and workload across a cluster - having the Java applications
be able to perform simultaneous requests into the Perl application is
a goal.

After reading though the various Perl man-pages (perlembed, perlapi,
etc.) I believe that it is possible to have multiple Interpreters per
process. It looks like it is not possible to have multiple
Interpreters per thread, or multiple threads running through an
Interpreter without synchronization (the interpreters, or core Perl
functions are not reentrant?). This doesn't yet make the dream
impossible since since it looks like Perl supports one Interpreter per
thread (dTHX) explicitly.

So the model we're attempting to implement is library code on the Java
side that manages a pool of Perl Interpreters. The Java-side library
will manage making sure the required level of synchronization takes
place so that an interpreter is not shared by two threads in the JVM.


What we've done so far is to create a thin C library that wraps
libperl that we've projected into Java via SWIG. That has made it
possible for us to run the following Java (pseudo-)code:

public static void runTest() throws Exception {
System.loadLibrary("TahitiPerl");
PerlInterpreter interp = PerlInterpreter.newInstance();
PoolingAllocator pool = PoolingAllocator.newInstance(interp);
pool.eval(
"my $i;\n"
+ "for ($i = 0; $i < 10; ++$i ) {\n"
+ " print STDERR \"i=$i\\n\";\n"
+ "}\n"
+ "$i;\n"
);

SV myi = SV.newInstance(pool);
System.out.println("myi="+myi);
System.out.println("setting myi=3");
myi.set(3);
dumpSv(myi);

System.out.println("setting myi=3.14159");
myi.set(3.14159);
dumpSv(myi);

String code = null;
SV result = null;
AV results = null;
String [] args = null;

code = "2*2";
result = pool.evalCtxSV(code);
System.out.println(code+"="+result.pv());

code = "print STDERR \"Perl: about to create tahitiTest1\\n\""; //
[5]
System.out.println("evalCtxVoid("+code+")");
pool.evalCtxVoid(code);

code = "sub tahitiTest1 {\n"
+ " print STDERR \"TahitiTest1: (@_)\\n\";\n"
+ " return wantarray ? ('len',scalar @_) : 'len:'.scalar
@_;\n"
+ "}\n";
args = new String[]{"this","that","other"};
System.out.println("creating sub tahitiTest1 on pool:\n" + code);
pool.evalCtxVoid(code);
System.out.println("calling sub tahitiTest1...(should print) in
void context");
pool.callCtxVoid("tahitiTest1",args);

System.out.println("calling sub tahitiTest1...(should print) in
scalar context");
result = pool.callCtxSV("tahitiTest1",args);
System.out.println("Result was: " + result.pv() );

System.out.println("calling sub tahitiTest1...(should print) in
array context");
results = pool.callCtxAV("tahitiTest1",args);
System.out.println("results.length=" + results.length());
System.out.println("results.fetch(0)=" + results.fetch(0).pv());
System.out.println("Results were: " + results.join(",") );

pool.drain();
interp.destroy();
}

public static void dumpSv(SV sv) throws Exception {
System.out.println("sv.curr="+sv.cur());
System.out.println("sv.iv="+sv.iv());
System.out.println("sv.nv="+sv.nv());
System.out.println("sv.pv="+sv.pv());
}

The experimental implementation also supports instantiating multiple
interpreters in the JVM though we haven't gotten around to doing
concurrency testing yet.

Once we made it that far we got very excited and the next thing we had
to try was calling through to LWP::UserAgent->get. Something like:

System.out.println("creating new LWP::UserAgent...");
SV ua = pool.newClass("LWP::UserAgent");
System.out.println("created new ua="+ua.pv());
String uri = "http://www.google.com/";
SV result = pool.callMethodCtxSV(ua,"get",new String[]{uri});
System.out.println(" called ua.get, result="+result.pv());
SV content = pool.callMethodCtxSV(result,"content");
System.out.println(" result.message=: " +
pool.callMethodCtxSV(result,"message").pv() );
System.out.println(" result.decoded_content=: " +
pool.callMethodCtxSV(result,"decoded_content").pv() );
System.out.println(" result.content=: " + content.pv() );


And that's where we hit our first snag. Calling ua->get failed. We
figured out that it was actually failing when trying to load the .so
for Socket.pm (we changed the code to just try 'use Socket;'). We had
the following error:

src/c/TahitiPerl.c(364) TahitiPerl_evalCtxVoid: evaled code,
dollarAt=0x8144944 : \
'Can't load module Socket, dynamic loading not available in this
perl.
(You may need to build a new perl executable which either supports
dynamic loading or has the Socket module statically linked into
it.)
at (eval 1) line 1
Compilation failed in require at (eval 1) line 1.
BEGIN failed--compilation aborted at (eval 1) line 1.
'

After some more reading (manpages and other projects) the next thing
we tried was linking DynaLoader.a, and installing the following
xs_init:

EXTERN_C void
xs_init(pTHX) {
const char *file = __FILE__;
/* DynaLoader is a special case */
dXSUB_SYS;
newXS("DynaLoader::boot_DynaLoader", boot_DynaLoader, file);
}

That resulted in a relocation error:

src/c/TahitiPerl.c(357) TahitiPerl_evalCtxVoid: evaling code: 'use
Socket;'
/home/mortis/local/java/SUNWappserver/jdk/bin/java: relocation error:
/usr/lib/perl/5.8/auto/Socket/Socket.so: undefined symbol:
Perl_Tstack_sp_ptr
make: *** [jtest] Error 127

I couldn't find much information on Google regarding relocation errors.
Grasping at straws (probably doing things that don't make sense), we
also tried linking in libperl.a. That resulted in the same error.

At this point, given my limited experience, I'm stuck and am seeking
the collective wisdom of the larger community. Does anyone know if
what we're trying to do is even tractable? Is the relocation error
solvable so that DynaLoader can be used from this approach?

Something else we're considering exploring is statically linking in
the XS modules that we need (into either the libperl.so or the
libTahitiPerl.so) to communicate with our Perl application (there are
at most a handful, and Socket.so may be the only one).

Also, we have received permission from our employer to open-source
the project if and when we get it working. I've also received
permission to put up the experimental code so that it can be looked at
and experimented with.

Notes on how to build are below, as well as information about Perl,
Java, etc.


Thank you for your time.

Kyle R. Burton


I've put up a snapshot of the code here:

http://www.neverlight.com/~mortis/TahitiPerl/

To build and run the code you'll need an installed JVM. You will need
to copy common.mk.example to common.mk and customize it for your
system (sorry, no configure script yet). The build requires
libperl.so with multiplicity enabled. We have built and tested so far
using gcc 3.3.5, Java 1.4.2_07, Perl v5.8.4, under Linux (Ubuntu, and
Mandrake).

Once that has been completed you should be able to run make jtest to
exercise the bug as described in the 2nd case (linking DynaLoader and
libperl.so). I've included the output from SWIG so that SWIG is not a
build dependency.

The example Java code is in the project root in the file Test.java.
The C code is in src/c, and Java wrapper code is under src/java.




[1] An inadequate, but sufficient for now, description of the
application is: a distributed network based data warehousing system.

[2] PLJava uses a single static Perl Interpreter. For concurrency and
performance (since the Perl application already handles multiple
simultaneous clients) we'd like to be able to have 1 Perl Interpreter
per thread in the JVM.

[3] I could be incorrect about this, but I think that JPL is
undesirable for us for a few reasons: it requires us to build a custom
Perl; it does not compile (without patches) in current versions of
Perl; it is no longer supported.

[4] Inline::Java has experimental support for embedded Perl
Interpreters via Inline::Java::perlInterpreter, but it documents it as
experimental and it documents it as a singleton (we're hoping to be
able to have an Interpreter per thread in the JVM).

[5] TahitiPerl was chosen as the name for a few reasons: we use
islands for our internal project names, black Tahiti pearls are
considered rare and valuable, we are embedding Perl -- something rare
and valuable.


Output from perl -V:

Summary of my perl5 (revision 5 version 8 subversion 4)
configuration:
Platform:
osname=linux, osvers=2.6.8.1, archname=i386-linux-thread-multi
uname='linux mcmurdo 2.6.8.1 #1 smp sat feb 26 13:03:40 utc 2005
i686 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.
8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr
-Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5
-Dsiteprefix=/usr/local -Dsitelib=/usr/loca
l/share/perl/5.8.4 -Dsitearch=/usr/local/lib/perl/5.8.4
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm
-Duseshrplib -Dlibperl=l
ibperl.so.5.8.4 -Dd_dosuid -des'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBIAN -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE
_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
-fno-strict-aliasing -I/usr/local/include'
ccversion='', gccversion='3.3.5 (Debian 1:3.3.5-8ubuntu2)',
gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
perllibs=-ldl -lm -lpthread -lc -lcrypt
libc=/lib/libc-2.3.2.so, so=so, useshrplib=true,
libperl=libperl.so.5.8.4
gnulibc_version='2.3.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES
PERL_IMPLICIT_CONTEXT
Built under linux
Compiled at Mar 10 2005 13:38:00
@INC:
/etc/perl
/usr/local/lib/perl/5.8.4
/usr/local/share/perl/5.8.4
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.8
/usr/share/perl/5.8
/usr/local/lib/site_perl
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,479
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top