Python,SWIG and libjvm

[sapsi]

Hello,
I'm not sure if this the correct list but here goes (and sorry for the
noise). I've been attempting to wrap python around libhdfs.
So far so good (i've attached the SWIG template at the end). The
compilation works without errors and the shared objects do have
references to all the functions.

However, when importing into python

import pyhdfs

i get the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/home/sguha/tmp/b/hdfs.py", line 7, in ?
import _hdfs
ImportError: libjvm.so: cannot open shared object file: No such file
or directory

However, libjvm.so is located in /home/sguha/mine/jdk1.6.0_02/jre/lib/
amd64/server which is present in the PYTHONPATH and sys.path.
I can understand it complaining <i>while</i> loading the libjvm.so
but python doesn't locate it even when in the path.

As an aside, the build command for hdfs.c has " -ljvm " - this
should affect SWIG, right? Meaning if my program (e.g) links to a
library and i want a wrapper around my program (to python) i need SWIG
templates for my headers only and not for the linked library...right?
( I guess the answer is i dont otherwise that would be quite a
nightmare of work)

Thank you for your time
Saptarshi


Attachments:
SWIG template
%module pyhdfs

%{
#include "hdfs.h"
%}

%include "hdfs.h"

 

[sapsi]

Hello,
I'm not sure if this the correct list but here goes (and sorry for the
noise). I've been attempting to wrap python around libhdfs.
So far so good (

I should point out that libhdfs is a c library to Hadoop Distributed
FileSystem.
Thank you
Saptarshi

 

[sapsi]

I should point out that libhdfs is a c library to Hadoop Distributed
FileSystem.
Thank you
Saptarshi

Hmm,here goes, you need -Wl,--rpart -Wl,path-to-jvm in the LDFLAGS in
the compile process.
HTH
Saptarshi

 

[Graham Dumpleton]

Hello,
I'm not sure if this the correct list but here goes (and sorry for the
noise). I've been attempting to wrap python around libhdfs.
So far so good (i've attached the SWIG template at the end). The
compilation works without errors and the shared objects do have
references to all the functions.

However, when importing into python

import pyhdfs

i get the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/home/sguha/tmp/b/hdfs.py", line 7, in ?
import _hdfs
ImportError: libjvm.so: cannot open shared object file: No such file
or directory

However, libjvm.so is located in /home/sguha/mine/jdk1.6.0_02/jre/lib/
amd64/server which is present in the PYTHONPATH and sys.path.
I can understand it complaining <i>while</i> loading the libjvm.so
but python doesn't locate it even when in the path.

As an aside, the build command for hdfs.c has " -ljvm " - this
should affect SWIG, right? Meaning if my program (e.g) links to a
library and i want a wrapper around my program (to python) i need SWIG
templates for my headers only and not for the linked library...right?
( I guess the answer is i dont otherwise that would be quite a
nightmare of work)

Thank you for your time
Saptarshi

Attachments:
SWIG template
%module pyhdfs

%{
#include "hdfs.h"
%}

%include "hdfs.h"

Your libjvm.so isn't being found because it isn't in a standard system
library directory. You have a few choices to solve this:

1. Install libjvm.so into /lib or /usr/lib. You may be able to get
away with putting it in /usr/local/lib if your OS is setup to look
there.

2. If a Linux/Solaris box, do:

LD_LIBRARY_PATH=/home/sguha/mine/jdk1.6.0_02/jre/lib/amd64/server
export LD_LIBRARY_PATH

then run your application. This will tell you application to look in
that directory for the shared library at run time.

3. If on Linux/Solaris, compile that library directory into the Python
module .so by setting before you compile your Python module:

LD_RUN_PATH=/home/sguha/mine/jdk1.6.0_02/jre/lib/amd64/server
export LD_RUN_PATH

This only needs to be done at compile time, but means your module will
always expect it to be in that location.

Graham