UnsatisfiedLinkError spuriously thrown

M

mtanquary

I am working on an Axis project. This project is using Axis2 under
Tomcat5, Java is 1.5.

I built a web service and all went fine. Then I added some error
handling and some additional utility functions...nothing complex.

Now, when I call the web service, Axis throws the following exception:

Exception in thread "-4db4b134:110e45e880c:-7fff"
java.lang.UnsatisfiedLinkError: create
at com.firstlogic.rapid.ace.AceJobApp.create(Native Method)
at com.firstlogic.rapid.ace.AceJobApp.<init>( AceJobApp.java:
26)
at com.amazingmail.cass.rt.provider.RT_AceCleanse
$AceApp.<init>(RT_AceCleanse.java:564)


Amazingly, this link error does not occur in a previous version of the
JAVA code from the same project which is only slight less complicated
than the current (ie. I have no idea why it should affect anything at
the native level).

Also, I created the code with a Main() method to test the java and it
works!

I did run nm on the .so file and the
Java_com_firstlogic_rapid_ace_AceJobApp_create symbol is there, which
I expected, since it worked before.

I have carefully examined my deployment of the code to make sure
everything is where it should be. The CLASSPATH and LIB PATH all
contain paths that are required for this to work.

This is thrown in the super() method of the AceJobApp class when it is
instantiated.

I have been reading posts and getting ideas for several days, but
nothing is working. Because it works outside of Axis, I am tempted to
think something is going on in Axis. Yet, as I stated earlier,
previous incantations of the same code (inside Axis) work just fine.

I suspect that somehow there is some level of corruption going on,
although peppering trace messages in the code is working, so it seems
healthy up to the point where the native code comes in.

I just wonder if somehow the symbols are getting buried or mangled.

The native code is commercial, so I have no access to it, nor do I
believe that there is really an issue with the native code itself as I
have used it in many other projects over the years.

I need some help thinking out of the box here. Any and all ideas are
greatly appreciated.

Here are snippets from working and then breaking code to give you an
idea of what the code looks like.

<-----Original Working Code------->
....
if (p != null) {
//Get ID
sId = uid.toString();
//Return thread identification
RT_AceCleanse AC =
new RT_AceCleanse(getProperties("rtAce.properties"),
addresses);
//Get a new thread
Thread t = new Thread(AC, sId);
//Creat a new message handler
threadHash.put(sId, new RT_Message());
//Start the thread
t.start();
//Return the process ID to the caller
}

{t.start() invokes the following}

public void run() {
try {
if (getProperties()) {
doBatch();
} else {
System.err.println(getDateStamp() + " - Thread " +
Thread.currentThread().getName() + " unable to set program
variables.");
}
.....

{and finally, here's doBatch}

private boolean doBatch() {
// Return code
boolean bRetVal = true;

// Open the main log
openLog(MAIN_LOG);

// Log the start of the processing
writeLog(MAIN_LOG, "ACE started");

ProgDb inDb = null;

try {
// Open the log
openLog(ACE_LOG);

// Create the firstlogic applications to process the data
AceApp aceApp = new AceApp(m_aceJobFile); {mtanquary: at
this point the new code fails}



<-----Non-working Code (in Axis, works in Main())------->
if (p != null) {
//Get a class instance
RT_AceCleanse AC =
new RT_AceCleanse(p, addresses);
//Get a new thread
Thread t = new Thread(AC, sId);
//Start the thread
t.start();
//Return the process ID to the caller
RT_Message mssg = new RT_Message();
setThreadStatus(sId, mssg.T_INITIATED);
}

.....
{t.start() invokes}

public void run() {
try {
//Set the Class State variables to get started
if (getProperties()) {
//Assuming all went well, process
m_Msg.setThreadStatus(m_Msg.T_STARTED);
if(doBatch() == false) {
m_Msg.setThreadStatus(m_Msg.T_FAILED);
}

{doBatch - identical to previous version}
 
G

Gordon Beaton

I built a web service and all went fine. Then I added some error
handling and some additional utility functions...nothing complex.
[...]

Amazingly, this link error does not occur in a previous version of
the JAVA code from the same project which is only slight less
complicated than the current (ie. I have no idea why it should
affect anything at the native level).
[...]

I need some help thinking out of the box here. Any and all ideas are
greatly appreciated.

Are multiple classloaders being used? Perhaps by Tomcat?

If so, class instances loaded in one classloader will be unable to see
the native library loaded in another. So even though you may see that
System.loadLibrary() and some native calls have succeeded, others may
fail.

Has a package declaration changed anywhere? You can see from the
native symbol name that the package name is encoded as part of it. If
there are different versions of the code that belong to different
packages, then the JVM will fail to find the correct symbols.

Are multiple versions of the native library and/or the corresponding
Java classes visible somewhere? Make sure there aren't.
I have carefully examined my deployment of the code to make sure
everything is where it should be. The CLASSPATH and LIB PATH all
contain paths that are required for this to work.

What is "LIB PATH", exactly? Do you mean LD_LIBRARY_PATH? Has it been
exported?

Does the native library have runtime depencies on other libraries? Are
you sure they are being resolved correctly? What does ldd say?

/gordon
 
M

mtanquary

I built a web service and all went fine. Then I added some error
handling and some additional utility functions...nothing complex.
[...]

Amazingly, this link error does not occur in a previous version of
the JAVA code from the same project which is only slight less
complicated than the current (ie. I have no idea why it should
affect anything at the native level).
[...]

I need some help thinking out of the box here. Any and all ideas are
greatly appreciated.

Are multiple classloaders being used? Perhaps by Tomcat?

If there are, I'm not sure how to find out. This seems to be a
possibility but I don't know how to tell. I started tomcat with -
verbose:jni and I can see that none of my symbols are being
Dynamically loaded as expected. I tried both System.load and
System.loadLibrary; both did not error out. Exception occurs when a
call is actually made to the native lib.
If so, class instances loaded in one classloader will be unable to see
the native library loaded in another. So even though you may see that
System.loadLibrary() and some native calls have succeeded, others may
fail.

Has a package declaration changed anywhere? You can see from the
native symbol name that the package name is encoded as part of it. If
there are different versions of the code that belong to different
packages, then the JVM will fail to find the correct symbols.
No.

Are multiple versions of the native library and/or the corresponding
Java classes visible somewhere? Make sure there aren't.

I ran a find on the system, only one version exists.
What is "LIB PATH", exactly? Do you mean LD_LIBRARY_PATH? Has it been
exported?
Yes, LD_LIBRARY_PATH and also output of java.lib.path.
Does the native library have runtime depencies on other libraries? Are
you sure they are being resolved correctly? What does ldd say?

ldd's output looks normal. all dependencies exist. ldd can find the
dependencies though most aren't in LD_LIBRARY_PATH (like /lib, which I
assume is a system default). The other commercial shared objects are
in a path that is listed in LD_LIBRARY_PATH.
 
G

Gordon Beaton


Along the same lines: have any methods been added or removed from the
class to which the native method belongs? If a method is overloaded,
the native symbols will be different than if the method name is
unique. So adding or removing a method can affect pre-existing or
remaining native methods.

And with that, I'm pretty much out of ideas.

/gordon
 
M

mtanquary

[SOLUTION]
Your question of whether Multiple Classloaders are being used was
right on. I had suspected that but at the time was unsure how to
handle it. I discovered the answer in the Tomcat Release notes. I
created a separate class who's sole purpose is to create static
linkage which resides outside of the Web Container. I had to do some
other code rearranging to pull it off, but that was the basic
solution. The notes are below for anyone else who experiences this. My
lesson learned....read the release notes.

Thanks for your help on this!!!!

-------------------------------------
Tomcat 5.0 and JNI Based Applications:
-------------------------------------

Applications that require native libraries must ensure that the
libraries have
been loaded prior to use. Typically, this is done with a call like:

static {
System.loadLibrary("path-to-library-file");
}

in some class. However, the application must also ensure that the
library is
not loaded more than once. If the above code were placed in a class
inside
the web application (i.e. under /WEB-INF/classes or /WEB-INF/lib), and
the
application were reloaded, the loadLibrary() call would be attempted a
second
time.

To avoid this problem, place classes that load native libraries
outside of the
web application, and ensure that the loadLibrary() call is executed
only once
during the lifetime of a particular JVM.


Along the same lines: have any methods been added or removed from the
class to which the native method belongs? If a method is overloaded,
the native symbols will be different than if the method name is
unique. So adding or removing a method can affect pre-existing or
remaining native methods.

And with that, I'm pretty much out of ideas.

/gordon

Your ideas and response was tremendous. I
 
G

Gordon Beaton

[SOLUTION]
Your question of whether Multiple Classloaders are being used was
right on. I had suspected that but at the time was unsure how to
handle it. I discovered the answer in the Tomcat Release notes.

I'd forgotten about this, but I'm glad you solved it (and I'm pleased
that my first suggestion was the right one).

/gordon
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top