StackOverflow Exception in JNI

[Bryan Castillo]

I'm getting a StackOverflow exception while using JNI.

I have a loop written in C/JNI which calls back a Java object (which
conforms to some interface). The loop is creating 2 byte arrays and
giving them to the object being called back. The arrays are being
created with the function NewByteArray. For some reason around 50,000
items into the iteration, I run into a StackOverflow exception. I'm
not actually doing anything with the byte arrays once passing to the
callback. (In the JNI code the byte arrays are not returned). Should I
be deleting them using DeleteLocalRef? (I'll try tomorrow - forgot to
bring the code home with me).

[side question]
Is the array returned from NewByteArray a local reference or global?
I thought it was a local reference. But if that is true should you
use NewGlobalRef before returning the byte array back to java code?
Currently, im just returning the jbyteArray and everything seems to be
working so far.

thanks.

 

[Gordon Beaton]

I'm getting a StackOverflow exception while using JNI.

I have a loop written in C/JNI which calls back a Java object (which
conforms to some interface). The loop is creating 2 byte arrays and
giving them to the object being called back. The arrays are being
created with the function NewByteArray. For some reason around
50,000 items into the iteration, I run into a StackOverflow
exception. I'm not actually doing anything with the byte arrays once
passing to the callback. (In the JNI code the byte arrays are not
returned). Should I be deleting them using DeleteLocalRef? (I'll try
tomorrow - forgot to bring the code home with me).

Without seeing your code, it's impossible to say what is causing your
exception. Presumably there is a recursive call somewhere.

In most normal situations, you don't need to explicitly free any
objects in your native code, that will be taken care of automatically
when the method returns. The regular reachability rules apply and the
objects become eligible for garbage collection at the end of the
method, while objects returned to java or still referenced by other
live objects will continue to be valid. In other words, you can create
temporary objects in your native method and then return a valid
reference without thinking too much about these issues.

If you create many temporary objects in the native method without an
intervening return, the objects will accumulate and you can cause an
out of memory situation. A typical scenario is a (native) loop where a
handful of objects are created on each pass and only used during that
specific pass of the loop.

To avoid that situation you can call DeleteLocalRef() on each
temporary reference at the end of every pass, or define a local stack
frame for the loop by enclosing the loop body in calls to
PushLocalFrame() and PopLocalFrame(). PopLocalFrame() is essentially
the same as calling DeleteLocalRef() on each reference created within
the frame, except for one that you can tell it to return to the
enclosing frame.

[side question]
Is the array returned from NewByteArray a local reference or global?
I thought it was a local reference. But if that is true should you
use NewGlobalRef before returning the byte array back to java code?
Currently, im just returning the jbyteArray and everything seems to
be working so far.

All references you create are local until you explicitly create a
global reference with NewGlobalRef().

However you only need to use NewGlobalRef() if you want to save a
reference across multiple calls to the native code (there may be other
situations but I can't think of any off hand). If you use
NewGlobalRef() on a return value, that object will not become eligible
for garbage collection until you explicitly free it.

You don't need to create a global reference for values you return back
to Java, or pass to other methods invoked from the native method.

/gordon

 

[Bryan Castillo]

Without seeing your code, it's impossible to say what is causing your
exception. Presumably there is a recursive call somewhere.

Here is some sample code that produced the same results for me. The
actual
code would have been hard to post. The code is not recursive. The
StackOverflow exception was throw by IBM's 1.4.1 R1 JDK running on RH
Linux 3, while Sun's JDK 1.4.2 had a segmentation fault. (hmm.....)

JNIEXPORT void JNICALL Java_JNITest_callCallback__LJNICallback2_2I
(JNIEnv * jenv, jclass jclazz, jobject callback, jint times)
{
jmethodID cb_mid;
jclass cb_class;
int i;
int result;
int length = 1000;
jbyteArray b1;
jbyteArray b2;

cb_class = (*jenv)->FindClass(jenv, "JNICallback2");
if (cb_class == NULL) {
return;
}
cb_mid = (*jenv)->GetMethodID(jenv, cb_class, "call", "([B[BI)I"
);
if (cb_mid == NULL) {
fprintf(stderr, "couldn't find method");
return;
}

for (i=0; i<times; i++) {
b1 = (*jenv)->NewByteArray(jenv, length);
b2 = (*jenv)->NewByteArray(jenv, length);
result = (*jenv)->CallIntMethod(jenv, callback, cb_mid, b1,
b2, i);
if ((*jenv)->ExceptionOccurred(jenv) != NULL) { return; }
//DeleteLocalRef fixes the problem
//if (b1 != NULL) { (*jenv)->DeleteLocalRef(jenv, b1); }
//if (b2 != NULL) { (*jenv)->DeleteLocalRef(jenv, b2); }
if (result != 1) { break; }
}
}

In most normal situations, you don't need to explicitly free any
objects in your native code, that will be taken care of automatically
when the method returns. The regular reachability rules apply and the
objects become eligible for garbage collection at the end of the
method, while objects returned to java or still referenced by other
live objects will continue to be valid. In other words, you can create
temporary objects in your native method and then return a valid
reference without thinking too much about these issues.

If you create many temporary objects in the native method without an
intervening return, the objects will accumulate and you can cause an
out of memory situation. A typical scenario is a (native) loop where a
handful of objects are created on each pass and only used during that
specific pass of the loop.

This is exactly what I was doing, so the DeleteLocalRef cleared it up.

 

[Gordon Beaton]

Here is some sample code that produced the same results for me. The
actual code would have been hard to post. The code is not recursive.
The StackOverflow exception was throw by IBM's 1.4.1 R1 JDK running
on RH Linux 3, while Sun's JDK 1.4.2 had a segmentation fault.
(hmm.....)

I can't see any issues with the native method you've posted.

Although you've said that there is no recursion involved, I wonder
what happens in the Java method you invoke? Is there no possibility
that this method gets called again from there, perhaps indirectly?

What happens if you invoke System.gc() at the end of call(), i.e. just
before returning from the Java method?

/gordon

 

[Liz]

how about if you run it with the debugger
just say 'run' and let it go until it dies
you might be able to see what went wrong by
dumping various things

 

[Bryan Castillo]

I can't see any issues with the native method you've posted.

Although you've said that there is no recursion involved, I wonder
what happens in the Java method you invoke? Is there no possibility
that this method gets called again from there, perhaps indirectly?

I get the same results when the java method just returns. The
System.gc does not help. As long as I call DeleteLocalRef in the JNI
its OK.

Here is the source to the full test for this.


// File: JNICallback.java
public interface JNICallback {
public int call(byte [] b1, byte [] b2, int count);
}

// File: JNITest.c
#include "JNITest.h"

JNIEXPORT void JNICALL Java_JNITest_callCallback
(JNIEnv * jenv, jclass jclazz, jobject callback, jint times)
{
jmethodID cb_mid;
jclass cb_class;
int i, result;
int length = 1000;
jbyteArray b1, b2;

cb_class = (*jenv)->FindClass(jenv, "JNICallback");
if (cb_class == NULL) { return; }
cb_mid = (*jenv)->GetMethodID(jenv, cb_class, "call", "([B[BI)I" );
if (cb_mid == NULL) { return; }

for (i=0; i<times; i++) {
b1 = (*jenv)->NewByteArray(jenv, length);
b2 = (*jenv)->NewByteArray(jenv, length);
result = (*jenv)->CallIntMethod(jenv, callback, cb_mid, b1, b2,
i);
if ((*jenv)->ExceptionOccurred(jenv) != NULL) { return; }


// uncomment this and the code will work
//if (b1 != NULL) { (*jenv)->DeleteLocalRef(jenv, b1); b1 = NULL;
}
//if (b2 != NULL) { (*jenv)->DeleteLocalRef(jenv, b2); b2 = NULL;
}


if (result != 1) { break; }
}
}


// File: JNITest.java
public class JNITest {
static { System.loadLibrary("JNITest"); }
public static native void callCallback(JNICallback callback, int
times);
}

// File: Main.java
public class Main {
public static void main(String [] args) {
try {
JNICallback cb = new JNICallback() {
public int call(byte [] b1, byte [] b2, int count) {
if ((count % 10000) == 0) {
System.err.println("Count: " + count);
}
return 1;
}
};
JNITest.callCallback(cb, 1000000);
}
catch (Exception e) {
e.printStackTrace();
}
}
}

# File: Makefile
JAVA_HOME=/usr/local/jvm/IBMJava2-141SR1
#JAVA_HOME=/home/maple/bcastill/j2sdk1.4.2_04

JNIARCH = linux
CC = gcc
JAVAC = $(JAVA_HOME)/bin/javac
JAVAH = $(JAVA_HOME)/bin/javah
JAVA = $(JAVA_HOME)/bin/java
CINCS = -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/$(JNIARCH)
CFLAGS = -Wall $(CINCS)

all: libJNITest.so

libJNITest.so: JNITest.c JNITest.h Makefile
$(CC) -shared -o libJNITest.so $(CFLAGS) JNITest.c

JNITest.h: JNITest.class
$(JAVAH) -jni JNITest

JNITest.class: JNICallback.java JNITest.java Main.java
$(JAVAC) *.java

retest: clean test

test: libJNITest.so
$(JAVA) -classpath . -Djava.library.path=. Main

clean:
-rm -rf *.class JNITest.h *.so *.o *~




Here is the output of it

/home/maple/bcastill/jnitest/t]
$ make test
/usr/local/jvm/IBMJava2-141SR1/bin/javac *.java
/usr/local/jvm/IBMJava2-141SR1/bin/javah -jni JNITest
gcc -shared -o libJNITest.so -Wall
-I/usr/local/jvm/IBMJava2-141SR1/include
-I/usr/local/jvm/IBMJava2-141SR1/include/linux JNITest.c
/usr/local/jvm/IBMJava2-141SR1/bin/java -classpath .
-Djava.library.path=. Main
Count: 0
Count: 10000
Count: 20000
Count: 30000
Count: 40000
Count: 50000
Exception in thread "main" java.lang.StackOverflowError
at JNITest.callCallback(Native Method)
at Main.main(Main.java:12)
make: *** [test] Error 1

 

[Gordon Beaton]

I get the same results when the java method just returns. The
System.gc does not help. As long as I call DeleteLocalRef in the JNI
its OK.

Ok, I misunderstood earlier, and didn't realize that you had solved
the problem with DeleteLocalRef().

BTW you should add -D_REENTRANT to the command line here:

libJNITest.so: JNITest.c JNITest.h Makefile
$(CC) -shared -o libJNITest.so $(CFLAGS) JNITest.c

/gordon