JNI - Passing and returning complex values

[adityar7]

I'm a C programmer with limited Java experience, and need to use JNI
to implement a library in Java on top of a C library that consists of
a bunch of functions, each receiving some parameters from the
operating system. For each C function, the corresponding Java method
will be called, and some values would be returned back to the C
function. i.e. C -> Java -> C.

I know how to deal with simple primitive types, but I need to pass and
return multiple fields which could be primitive types or C structs. I
understand that to pass C structs I would need to define a
corresponding Java class and copy over the struct fields.

The question, though, is how to deal with multiple parameters/return
values of different types. Here is a sample C function specification:

int getattr (const char *, struct stat *)

I have approximately 35 such functions with differing number and types
of parameters. The corresponding Java method needs to be passed both
these parameters, and it also needs to return both of them! Since I'm
new to JNI the only way I can think of is to define 35 Java classes,
each corresponding to a particular set of parameters. Hopefully there
is a better way to do it!

Any help would be greatly appreciated

 

[Arne Vajhøj]

I'm a C programmer with limited Java experience, and need to use JNI
to implement a library in Java on top of a C library that consists of
a bunch of functions, each receiving some parameters from the
operating system. For each C function, the corresponding Java method
will be called, and some values would be returned back to the C
function. i.e. C -> Java -> C.

I know how to deal with simple primitive types, but I need to pass and
return multiple fields which could be primitive types or C structs. I
understand that to pass C structs I would need to define a
corresponding Java class and copy over the struct fields.

The question, though, is how to deal with multiple parameters/return
values of different types. Here is a sample C function specification:

int getattr (const char *, struct stat *)

I have approximately 35 such functions with differing number and types
of parameters. The corresponding Java method needs to be passed both
these parameters, and it also needs to return both of them! Since I'm
new to JNI the only way I can think of is to define 35 Java classes,
each corresponding to a particular set of parameters. Hopefully there
is a better way to do it!

You can put all the native methods in one class.

But if you need 35 different classes with data, then I can not
see any way to avoid writing them.

Arne

 

[marlow.andrew]

I'm a C programmer with limited Java experience, and need to use JNI
to implement a library in Java on top of a C library

The question, though, is how to deal with multiple parameters/return
values of different types. Here is a sample C function specification:

int getattr (const char *, struct stat *)

I recommend SWIG. See www.swig.org. This is what I used to interface
to a proprietary C library for a server that we wanted to be in java.
SWIG would take the prototype above and create corresponding java
classes for where there is no natural mapping, e.g it would create a
class for stat.

Output from SWIG is in the form of two files, one is C glue to go into
a dynamic library, the other is the java class that contains all the
methods for the functions you want to call. Other java files are
created for the new classes it creates, like stat. Your java program
has to load the dynamic library first.

Regards,

Andrew Marlow

 

[Roedy Green]

I have approximately 35 such functions with differing number and types
of parameters. The corresponding Java method needs to be passed both
these parameters, and it also needs to return both of them! Since I'm
new to JNI the only way I can think of is to define 35 Java classes,
each corresponding to a particular set of parameters. Hopefully there
is a better way to do it!

This is one of the uglier parts of Java. It supports multiple inputs
to a method but only one output.

How to kludge?

one way is to return an array of double or array of objects. You to
lose typing and keyword identification of results.

Another approach is to create a class with a number of internal
variables. To use it you call setXXX to set each of the inputs, then
call calcXXX to calculate. Then call getXXX to get each of the
results. This is the most Javaesque solution.

It is considered bad form to create result classes of scattered crap
that is not logically related.

Have your method return a result, and if there is a problem throw an
exception, rather than returning a result and status as you would in
C.

Rethink your methods so that there are a small number of possible
result classes that have physical meaning.

They problem you are having is made worse if you try to write C in
Java, rather than thinking OO.

 

[marlow.andrew]

This is one of the uglier parts of Java. It supports
multiple inputs to a method but only one output.

How to kludge?

Another approach is to create a class with a number of internal
variables.

This is what SWIG does. The classes it creates has setters and
getters. You have to write a very thin C wrapper which calls the C
code you first thought of. The wrapper also manages this way of
interfacing. The C code will have to populate the structure ready for
return, your java code just puts the result into an object of the type
generated by SWIG. You use the getters to obtain the member values.

I reckon this is MUCH easier than doing raw JNI code.

-Andrew Marlow

 

[marlow.andrew]

SWIG sounds like a real find. Thanks for bringing it to our attention.

Regarding the example with the 'stat' struct, does SWIG let you turn that into
a convention-compliant type name, 'Stat'?

Yes it does. I must admit, I am suprised when java people tell me they
don't know about SWIG. I am relatively new to java and I was lukcily
to come across it quite early. I saw it used about a year ago on a C++
project where suddenly a java interface was required by certain
customers. It was quickly knocked up using SWIG. The C++ was concerned
mainly with returning structures that represented trades for
particular financial instruments. Each instrument had its own type and
several were expressed in terms of other types.

-Andrew Marlow

 

[Tom Anderson]

I'm a C programmer with limited Java experience, and need to use JNI
to implement a library in Java on top of a C library that consists of
a bunch of functions, each receiving some parameters from the
operating system. For each C function, the corresponding Java method
will be called, and some values would be returned back to the C
function. i.e. C -> Java -> C.

I know how to deal with simple primitive types, but I need to pass and
return multiple fields which could be primitive types or C structs. I
understand that to pass C structs I would need to define a
corresponding Java class and copy over the struct fields.

The question, though, is how to deal with multiple parameters/return
values of different types. Here is a sample C function specification:

int getattr (const char *, struct stat *)

I have approximately 35 such functions with differing number and types
of parameters. The corresponding Java method needs to be passed both
these parameters, and it also needs to return both of them! Since I'm
new to JNI the only way I can think of is to define 35 Java classes,
each corresponding to a particular set of parameters. Hopefully there
is a better way to do it!

Any help would be greatly appreciated

Well, firstly, are you aware that there is already a java binding of FUSE?

http://sourceforge.net/projects/fuse-j/

If you don't want to use that, or if this isn't what you're working on, to
answer your question, i'd do a straightforward conversion of the C
interface, and pass a writeable object from C to java, get java to write
into it, then copy back from the java object to the C one. Like so:


// java:

public class Stat {
public int ino ;
// etc
}

public interface Filesystem {
public int getattr(String path, Stat sb) ;
}

public class FUSEManager {
static {
System.loadLibrary("fuse") ;
}
public static native void init(Filesystem fs) ;
}

// C:

/* error checking omitted for clarity! */

#define OK 0

JavaVM *jvm ; // JVM ref, get in a setup method

jobject fs ; // ref to a Filesystem object, get in a setup method
jclass fsClass ;
jmethodID getattrID ;

jclass statClass ;
jmethodID statCtorID ;
jfieldID statInoID ;
// etc for other fields

JNIEXPORT void JNICALL Java_FUSEManager_init(JNIEnv *env, jclass class, jobject _fs) {
(*env)->GetJavaVM(env, &jvm) ;
fs = (*env)->NewGlobalRef(env, _fs) ;
jclass _fsClass = (*env)->GetObjectClass(env, fs) ;
fsClass = (*env)->NewGlobalRef(env, _fsClass) ;
getattrID = (*env)->GetMethodID(env, fsClass, "getattr", "(Ljava/lang/String;LStatI") ;
jclass _statClass = (*env)->FindClass(env, "com/mypackage/Stat") ;
statClass = (*env)->NewGlobalRef(env, _statClass) ;
statCtorID = (*env)->GetMethodID(env, statClass, "<init>", "()V") ;
statInoID = (*env)->GetFieldID(env, statClass, "ino", "I") ;
// etc for other fields
}

void copyStatIn(JNIEnv *env, jobject statObj, struct stat *stat) {
stat->st_ino = (*env)->GetIntField(env, statObj, statInoID) ;
// etc for other fields
}

// would define a copyStatOut for setattr

int getattr(const char *path, struct stat *stat) {
JNIEnv *env = NULL ;
(*jvm)->AttachCurrentThread(jvm, &env, NULL) ;
jstring pathStr = (*env)->NewStringUTF(env, path) ;
jobject statObj = (*env)->NewObject(env, statClass, ctorID) ;
/* i don't copy the struct into the object here - hope this is right */
jint ret = (*env)->CallIntMethod(env, fs, getattrID, pathStr, statObj) ;
/* don't copy back if call failed */
if (ret == OK) copyStatIn(statObj, stat) ;
/* don't think we need to release local refs explicitly before detach */
(*jvm)->DetachCurrentThread(jvm) ;
return ret ;
}

Does that make sense? This means defining one class per struct, plus one
copy-in method and one copy-out method, as needed. No need to write 35
different classes!

I've assumed that, despite what you said, the const char* parameter isn't
'returned' in any sense.

tom

 

[adityar7]

I recommend SWIG. Seewww.swig.org. This is what I used to interface
to a proprietary C library for a server that we wanted to be in java.
SWIG would take the prototype above and create corresponding java
classes for where there is no natural mapping, e.g it would create a
class for stat.

Output from SWIG is in the form of two files, one is C glue to go into
a dynamic library, the other is the java class that contains all the
methods for the functions you want to call. Other java files are
created for the new classes it creates, like stat. Your java program
has to load the dynamic library first.

Regards,

Andrew Marlow

Sounds excellent. I have seen that term (SWIG) often but not being a
regular Java programmer, didn't know what that means. I'll check it
out and see how it goes. Thanks for pointing it out!

 

[adityar7]

Well, firstly, are you aware that there is already a java binding of FUSE?

http://sourceforge.net/projects/fuse-j/

If you don't want to use that, or if this isn't what you're working on, to
answer your question, i'd do a straightforward conversion of the C
interface, and pass a writeable object from C to java, get java to write
into it, then copy back from the java object to the C one. Like so:

// java:

public class Stat {
public int ino ;
// etc

}

public interface Filesystem {
public int getattr(String path, Stat sb) ;

}

public class FUSEManager {
static {
System.loadLibrary("fuse") ;
}
public static native void init(Filesystem fs) ;

}

// C:

/* error checking omitted for clarity! */

#define OK 0

JavaVM *jvm ; // JVM ref, get in a setup method

jobject fs ; // ref to a Filesystem object, get in a setup method
jclass fsClass ;
jmethodID getattrID ;

jclass statClass ;
jmethodID statCtorID ;
jfieldID statInoID ;
// etc for other fields

JNIEXPORT void JNICALL Java_FUSEManager_init(JNIEnv *env, jclass class, jobject _fs) {
(*env)->GetJavaVM(env, &jvm) ;
fs = (*env)->NewGlobalRef(env, _fs) ;
jclass _fsClass = (*env)->GetObjectClass(env, fs) ;
fsClass = (*env)->NewGlobalRef(env, _fsClass) ;
getattrID = (*env)->GetMethodID(env, fsClass, "getattr", "(Ljava/lang/String;LStatI") ;
jclass _statClass = (*env)->FindClass(env, "com/mypackage/Stat") ;
statClass = (*env)->NewGlobalRef(env, _statClass) ;
statCtorID = (*env)->GetMethodID(env, statClass, "<init>", "()V") ;
statInoID = (*env)->GetFieldID(env, statClass, "ino", "I") ;
// etc for other fields

}

void copyStatIn(JNIEnv *env, jobject statObj, struct stat *stat) {
stat->st_ino = (*env)->GetIntField(env, statObj, statInoID) ;
// etc for other fields

}

// would define a copyStatOut for setattr

int getattr(const char *path, struct stat *stat) {
JNIEnv *env = NULL ;
(*jvm)->AttachCurrentThread(jvm, &env, NULL) ;
jstring pathStr = (*env)->NewStringUTF(env, path) ;
jobject statObj = (*env)->NewObject(env, statClass, ctorID) ;
/* i don't copy the struct into the object here - hope this is right */
jint ret = (*env)->CallIntMethod(env, fs, getattrID, pathStr, statObj) ;
/* don't copy back if call failed */
if (ret == OK) copyStatIn(statObj, stat) ;
/* don't think we need to release local refs explicitly before detach */
(*jvm)->DetachCurrentThread(jvm) ;
return ret ;

}

Does that make sense? This means defining one class per struct, plus one
copy-in method and one copy-out method, as needed. No need to write 35
different classes!

I've assumed that, despite what you said, the const char* parameter isn't
'returned' in any sense.

tom

Tom, I am indeed working with FUSE (good guess there! I didn't mention
it in my post to keep things simple). I can't use the Java bindings
for a couple of fundamental reasons. Most importantly, I need a mix of
C and Java. Your solution would certainly be better than writing
individual classes, and I will look into this before messing with
SWIG. Actually, I do need to return the const char* parameters, which
I first copy into a jstring and copy the result back into a char*
similar to what you've done for the struct.

What's unfortunate is that everything needs to be copied. In the ideal
situation the Java methods would operate directly on the memory (using
the C pointer or any other method). If anyone know if that's possible,
please let me know

Thanks for your reply!

 

[Tom Anderson]

What's unfortunate is that everything needs to be copied. In the ideal
situation the Java methods would operate directly on the memory (using
the C pointer or any other method). If anyone know if that's possible,
please let me know

Actually, it is, but it's not pretty.

Do you know about the New I/O, aka nio, package? It has things called
ByteBuffers, which are basically fancy dressed-up byte arrays. Except that
they can be 'direct', which means they're sort of separate from the java
heap, and can be accessed by native code directly, without having to do
the copying that you'd usually do to access a byte array. Effectively,
this means that java and C can share a region of memory. You can create
these buffers from native code, and, crucially, you can create them using
an existing area of memory - you don't have to create a buffer and then
copy things into it. You do this with the NewDirectByteBuffer call.

So, you could do something like:

int getattr(const char *path, struct stat *stat) {
JNIEnv *env = NULL ;
(*jvm)->AttachCurrentThread(jvm, &env, NULL) ;
jobject pathBuf = (*env)->NewDirectByteBuffer(env, (void*)path, (strlen(path) + 1)) ;
jobject statBuf = (*env)->NewDirectByteBuffer(env, (void*)stat, sizeof(struct stat) ;
jint ret = (*env)->CallIntMethod(env, fs, getattrID,pathBuf, statBuf) ;
(*jvm)->DetachCurrentThread(jvm) ;
return ret ;
}

With the java looking something like:

public class Filesystem {
public int getattr(ByteBuffer pathBuf, ByteBuffer statBuf) {
// might need to set endianness of buffers here
int dev = ... ;
int ino = ... ;
// etc
statBuf.putInt(dev) ;
statBuf.putInt(ino) ;
// etc
return 0 ;
}
}

Now, although you can pass the path this way, it probably doesn't do you a
lot of good, since you're really, really going to want to have the path as
a java string. For that, i'd stick to converting to a java string in the
native code. Using a direct buffer does let you share the stat struct -
but here, you are *entirely* dependent on knowing *exactly* how your C
compiler has laid the structure out in memory. Above, i've assumed it just
concatenated the fields in order, but, AIUI, there is absolutely no
guarantee of this happening in the C standard. If you want this code to
work robustly and portably, you'd probably have to write some crazy
routine in C which uses pointer twiddling to build a 'map' of the struct,
which it could then transmit to the java code. Something like:

struct stat *stat ;
int *dev = &(stat->dev) ;
int devOffset = (int)((void *)dev - (void *)stat) ;

But this is clearly totally bletcherous.

Even if you did that, you're still allocating little objects all over the
place to do the buffering, and that could be as slow or slower than
copying the struct into an object.

tom

 

[Roedy Green]

How to kludge?

see http://mindprod.com/jgloss/tuples.html

 

[Hendrik Maryns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(e-mail address removed) schreef:
|> This is one of the uglier parts of Java. It supports
|> multiple inputs to a method but only one output.
|>
|> How to kludge?
|>
|> Another approach is to create a class with a number of internal
|> variables.
|
| This is what SWIG does. The classes it creates has setters and
| getters. You have to write a very thin C wrapper which calls the C
| code you first thought of. The wrapper also manages this way of
| interfacing. The C code will have to populate the structure ready for
| return, your java code just puts the result into an object of the type
| generated by SWIG. You use the getters to obtain the member values.

But let me warn you about memory issues: the getters and setters write
through to the C side, so Java does not keep references to anything you
set. This means that if the thing that was set becomes unreachable
elsewhere, it will get garbage collected, although C still has a pointer
to it. Expect segfaults. The trick is to extend the wrapper class to
keep a reference to the element as well. It’s sort of available in the
manual, but not easy. Look for threads started by me on the SWIG
newsgroup, or ask me if you get in trouble.

H.
- --
Hendrik Maryns
http://tcl.sfs.uni-tuebingen.de/~hendrik/
==================
http://aouw.org
Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iEYEARECAAYFAkhx934ACgkQe+7xMGD3itQg+wCeIHA57NjUtfjZQ1E9zedpr7t6
lzQAnj7Z4kRbzQNT5DBXT0PMBliMj42I
=ZnBC
-----END PGP SIGNATURE-----

 

[Tom Anderson]

(e-mail address removed) schreef:
|> This is one of the uglier parts of Java. It supports
|> multiple inputs to a method but only one output.
|>
|> How to kludge?
|>
|> Another approach is to create a class with a number of internal
|> variables.
|
| This is what SWIG does. The classes it creates has setters and
| getters. You have to write a very thin C wrapper which calls the C
| code you first thought of. The wrapper also manages this way of
| interfacing. The C code will have to populate the structure ready for
| return, your java code just puts the result into an object of the type
| generated by SWIG. You use the getters to obtain the member values.

But let me warn you about memory issues: the getters and setters write
through to the C side, so Java does not keep references to anything you
set. This means that if the thing that was set becomes unreachable
elsewhere, it will get garbage collected, although C still has a pointer
to it. Expect segfaults. The trick is to extend the wrapper class to
keep a reference to the element as well.

Hang on, what? If you're talking about objects getting garbage-collected,
then you must be talking about java objects. Does SWIG support storing
pointers to java objects on the C side? I'm no SWIG expert, but i didn't
realise that.

I'm surprised to hear that it gets it wrong, though. It would be really
quite easy to do the right GlobalRef magic in the setters.

tom

 

[Hendrik Maryns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tom Anderson schreef:
| On Mon, 7 Jul 2008, Hendrik Maryns wrote:
|
|> (e-mail address removed) schreef:
|> |> This is one of the uglier parts of Java. It supports
|> |> multiple inputs to a method but only one output.
|> |>
|> |> How to kludge?
|> |>
|> |> Another approach is to create a class with a number of internal
|> |> variables.
|> |
|> | This is what SWIG does. The classes it creates has setters and
|> | getters. You have to write a very thin C wrapper which calls the C
|> | code you first thought of. The wrapper also manages this way of
|> | interfacing. The C code will have to populate the structure ready for
|> | return, your java code just puts the result into an object of the type
|> | generated by SWIG. You use the getters to obtain the member values.
|>
|> But let me warn you about memory issues: the getters and setters write
|> through to the C side, so Java does not keep references to anything
|> you set. This means that if the thing that was set becomes
|> unreachable elsewhere, it will get garbage collected, although C still
|> has a pointer to it. Expect segfaults. The trick is to extend the
|> wrapper class to keep a reference to the element as well.
|
| Hang on, what? If you're talking about objects getting
| garbage-collected, then you must be talking about java objects. Does
| SWIG support storing pointers to java objects on the C side? I'm no SWIG
| expert, but i didn't realise that.
|
| I'm surprised to hear that it gets it wrong, though. It would be really
| quite easy to do the right GlobalRef magic in the setters.

I have binary trees in Java, which I convert to binary trees in C, such
that some wrapped method can process them. I convert them bottom-up,
adding the daughters to a new node in a loop. At the end of the method,
only the root node is still referenced from Java. However, at each
iteration, two wrapper objects are created from Java, which hold a
pointer to a corresponding C node. On the C side, all references are
fine. However, for GC, the wrapper objects created in each loop aren’t
referenced anywhere, so if memory gets close, they are discarded. The
wrappers, however, contain code that frees the C object as well and
chaos follows. The trick is to write a reference to the wrapped
daughters in the node wrapper:

/* Stuff to take care of memory management. See SWIG manual section
20.9.12: Memory management
~ * for objects passed to the C++ layer
~ */
/* match the left daughter in the setter */
%typemap(javain) mgTreeNode *left
"getCPtrAndAddLeftDaughterReference($javainput)"
/* match the right daughter in the setter*/
%typemap(javain) mgTreeNode *right
"getCPtrAndAddRightDaughterReference($javainput)"

%typemap(javacode) struct mgTreeNode %{
~ /**
~ * Ensure that the GC doesn’t collect any element set from Java
~ * as the underlying C struct stores a shallow copy.
~ *
~ * The left element.
~ *
~ * @param element
~ * The element to store and get a pointer for.
~ */
~ private long getCPtrAndAddLeftDaughterReference(mgTreeNode daughter) {
~ leftDaughter = daughter;
~ return mgTreeNode.getCPtr(daughter);
~ }

~ /**
~ * The reference for the left daughter.
~ */
~ private mgTreeNode leftDaughter;

~ /**
~ * Ensure that the GC doesn’t collect any element set from Java
~ * as the underlying C struct stores a shallow copy.
~ *
~ * The right element.
~ *
~ * @param element
~ * The element to store and get a pointer for.
~ */
~ private long getCPtrAndAddRightDaughterReference(mgTreeNode daughter) {
~ rightDaughter = daughter;
~ return mgTreeNode.getCPtr(daughter);
~ }

~ /**
~ * The reference for the right daughter.
~ */
~ private mgTreeNode rightDaughter;

%}

Believe me, it doesn’t work without this.

H.
- --
Hendrik Maryns
http://tcl.sfs.uni-tuebingen.de/~hendrik/
==================
http://aouw.org
Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iEYEARECAAYFAkhyJ/cACgkQe+7xMGD3itT7UQCeM2nvlhx/thMvANjRioCnYAry
8bQAnAvtkPMSbSRudUDLmmuQHbBcZHvf
=MXSt
-----END PGP SIGNATURE-----

 

[Tom Anderson]

Tom Anderson schreef:
| On Mon, 7 Jul 2008, Hendrik Maryns wrote:
|
|> (e-mail address removed) schreef:
|> |> This is one of the uglier parts of Java. It supports
|> |> multiple inputs to a method but only one output.
|> |>
|> |> How to kludge?
|> |>
|> |> Another approach is to create a class with a number of internal
|> |> variables.
|> |
|> | This is what SWIG does. The classes it creates has setters and
|> | getters. You have to write a very thin C wrapper which calls the C
|> | code you first thought of. The wrapper also manages this way of
|> | interfacing. The C code will have to populate the structure ready for
|> | return, your java code just puts the result into an object of the type
|> | generated by SWIG. You use the getters to obtain the member values.
|>
|> But let me warn you about memory issues: the getters and setters write
|> through to the C side, so Java does not keep references to anything
|> you set. This means that if the thing that was set becomes
|> unreachable elsewhere, it will get garbage collected, although C still
|> has a pointer to it. Expect segfaults. The trick is to extend the
|> wrapper class to keep a reference to the element as well.
|
| Hang on, what? If you're talking about objects getting
| garbage-collected, then you must be talking about java objects. Does
| SWIG support storing pointers to java objects on the C side? I'm no SWIG
| expert, but i didn't realise that.
|
| I'm surprised to hear that it gets it wrong, though. It would be really
| quite easy to do the right GlobalRef magic in the setters.

I have binary trees in Java, which I convert to binary trees in C, such
that some wrapped method can process them. I convert them bottom-up,
adding the daughters to a new node in a loop. At the end of the method,
only the root node is still referenced from Java. However, at each
iteration, two wrapper objects are created from Java, which hold a
pointer to a corresponding C node. On the C side, all references are
fine. However, for GC, the wrapper objects created in each loop arent
referenced anywhere, so if memory gets close, they are discarded.
Right.

The wrappers, however, contain code that frees the C object as well and
chaos follows.

So i imagine!

The trick is to write a reference to the wrapped daughters in the node
wrapper:

I had to think about this for a while, but yes, you're absolutely right.
Then you can release them all in one go on the java side, by letting go of
the root reference, and let finalization free the C objects.

Your other option would be to tell SWIG not to do clever memory
management, and then free the C-side nodes by hand. But that's not at all
elegant.

Believe me, it doesnt work without this.

No. Applying this to the OP's case, the problem would be that if the java
callback created and returned a SWIG GetattrResults (say) object, then the
JVM and SWIG could conspire to GC it and free the C peer before the C
function had time to extract the results and return them. Which would
suck.

On the other hand, the OP could use SWIG to do something similar to what i
suggested: wrap the structs to be filled in in java wrappers which have
write-through setters, and pass these to the java callback, which could
then write to the structs directly. There wouldn't be any copying -
instead, the called-back java method would call back across the border
again to mutate the original C structs.

And this is in fact almost exactly isomorphic to my idea of using direct
byte buffers, except SWIG does all the hard work for you.

tom

 

[adityar7]

Ok, so the original solution posted by Tom, using the copyStatIn/
copyStatOut -- would that be vulnerable to the garbage collector
problem? From what I understand, it would unless I extend the wrapper
class to keep a reference to the element as well, as Hendrik points
out. So the Java wrapper class would keep a reference to the C struct?
But I thought that the JVM can't access the C memory in which case how
does it keep a reference to it

Unfortunately I don't completely understand Hendrik's SWIG code and
what exactly it does. So if I use SWIG, will I have to manually edit
the classes to keep the reference? Or is there a way SWIG can do all
this automatically? One would think that it could, if this is what
it's meant for.

Thanks,

Aditya

 

[adityar7]

Hmmm. Looks like my last reply didn't get posted. Anyway, so is Tom's
original solution (with the copyStatIn/copyStatOut) susceptible to the
garbage collection problem? From my understanding of the issue it
seems that the GC problem is indeed present unless we extend the
wrapper class to keep a reference to the element too, as pointed out
by Hendrik.

Just to confirm, what element exactly would the reference be to? Also,
how can the Java wrapper class have a reference to the C element given
that it can't access the C memory?

What about SWIG, does it automatically put the reference in the
wrapper class? It seems to me that this would be a common issue for an
interface generator meant for this kind of stuff. I don't completely
follow the code snippet posted by Hendrik so I can't tell how much
SWIG does.

Really appreciate your replies -- we are just beginning our project
right now and want to make sure we go the right way!

Thanks

 

[Hendrik Maryns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

adityar7 schreef:
| Hmmm. Looks like my last reply didn't get posted. Anyway, so is Tom's
| original solution (with the copyStatIn/copyStatOut) susceptible to the
| garbage collection problem? From my understanding of the issue it
| seems that the GC problem is indeed present unless we extend the
| wrapper class to keep a reference to the element too, as pointed out
| by Hendrik.
|
| Just to confirm, what element exactly would the reference be to? Also,
| how can the Java wrapper class have a reference to the C element given
| that it can't access the C memory?
|
| What about SWIG, does it automatically put the reference in the
| wrapper class? It seems to me that this would be a common issue for an
| interface generator meant for this kind of stuff. I don't completely
| follow the code snippet posted by Hendrik so I can't tell how much
| SWIG does.
|
| Really appreciate your replies -- we are just beginning our project
| right now and want to make sure we go the right way!

I am sorry for having caused confusion here.

I just wanted to say a word of warning: SWIG is great, and I’m very
satisfied with it, but it can’t do miracles. So go ahead, start using
it in the standard way. However, if you encounter unexpected segfaults,
remember my comment and read it again. By then, you’ll probably
understand my babbling. If not, ask. (But maybe better on the SWIG
mailing list.)

Tom is right in his latest post that his setup will probably work. I
didn’t read the original problem thoroughly when writing my warning,
just wanted to hold a warning finger in the air. (That’s prolly no
English, but what the hell.)

H.
- --
Hendrik Maryns
http://tcl.sfs.uni-tuebingen.de/~hendrik/
==================
http://aouw.org
Ask smart questions, get good answers:
http://www.catb.org/~esr/faqs/smart-questions.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iEYEARECAAYFAkhzPycACgkQe+7xMGD3itTEDwCdEaHia17PKCcB4Cw2lHfCOy0H
Af0AnR4XrydLPEBwhPYHkuTeWfuAvJiC
=1UdI
-----END PGP SIGNATURE-----

 

[Tom Anderson]

Hmmm. Looks like my last reply didn't get posted.

It did.

Anyway, so is Tom's original solution (with the copyStatIn/copyStatOut)
susceptible to the garbage collection problem?
No.

From my understanding of the issue it seems that the GC problem is
indeed present unless we extend the wrapper class to keep a reference to
the element too, as pointed out by Hendrik.

The GC problem is introduced by SWIG. My solution didn't use SWIG.

Just to confirm, what element exactly would the reference be to?
Exactly!

Also, how can the Java wrapper class have a reference to the C element
given that it can't access the C memory?

I think you've misunderstood. Hendrik's solution is to keep java
references to the java objects. No references to C objects have to be
managed explicitly. The thing is that if a SWIG-generated java object
dies, it will delete the corresponding C object, which we don't want. So,
keep references to the objects, which stops them dying, which stops them
deleting the C objects.

But to actually answer your question, in SWIG, C pointers are stored in
java longs, via a cast to long on the C side.

What about SWIG, does it automatically put the reference in the
wrapper class?
Yes.

It seems to me that this would be a common issue for an interface
generator meant for this kind of stuff. I don't completely follow the
code snippet posted by Hendrik so I can't tell how much SWIG does.

It is a common issue, and SWIG does a lot to deal with it - have a read of
the manual. However, as long as GC can't reach into the C side and read
its fields, which SWIG can't, there will always be corner cases where you
need to push it in the right direction wrt memory management. This is one
of them.

Really appreciate your replies -- we are just beginning our project
right now and want to make sure we go the right way!

Did you see my suggestion about using SWIG to wrap the structs and passing
the wrapped structs to java, providing write-through access and so
avoiding any copying? The more i think about it, the more i think this is
a better way to do it than copying. And there's no GC problem.

tom

 

[Aditya Rajgarhia]

Did you see my suggestion about using SWIG to wrap the structs and passing

the wrapped structs to java, providing write-through access and so
avoiding any copying? The more i think about it, the more i think this is
a better way to do it than copying. And there's no GC problem.

Ok, I read the Java part of the manual and ran SWIG on my C code. It
generated the wrapper classes in Java. However, the getters and
setters are created in Java, not in C. For my problem it seems that
they should be in C, like in Tom's original solution. I want to set
the struct in the C code before passing to Java, and then retrieve the
returned (updated) values in C.

Also, SWIG generated the JNI code for calling C code from the Java
code, which is the opposite of what I need.

So if I use SWIG, am I supposed to write the C getters and setter
myself? In that case, there would be no point in using SWIG, if all it
did was to type out the data fields of the classes. The majority of
the work in my case is writing the C getters and setters and the C JNI
code for calling the equivalent Java methods.

Unless I am missing something completely, and the generated code is
indeed what I need. Please point out if so

Thanks