JNI - Passing and returning complex values

Discussion in 'Java' started by adityar7, Jul 4, 2008.

  1. adityar7

    adityar7 Guest

    I'm a C programmer with limited Java experience, and need to use JNI
    to implement a library in Java on top of a C library that consists of
    a bunch of functions, each receiving some parameters from the
    operating system. For each C function, the corresponding Java method
    will be called, and some values would be returned back to the C
    function. i.e. C -> Java -> C.

    I know how to deal with simple primitive types, but I need to pass and
    return multiple fields which could be primitive types or C structs. I
    understand that to pass C structs I would need to define a
    corresponding Java class and copy over the struct fields.

    The question, though, is how to deal with multiple parameters/return
    values of different types. Here is a sample C function specification:

    int getattr (const char *, struct stat *)

    I have approximately 35 such functions with differing number and types
    of parameters. The corresponding Java method needs to be passed both
    these parameters, and it also needs to return both of them! Since I'm
    new to JNI the only way I can think of is to define 35 Java classes,
    each corresponding to a particular set of parameters. Hopefully there
    is a better way to do it!

    Any help would be greatly appreciated :D
    adityar7, Jul 4, 2008
    #1
    1. Advertising

  2. adityar7

    Arne Vajhøj Guest

    adityar7 wrote:
    > I'm a C programmer with limited Java experience, and need to use JNI
    > to implement a library in Java on top of a C library that consists of
    > a bunch of functions, each receiving some parameters from the
    > operating system. For each C function, the corresponding Java method
    > will be called, and some values would be returned back to the C
    > function. i.e. C -> Java -> C.
    >
    > I know how to deal with simple primitive types, but I need to pass and
    > return multiple fields which could be primitive types or C structs. I
    > understand that to pass C structs I would need to define a
    > corresponding Java class and copy over the struct fields.
    >
    > The question, though, is how to deal with multiple parameters/return
    > values of different types. Here is a sample C function specification:
    >
    > int getattr (const char *, struct stat *)
    >
    > I have approximately 35 such functions with differing number and types
    > of parameters. The corresponding Java method needs to be passed both
    > these parameters, and it also needs to return both of them! Since I'm
    > new to JNI the only way I can think of is to define 35 Java classes,
    > each corresponding to a particular set of parameters. Hopefully there
    > is a better way to do it!


    You can put all the native methods in one class.

    But if you need 35 different classes with data, then I can not
    see any way to avoid writing them.

    Arne
    Arne Vajhøj, Jul 4, 2008
    #2
    1. Advertising

  3. adityar7

    Guest

    On 4 Jul, 03:34, adityar7 <> wrote:
    > I'm a C programmer with limited Java experience, and need to use JNI
    > to implement a library in Java on top of a C library


    > The question, though, is how to deal with multiple parameters/return
    > values of different types. Here is a sample C function specification:
    >
    > int getattr (const char *, struct stat *)


    I recommend SWIG. See www.swig.org. This is what I used to interface
    to a proprietary C library for a server that we wanted to be in java.
    SWIG would take the prototype above and create corresponding java
    classes for where there is no natural mapping, e.g it would create a
    class for stat.

    Output from SWIG is in the form of two files, one is C glue to go into
    a dynamic library, the other is the java class that contains all the
    methods for the functions you want to call. Other java files are
    created for the new classes it creates, like stat. Your java program
    has to load the dynamic library first.

    Regards,

    Andrew Marlow
    , Jul 4, 2008
    #3
  4. adityar7

    Roedy Green Guest

    On Thu, 3 Jul 2008 19:34:30 -0700 (PDT), adityar7 <>
    wrote, quoted or indirectly quoted someone who said :

    >I have approximately 35 such functions with differing number and types
    >of parameters. The corresponding Java method needs to be passed both
    >these parameters, and it also needs to return both of them! Since I'm
    >new to JNI the only way I can think of is to define 35 Java classes,
    >each corresponding to a particular set of parameters. Hopefully there
    >is a better way to do it!

    This is one of the uglier parts of Java. It supports multiple inputs
    to a method but only one output.

    How to kludge?

    one way is to return an array of double or array of objects. You to
    lose typing and keyword identification of results.

    Another approach is to create a class with a number of internal
    variables. To use it you call setXXX to set each of the inputs, then
    call calcXXX to calculate. Then call getXXX to get each of the
    results. This is the most Javaesque solution.

    It is considered bad form to create result classes of scattered crap
    that is not logically related.

    Have your method return a result, and if there is a problem throw an
    exception, rather than returning a result and status as you would in
    C.

    Rethink your methods so that there are a small number of possible
    result classes that have physical meaning.

    They problem you are having is made worse if you try to write C in
    Java, rather than thinking OO.
    --

    Roedy Green Canadian Mind Products
    The Java Glossary
    http://mindprod.com
    Roedy Green, Jul 4, 2008
    #4
  5. adityar7

    Guest

    On 4 Jul, 13:54, Roedy Green <> wrote:
    > This is one of the uglier parts of Java. It supports
    > multiple inputs to a method but only one output.
    >
    > How to kludge?
    >
    > Another approach is to create a class with a number of internal
    > variables.


    This is what SWIG does. The classes it creates has setters and
    getters. You have to write a very thin C wrapper which calls the C
    code you first thought of. The wrapper also manages this way of
    interfacing. The C code will have to populate the structure ready for
    return, your java code just puts the result into an object of the type
    generated by SWIG. You use the getters to obtain the member values.

    I reckon this is MUCH easier than doing raw JNI code.

    -Andrew Marlow
    , Jul 4, 2008
    #5
  6. adityar7

    Guest

    On 4 Jul, 14:39, Lew <> wrote:
    > wrote:
    > > On 4 Jul, 03:34, adityar7 <> wrote:
    > >> I'm a C programmer with limited Java experience, and need to use JNI
    > >> to implement a library in Java on top of a C library

    >
    > >> The question, though, is how to deal with multiple parameters/return
    > >> values of different types. Here is a sample C function specification:

    >
    > >> int getattr (const char *, struct stat *)

    >
    > > I recommend SWIG. Seewww.swig.org. This is what I used to interface
    > > to a proprietary C library for a server that we wanted to be in java.
    > > SWIG would take the prototype above and create corresponding java
    > > classes for where there is no natural mapping, e.g it would create a
    > > class for stat.

    >
    > > Output from SWIG is in the form of two files, one is C glue to go into
    > > a dynamic library, the other is the java class that contains all the
    > > methods for the functions you want to call. Other java files are
    > > created for the new classes it creates, like stat. Your java program
    > > has to load the dynamic library first.

    >
    > SWIG sounds like a real find. Thanks for bringing it to our attention.
    >
    > Regarding the example with the 'stat' struct, does SWIG let you turn that into
    > a convention-compliant type name, 'Stat'?
    >
    > --
    > Lew


    Yes it does. I must admit, I am suprised when java people tell me they
    don't know about SWIG. I am relatively new to java and I was lukcily
    to come across it quite early. I saw it used about a year ago on a C++
    project where suddenly a java interface was required by certain
    customers. It was quickly knocked up using SWIG. The C++ was concerned
    mainly with returning structures that represented trades for
    particular financial instruments. Each instrument had its own type and
    several were expressed in terms of other types.

    -Andrew Marlow
    , Jul 4, 2008
    #6
  7. adityar7

    Tom Anderson Guest

    On Thu, 3 Jul 2008, adityar7 wrote:

    > I'm a C programmer with limited Java experience, and need to use JNI
    > to implement a library in Java on top of a C library that consists of
    > a bunch of functions, each receiving some parameters from the
    > operating system. For each C function, the corresponding Java method
    > will be called, and some values would be returned back to the C
    > function. i.e. C -> Java -> C.
    >
    > I know how to deal with simple primitive types, but I need to pass and
    > return multiple fields which could be primitive types or C structs. I
    > understand that to pass C structs I would need to define a
    > corresponding Java class and copy over the struct fields.
    >
    > The question, though, is how to deal with multiple parameters/return
    > values of different types. Here is a sample C function specification:
    >
    > int getattr (const char *, struct stat *)
    >
    > I have approximately 35 such functions with differing number and types
    > of parameters. The corresponding Java method needs to be passed both
    > these parameters, and it also needs to return both of them! Since I'm
    > new to JNI the only way I can think of is to define 35 Java classes,
    > each corresponding to a particular set of parameters. Hopefully there
    > is a better way to do it!
    >
    > Any help would be greatly appreciated :D


    Well, firstly, are you aware that there is already a java binding of FUSE?

    http://sourceforge.net/projects/fuse-j/

    If you don't want to use that, or if this isn't what you're working on, to
    answer your question, i'd do a straightforward conversion of the C
    interface, and pass a writeable object from C to java, get java to write
    into it, then copy back from the java object to the C one. Like so:


    // java:

    public class Stat {
    public int ino ;
    // etc
    }

    public interface Filesystem {
    public int getattr(String path, Stat sb) ;
    }

    public class FUSEManager {
    static {
    System.loadLibrary("fuse") ;
    }
    public static native void init(Filesystem fs) ;
    }

    // C:

    /* error checking omitted for clarity! */

    #define OK 0

    JavaVM *jvm ; // JVM ref, get in a setup method

    jobject fs ; // ref to a Filesystem object, get in a setup method
    jclass fsClass ;
    jmethodID getattrID ;

    jclass statClass ;
    jmethodID statCtorID ;
    jfieldID statInoID ;
    // etc for other fields

    JNIEXPORT void JNICALL Java_FUSEManager_init(JNIEnv *env, jclass class, jobject _fs) {
    (*env)->GetJavaVM(env, &jvm) ;
    fs = (*env)->NewGlobalRef(env, _fs) ;
    jclass _fsClass = (*env)->GetObjectClass(env, fs) ;
    fsClass = (*env)->NewGlobalRef(env, _fsClass) ;
    getattrID = (*env)->GetMethodID(env, fsClass, "getattr", "(Ljava/lang/String;LStat;)I") ;
    jclass _statClass = (*env)->FindClass(env, "com/mypackage/Stat") ;
    statClass = (*env)->NewGlobalRef(env, _statClass) ;
    statCtorID = (*env)->GetMethodID(env, statClass, "<init>", "()V") ;
    statInoID = (*env)->GetFieldID(env, statClass, "ino", "I") ;
    // etc for other fields
    }

    void copyStatIn(JNIEnv *env, jobject statObj, struct stat *stat) {
    stat->st_ino = (*env)->GetIntField(env, statObj, statInoID) ;
    // etc for other fields
    }

    // would define a copyStatOut for setattr

    int getattr(const char *path, struct stat *stat) {
    JNIEnv *env = NULL ;
    (*jvm)->AttachCurrentThread(jvm, &env, NULL) ;
    jstring pathStr = (*env)->NewStringUTF(env, path) ;
    jobject statObj = (*env)->NewObject(env, statClass, ctorID) ;
    /* i don't copy the struct into the object here - hope this is right */
    jint ret = (*env)->CallIntMethod(env, fs, getattrID, pathStr, statObj) ;
    /* don't copy back if call failed */
    if (ret == OK) copyStatIn(statObj, stat) ;
    /* don't think we need to release local refs explicitly before detach */
    (*jvm)->DetachCurrentThread(jvm) ;
    return ret ;
    }

    Does that make sense? This means defining one class per struct, plus one
    copy-in method and one copy-out method, as needed. No need to write 35
    different classes!

    I've assumed that, despite what you said, the const char* parameter isn't
    'returned' in any sense.

    tom

    --
    Know who said that? Fucking Terrorvision, that's who. -- D
    Tom Anderson, Jul 4, 2008
    #7
  8. adityar7

    adityar7 Guest

    On Jul 4, 4:18 am, wrote:
    > On 4 Jul, 03:34, adityar7 <> wrote:
    >
    > > I'm a C programmer with limited Java experience, and need to use JNI
    > > to implement a library in Java on top of a C library
    > > The question, though, is how to deal with multiple parameters/return
    > > values of different types. Here is a sample C function specification:

    >
    > > int getattr (const char *, struct stat *)

    >
    > I recommend SWIG. Seewww.swig.org. This is what I used to interface
    > to a proprietary C library for a server that we wanted to be in java.
    > SWIG would take the prototype above and create corresponding java
    > classes for where there is no natural mapping, e.g it would create a
    > class for stat.
    >
    > Output from SWIG is in the form of two files, one is C glue to go into
    > a dynamic library, the other is the java class that contains all the
    > methods for the functions you want to call. Other java files are
    > created for the new classes it creates, like stat. Your java program
    > has to load the dynamic library first.
    >
    > Regards,
    >
    > Andrew Marlow


    Sounds excellent. I have seen that term (SWIG) often but not being a
    regular Java programmer, didn't know what that means. I'll check it
    out and see how it goes. Thanks for pointing it out!
    adityar7, Jul 5, 2008
    #8
  9. adityar7

    adityar7 Guest

    On Jul 4, 8:12 am, Tom Anderson <> wrote:
    > On Thu, 3 Jul 2008, adityar7 wrote:
    > > I'm a C programmer with limited Java experience, and need to use JNI
    > > to implement a library in Java on top of a C library that consists of
    > > a bunch of functions, each receiving some parameters from the
    > > operating system. For each C function, the corresponding Java method
    > > will be called, and some values would be returned back to the C
    > > function. i.e. C -> Java -> C.

    >
    > > I know how to deal with simple primitive types, but I need to pass and
    > > return multiple fields which could be primitive types or C structs. I
    > > understand that to pass C structs I would need to define a
    > > corresponding Java class and copy over the struct fields.

    >
    > > The question, though, is how to deal with multiple parameters/return
    > > values of different types. Here is a sample C function specification:

    >
    > > int getattr (const char *, struct stat *)

    >
    > > I have approximately 35 such functions with differing number and types
    > > of parameters. The corresponding Java method needs to be passed both
    > > these parameters, and it also needs to return both of them! Since I'm
    > > new to JNI the only way I can think of is to define 35 Java classes,
    > > each corresponding to a particular set of parameters. Hopefully there
    > > is a better way to do it!

    >
    > > Any help would be greatly appreciated :D

    >
    > Well, firstly, are you aware that there is already a java binding of FUSE?
    >
    > http://sourceforge.net/projects/fuse-j/
    >
    > If you don't want to use that, or if this isn't what you're working on, to
    > answer your question, i'd do a straightforward conversion of the C
    > interface, and pass a writeable object from C to java, get java to write
    > into it, then copy back from the java object to the C one. Like so:
    >
    > // java:
    >
    > public class Stat {
    > public int ino ;
    > // etc
    >
    > }
    >
    > public interface Filesystem {
    > public int getattr(String path, Stat sb) ;
    >
    > }
    >
    > public class FUSEManager {
    > static {
    > System.loadLibrary("fuse") ;
    > }
    > public static native void init(Filesystem fs) ;
    >
    > }
    >
    > // C:
    >
    > /* error checking omitted for clarity! */
    >
    > #define OK 0
    >
    > JavaVM *jvm ; // JVM ref, get in a setup method
    >
    > jobject fs ; // ref to a Filesystem object, get in a setup method
    > jclass fsClass ;
    > jmethodID getattrID ;
    >
    > jclass statClass ;
    > jmethodID statCtorID ;
    > jfieldID statInoID ;
    > // etc for other fields
    >
    > JNIEXPORT void JNICALL Java_FUSEManager_init(JNIEnv *env, jclass class, jobject _fs) {
    > (*env)->GetJavaVM(env, &jvm) ;
    > fs = (*env)->NewGlobalRef(env, _fs) ;
    > jclass _fsClass = (*env)->GetObjectClass(env, fs) ;
    > fsClass = (*env)->NewGlobalRef(env, _fsClass) ;
    > getattrID = (*env)->GetMethodID(env, fsClass, "getattr", "(Ljava/lang/String;LStat;)I") ;
    > jclass _statClass = (*env)->FindClass(env, "com/mypackage/Stat") ;
    > statClass = (*env)->NewGlobalRef(env, _statClass) ;
    > statCtorID = (*env)->GetMethodID(env, statClass, "<init>", "()V") ;
    > statInoID = (*env)->GetFieldID(env, statClass, "ino", "I") ;
    > // etc for other fields
    >
    > }
    >
    > void copyStatIn(JNIEnv *env, jobject statObj, struct stat *stat) {
    > stat->st_ino = (*env)->GetIntField(env, statObj, statInoID) ;
    > // etc for other fields
    >
    > }
    >
    > // would define a copyStatOut for setattr
    >
    > int getattr(const char *path, struct stat *stat) {
    > JNIEnv *env = NULL ;
    > (*jvm)->AttachCurrentThread(jvm, &env, NULL) ;
    > jstring pathStr = (*env)->NewStringUTF(env, path) ;
    > jobject statObj = (*env)->NewObject(env, statClass, ctorID) ;
    > /* i don't copy the struct into the object here - hope this is right */
    > jint ret = (*env)->CallIntMethod(env, fs, getattrID, pathStr, statObj) ;
    > /* don't copy back if call failed */
    > if (ret == OK) copyStatIn(statObj, stat) ;
    > /* don't think we need to release local refs explicitly before detach */
    > (*jvm)->DetachCurrentThread(jvm) ;
    > return ret ;
    >
    > }
    >
    > Does that make sense? This means defining one class per struct, plus one
    > copy-in method and one copy-out method, as needed. No need to write 35
    > different classes!
    >
    > I've assumed that, despite what you said, the const char* parameter isn't
    > 'returned' in any sense.
    >
    > tom
    >
    > --
    > Know who said that? Fucking Terrorvision, that's who. -- D


    Tom, I am indeed working with FUSE (good guess there! I didn't mention
    it in my post to keep things simple). I can't use the Java bindings
    for a couple of fundamental reasons. Most importantly, I need a mix of
    C and Java. Your solution would certainly be better than writing
    individual classes, and I will look into this before messing with
    SWIG. Actually, I do need to return the const char* parameters, which
    I first copy into a jstring and copy the result back into a char*
    similar to what you've done for the struct.

    What's unfortunate is that everything needs to be copied. In the ideal
    situation the Java methods would operate directly on the memory (using
    the C pointer or any other method). If anyone know if that's possible,
    please let me know :D

    Thanks for your reply!
    adityar7, Jul 5, 2008
    #9
  10. adityar7

    Tom Anderson Guest

    On Fri, 4 Jul 2008, adityar7 wrote:

    > What's unfortunate is that everything needs to be copied. In the ideal
    > situation the Java methods would operate directly on the memory (using
    > the C pointer or any other method). If anyone know if that's possible,
    > please let me know :D


    Actually, it is, but it's not pretty.

    Do you know about the New I/O, aka nio, package? It has things called
    ByteBuffers, which are basically fancy dressed-up byte arrays. Except that
    they can be 'direct', which means they're sort of separate from the java
    heap, and can be accessed by native code directly, without having to do
    the copying that you'd usually do to access a byte array. Effectively,
    this means that java and C can share a region of memory. You can create
    these buffers from native code, and, crucially, you can create them using
    an existing area of memory - you don't have to create a buffer and then
    copy things into it. You do this with the NewDirectByteBuffer call.

    So, you could do something like:

    int getattr(const char *path, struct stat *stat) {
    JNIEnv *env = NULL ;
    (*jvm)->AttachCurrentThread(jvm, &env, NULL) ;
    jobject pathBuf = (*env)->NewDirectByteBuffer(env, (void*)path, (strlen(path) + 1)) ;
    jobject statBuf = (*env)->NewDirectByteBuffer(env, (void*)stat, sizeof(struct stat) ;
    jint ret = (*env)->CallIntMethod(env, fs, getattrID,pathBuf, statBuf) ;
    (*jvm)->DetachCurrentThread(jvm) ;
    return ret ;
    }

    With the java looking something like:

    public class Filesystem {
    public int getattr(ByteBuffer pathBuf, ByteBuffer statBuf) {
    // might need to set endianness of buffers here
    int dev = ... ;
    int ino = ... ;
    // etc
    statBuf.putInt(dev) ;
    statBuf.putInt(ino) ;
    // etc
    return 0 ;
    }
    }

    Now, although you can pass the path this way, it probably doesn't do you a
    lot of good, since you're really, really going to want to have the path as
    a java string. For that, i'd stick to converting to a java string in the
    native code. Using a direct buffer does let you share the stat struct -
    but here, you are *entirely* dependent on knowing *exactly* how your C
    compiler has laid the structure out in memory. Above, i've assumed it just
    concatenated the fields in order, but, AIUI, there is absolutely no
    guarantee of this happening in the C standard. If you want this code to
    work robustly and portably, you'd probably have to write some crazy
    routine in C which uses pointer twiddling to build a 'map' of the struct,
    which it could then transmit to the java code. Something like:

    struct stat *stat ;
    int *dev = &(stat->dev) ;
    int devOffset = (int)((void *)dev - (void *)stat) ;

    But this is clearly totally bletcherous.

    Even if you did that, you're still allocating little objects all over the
    place to do the buffering, and that could be as slow or slower than
    copying the struct into an object.

    tom

    --
    Don't anthropomorphize computers: they don't like that.
    Tom Anderson, Jul 5, 2008
    #10
  11. adityar7

    Roedy Green Guest

    Roedy Green, Jul 5, 2008
    #11
  12. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    schreef:
    | On 4 Jul, 13:54, Roedy Green <> wrote:
    |> This is one of the uglier parts of Java. It supports
    |> multiple inputs to a method but only one output.
    |>
    |> How to kludge?
    |>
    |> Another approach is to create a class with a number of internal
    |> variables.
    |
    | This is what SWIG does. The classes it creates has setters and
    | getters. You have to write a very thin C wrapper which calls the C
    | code you first thought of. The wrapper also manages this way of
    | interfacing. The C code will have to populate the structure ready for
    | return, your java code just puts the result into an object of the type
    | generated by SWIG. You use the getters to obtain the member values.

    But let me warn you about memory issues: the getters and setters write
    through to the C side, so Java does not keep references to anything you
    set. This means that if the thing that was set becomes unreachable
    elsewhere, it will get garbage collected, although C still has a pointer
    to it. Expect segfaults. The trick is to extend the wrapper class to
    keep a reference to the element as well. It’s sort of available in the
    manual, but not easy. Look for threads started by me on the SWIG
    newsgroup, or ask me if you get in trouble.

    H.
    - --
    Hendrik Maryns
    http://tcl.sfs.uni-tuebingen.de/~hendrik/
    ==================
    http://aouw.org
    Ask smart questions, get good answers:
    http://www.catb.org/~esr/faqs/smart-questions.html
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.9 (GNU/Linux)
    Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

    iEYEARECAAYFAkhx934ACgkQe+7xMGD3itQg+wCeIHA57NjUtfjZQ1E9zedpr7t6
    lzQAnj7Z4kRbzQNT5DBXT0PMBliMj42I
    =ZnBC
    -----END PGP SIGNATURE-----
    Hendrik Maryns, Jul 7, 2008
    #12
  13. adityar7

    Tom Anderson Guest

    On Mon, 7 Jul 2008, Hendrik Maryns wrote:

    > schreef:
    > | On 4 Jul, 13:54, Roedy Green <> wrote:
    > |> This is one of the uglier parts of Java. It supports
    > |> multiple inputs to a method but only one output.
    > |>
    > |> How to kludge?
    > |>
    > |> Another approach is to create a class with a number of internal
    > |> variables.
    > |
    > | This is what SWIG does. The classes it creates has setters and
    > | getters. You have to write a very thin C wrapper which calls the C
    > | code you first thought of. The wrapper also manages this way of
    > | interfacing. The C code will have to populate the structure ready for
    > | return, your java code just puts the result into an object of the type
    > | generated by SWIG. You use the getters to obtain the member values.
    >
    > But let me warn you about memory issues: the getters and setters write
    > through to the C side, so Java does not keep references to anything you
    > set. This means that if the thing that was set becomes unreachable
    > elsewhere, it will get garbage collected, although C still has a pointer
    > to it. Expect segfaults. The trick is to extend the wrapper class to
    > keep a reference to the element as well.


    Hang on, what? If you're talking about objects getting garbage-collected,
    then you must be talking about java objects. Does SWIG support storing
    pointers to java objects on the C side? I'm no SWIG expert, but i didn't
    realise that.

    I'm surprised to hear that it gets it wrong, though. It would be really
    quite easy to do the right GlobalRef magic in the setters.

    tom

    --
    unconstrained by any considerations of humanity or decency
    Tom Anderson, Jul 7, 2008
    #13
  14. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Tom Anderson schreef:
    | On Mon, 7 Jul 2008, Hendrik Maryns wrote:
    |
    |> schreef:
    |> | On 4 Jul, 13:54, Roedy Green <> wrote:
    |> |> This is one of the uglier parts of Java. It supports
    |> |> multiple inputs to a method but only one output.
    |> |>
    |> |> How to kludge?
    |> |>
    |> |> Another approach is to create a class with a number of internal
    |> |> variables.
    |> |
    |> | This is what SWIG does. The classes it creates has setters and
    |> | getters. You have to write a very thin C wrapper which calls the C
    |> | code you first thought of. The wrapper also manages this way of
    |> | interfacing. The C code will have to populate the structure ready for
    |> | return, your java code just puts the result into an object of the type
    |> | generated by SWIG. You use the getters to obtain the member values.
    |>
    |> But let me warn you about memory issues: the getters and setters write
    |> through to the C side, so Java does not keep references to anything
    |> you set. This means that if the thing that was set becomes
    |> unreachable elsewhere, it will get garbage collected, although C still
    |> has a pointer to it. Expect segfaults. The trick is to extend the
    |> wrapper class to keep a reference to the element as well.
    |
    | Hang on, what? If you're talking about objects getting
    | garbage-collected, then you must be talking about java objects. Does
    | SWIG support storing pointers to java objects on the C side? I'm no SWIG
    | expert, but i didn't realise that.
    |
    | I'm surprised to hear that it gets it wrong, though. It would be really
    | quite easy to do the right GlobalRef magic in the setters.

    I have binary trees in Java, which I convert to binary trees in C, such
    that some wrapped method can process them. I convert them bottom-up,
    adding the daughters to a new node in a loop. At the end of the method,
    only the root node is still referenced from Java. However, at each
    iteration, two wrapper objects are created from Java, which hold a
    pointer to a corresponding C node. On the C side, all references are
    fine. However, for GC, the wrapper objects created in each loop aren’t
    referenced anywhere, so if memory gets close, they are discarded. The
    wrappers, however, contain code that frees the C object as well and
    chaos follows. The trick is to write a reference to the wrapped
    daughters in the node wrapper:

    /* Stuff to take care of memory management. See SWIG manual section
    20.9.12: Memory management
    ~ * for objects passed to the C++ layer
    ~ */
    /* match the left daughter in the setter */
    %typemap(javain) mgTreeNode *left
    "getCPtrAndAddLeftDaughterReference($javainput)"
    /* match the right daughter in the setter*/
    %typemap(javain) mgTreeNode *right
    "getCPtrAndAddRightDaughterReference($javainput)"

    %typemap(javacode) struct mgTreeNode %{
    ~ /**
    ~ * Ensure that the GC doesn’t collect any element set from Java
    ~ * as the underlying C struct stores a shallow copy.
    ~ *
    ~ * The left element.
    ~ *
    ~ * @param element
    ~ * The element to store and get a pointer for.
    ~ */
    ~ private long getCPtrAndAddLeftDaughterReference(mgTreeNode daughter) {
    ~ leftDaughter = daughter;
    ~ return mgTreeNode.getCPtr(daughter);
    ~ }

    ~ /**
    ~ * The reference for the left daughter.
    ~ */
    ~ private mgTreeNode leftDaughter;

    ~ /**
    ~ * Ensure that the GC doesn’t collect any element set from Java
    ~ * as the underlying C struct stores a shallow copy.
    ~ *
    ~ * The right element.
    ~ *
    ~ * @param element
    ~ * The element to store and get a pointer for.
    ~ */
    ~ private long getCPtrAndAddRightDaughterReference(mgTreeNode daughter) {
    ~ rightDaughter = daughter;
    ~ return mgTreeNode.getCPtr(daughter);
    ~ }

    ~ /**
    ~ * The reference for the right daughter.
    ~ */
    ~ private mgTreeNode rightDaughter;

    %}

    Believe me, it doesn’t work without this.

    H.
    - --
    Hendrik Maryns
    http://tcl.sfs.uni-tuebingen.de/~hendrik/
    ==================
    http://aouw.org
    Ask smart questions, get good answers:
    http://www.catb.org/~esr/faqs/smart-questions.html
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.9 (GNU/Linux)
    Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

    iEYEARECAAYFAkhyJ/cACgkQe+7xMGD3itT7UQCeM2nvlhx/thMvANjRioCnYAry
    8bQAnAvtkPMSbSRudUDLmmuQHbBcZHvf
    =MXSt
    -----END PGP SIGNATURE-----
    Hendrik Maryns, Jul 7, 2008
    #14
  15. adityar7

    Tom Anderson Guest

    On Mon, 7 Jul 2008, Hendrik Maryns wrote:

    > Tom Anderson schreef:
    > | On Mon, 7 Jul 2008, Hendrik Maryns wrote:
    > |
    > |> schreef:
    > |> | On 4 Jul, 13:54, Roedy Green <> wrote:
    > |> |> This is one of the uglier parts of Java. It supports
    > |> |> multiple inputs to a method but only one output.
    > |> |>
    > |> |> How to kludge?
    > |> |>
    > |> |> Another approach is to create a class with a number of internal
    > |> |> variables.
    > |> |
    > |> | This is what SWIG does. The classes it creates has setters and
    > |> | getters. You have to write a very thin C wrapper which calls the C
    > |> | code you first thought of. The wrapper also manages this way of
    > |> | interfacing. The C code will have to populate the structure ready for
    > |> | return, your java code just puts the result into an object of the type
    > |> | generated by SWIG. You use the getters to obtain the member values.
    > |>
    > |> But let me warn you about memory issues: the getters and setters write
    > |> through to the C side, so Java does not keep references to anything
    > |> you set. This means that if the thing that was set becomes
    > |> unreachable elsewhere, it will get garbage collected, although C still
    > |> has a pointer to it. Expect segfaults. The trick is to extend the
    > |> wrapper class to keep a reference to the element as well.
    > |
    > | Hang on, what? If you're talking about objects getting
    > | garbage-collected, then you must be talking about java objects. Does
    > | SWIG support storing pointers to java objects on the C side? I'm no SWIG
    > | expert, but i didn't realise that.
    > |
    > | I'm surprised to hear that it gets it wrong, though. It would be really
    > | quite easy to do the right GlobalRef magic in the setters.
    >
    > I have binary trees in Java, which I convert to binary trees in C, such
    > that some wrapped method can process them. I convert them bottom-up,
    > adding the daughters to a new node in a loop. At the end of the method,
    > only the root node is still referenced from Java. However, at each
    > iteration, two wrapper objects are created from Java, which hold a
    > pointer to a corresponding C node. On the C side, all references are
    > fine. However, for GC, the wrapper objects created in each loop arent
    > referenced anywhere, so if memory gets close, they are discarded.


    Right.

    > The wrappers, however, contain code that frees the C object as well and
    > chaos follows.


    So i imagine!

    > The trick is to write a reference to the wrapped daughters in the node
    > wrapper:


    I had to think about this for a while, but yes, you're absolutely right.
    Then you can release them all in one go on the java side, by letting go of
    the root reference, and let finalization free the C objects.

    Your other option would be to tell SWIG not to do clever memory
    management, and then free the C-side nodes by hand. But that's not at all
    elegant.

    > Believe me, it doesnt work without this.


    No. Applying this to the OP's case, the problem would be that if the java
    callback created and returned a SWIG GetattrResults (say) object, then the
    JVM and SWIG could conspire to GC it and free the C peer before the C
    function had time to extract the results and return them. Which would
    suck.

    On the other hand, the OP could use SWIG to do something similar to what i
    suggested: wrap the structs to be filled in in java wrappers which have
    write-through setters, and pass these to the java callback, which could
    then write to the structs directly. There wouldn't be any copying -
    instead, the called-back java method would call back across the border
    again to mutate the original C structs.

    And this is in fact almost exactly isomorphic to my idea of using direct
    byte buffers, except SWIG does all the hard work for you.

    tom

    --
    There are lousy reviews, and then there's empirical shitness. -- pikelet
    Tom Anderson, Jul 7, 2008
    #15
  16. adityar7

    adityar7 Guest

    Ok, so the original solution posted by Tom, using the copyStatIn/
    copyStatOut -- would that be vulnerable to the garbage collector
    problem? From what I understand, it would unless I extend the wrapper
    class to keep a reference to the element as well, as Hendrik points
    out. So the Java wrapper class would keep a reference to the C struct?
    But I thought that the JVM can't access the C memory in which case how
    does it keep a reference to it :(

    Unfortunately I don't completely understand Hendrik's SWIG code and
    what exactly it does. So if I use SWIG, will I have to manually edit
    the classes to keep the reference? Or is there a way SWIG can do all
    this automatically? One would think that it could, if this is what
    it's meant for.

    Thanks,

    Aditya
    adityar7, Jul 8, 2008
    #16
  17. adityar7

    adityar7 Guest

    Hmmm. Looks like my last reply didn't get posted. Anyway, so is Tom's
    original solution (with the copyStatIn/copyStatOut) susceptible to the
    garbage collection problem? From my understanding of the issue it
    seems that the GC problem is indeed present unless we extend the
    wrapper class to keep a reference to the element too, as pointed out
    by Hendrik.

    Just to confirm, what element exactly would the reference be to? Also,
    how can the Java wrapper class have a reference to the C element given
    that it can't access the C memory?

    What about SWIG, does it automatically put the reference in the
    wrapper class? It seems to me that this would be a common issue for an
    interface generator meant for this kind of stuff. I don't completely
    follow the code snippet posted by Hendrik so I can't tell how much
    SWIG does.

    Really appreciate your replies -- we are just beginning our project
    right now and want to make sure we go the right way!

    Thanks
    adityar7, Jul 8, 2008
    #17
  18. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    adityar7 schreef:
    | Hmmm. Looks like my last reply didn't get posted. Anyway, so is Tom's
    | original solution (with the copyStatIn/copyStatOut) susceptible to the
    | garbage collection problem? From my understanding of the issue it
    | seems that the GC problem is indeed present unless we extend the
    | wrapper class to keep a reference to the element too, as pointed out
    | by Hendrik.
    |
    | Just to confirm, what element exactly would the reference be to? Also,
    | how can the Java wrapper class have a reference to the C element given
    | that it can't access the C memory?
    |
    | What about SWIG, does it automatically put the reference in the
    | wrapper class? It seems to me that this would be a common issue for an
    | interface generator meant for this kind of stuff. I don't completely
    | follow the code snippet posted by Hendrik so I can't tell how much
    | SWIG does.
    |
    | Really appreciate your replies -- we are just beginning our project
    | right now and want to make sure we go the right way!

    I am sorry for having caused confusion here.

    I just wanted to say a word of warning: SWIG is great, and I’m very
    satisfied with it, but it can’t do miracles. So go ahead, start using
    it in the standard way. However, if you encounter unexpected segfaults,
    remember my comment and read it again. By then, you’ll probably
    understand my babbling. If not, ask. (But maybe better on the SWIG
    mailing list.)

    Tom is right in his latest post that his setup will probably work. I
    didn’t read the original problem thoroughly when writing my warning,
    just wanted to hold a warning finger in the air. (That’s prolly no
    English, but what the hell.)

    H.
    - --
    Hendrik Maryns
    http://tcl.sfs.uni-tuebingen.de/~hendrik/
    ==================
    http://aouw.org
    Ask smart questions, get good answers:
    http://www.catb.org/~esr/faqs/smart-questions.html
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.9 (GNU/Linux)
    Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

    iEYEARECAAYFAkhzPycACgkQe+7xMGD3itTEDwCdEaHia17PKCcB4Cw2lHfCOy0H
    Af0AnR4XrydLPEBwhPYHkuTeWfuAvJiC
    =1UdI
    -----END PGP SIGNATURE-----
    Hendrik Maryns, Jul 8, 2008
    #18
  19. adityar7

    Tom Anderson Guest

    On Tue, 8 Jul 2008, adityar7 wrote:

    > Hmmm. Looks like my last reply didn't get posted.


    It did.

    > Anyway, so is Tom's original solution (with the copyStatIn/copyStatOut)
    > susceptible to the garbage collection problem?


    No.

    > From my understanding of the issue it seems that the GC problem is
    > indeed present unless we extend the wrapper class to keep a reference to
    > the element too, as pointed out by Hendrik.


    The GC problem is introduced by SWIG. My solution didn't use SWIG.

    > Just to confirm, what element exactly would the reference be to?


    Exactly!

    > Also, how can the Java wrapper class have a reference to the C element
    > given that it can't access the C memory?


    I think you've misunderstood. Hendrik's solution is to keep java
    references to the java objects. No references to C objects have to be
    managed explicitly. The thing is that if a SWIG-generated java object
    dies, it will delete the corresponding C object, which we don't want. So,
    keep references to the objects, which stops them dying, which stops them
    deleting the C objects.

    But to actually answer your question, in SWIG, C pointers are stored in
    java longs, via a cast to long on the C side.

    > What about SWIG, does it automatically put the reference in the
    > wrapper class?


    Yes.

    > It seems to me that this would be a common issue for an interface
    > generator meant for this kind of stuff. I don't completely follow the
    > code snippet posted by Hendrik so I can't tell how much SWIG does.


    It is a common issue, and SWIG does a lot to deal with it - have a read of
    the manual. However, as long as GC can't reach into the C side and read
    its fields, which SWIG can't, there will always be corner cases where you
    need to push it in the right direction wrt memory management. This is one
    of them.

    > Really appreciate your replies -- we are just beginning our project
    > right now and want to make sure we go the right way!


    Did you see my suggestion about using SWIG to wrap the structs and passing
    the wrapped structs to java, providing write-through access and so
    avoiding any copying? The more i think about it, the more i think this is
    a better way to do it than copying. And there's no GC problem.

    tom

    --
    When you mentioned INSERT-MIND-INPUT ... did they look at you like this?
    Tom Anderson, Jul 8, 2008
    #19
  20. > Did you see my suggestion about using SWIG to wrap the structs and passing
    > the wrapped structs to java, providing write-through access and so
    > avoiding any copying? The more i think about it, the more i think this is
    > a better way to do it than copying. And there's no GC problem.


    Ok, I read the Java part of the manual and ran SWIG on my C code. It
    generated the wrapper classes in Java. However, the getters and
    setters are created in Java, not in C. For my problem it seems that
    they should be in C, like in Tom's original solution. I want to set
    the struct in the C code before passing to Java, and then retrieve the
    returned (updated) values in C.

    Also, SWIG generated the JNI code for calling C code from the Java
    code, which is the opposite of what I need.

    So if I use SWIG, am I supposed to write the C getters and setter
    myself? In that case, there would be no point in using SWIG, if all it
    did was to type out the data fields of the classes. The majority of
    the work in my case is writing the C getters and setters and the C JNI
    code for calling the equivalent Java methods.

    Unless I am missing something completely, and the generated code is
    indeed what I need. Please point out if so :)

    Thanks
    Aditya Rajgarhia, Jul 10, 2008
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jorgen Gustafsson
    Replies:
    4
    Views:
    565
    Jorgen Gustafsson
    Dec 12, 2003
  2. =?Utf-8?B?TGFycnkgR3JhZHk=?=

    Complex GirdView Problem, returning values from dynamic TemplateFi

    =?Utf-8?B?TGFycnkgR3JhZHk=?=, Sep 9, 2005, in forum: ASP .Net
    Replies:
    4
    Views:
    2,239
    S. Justin Gengo
    Sep 9, 2005
  3. Vikas
    Replies:
    2
    Views:
    14,325
    Vikas
    Aug 27, 2003
  4. news.amnet.net.au
    Replies:
    1
    Views:
    553
    =?UTF-8?b?TMSByrtpZSBUZWNoaWU=?=
    Apr 13, 2004
  5. Victor Bazarov
    Replies:
    25
    Views:
    838
    E. Robert Tisdale
    Mar 23, 2005
Loading...

Share This Page