Redirecting IO from C++ native library

[Josef Svitak]

Wondering if there is any way to redirect the IO from a library loaded
with System.loadLibrary()? The library is written in C++.

Thanks,
josef

 

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Wondering if there is any way to redirect the IO from a library loaded
with System.loadLibrary()? The library is written in C++.

Have you tried the obvious:
* create a new DLL/SO
* have your Java code:
- call the new DLL/SO and redirect stdout
- call the real DLL/SO
- call the new DLL/SO and reenable stdout
?

Arne

 

[Mark Space]

Wondering if there is any way to redirect the IO from a library loaded
with System.loadLibrary()? The library is written in C++.

Thanks,
josef

shouldn't any native code called from a Java process use the same
standard IO as the Java process? That is, shouldn't the IO descriptors
be inherited from the Java process by the native C++ lib?

I mean, I don't know, so maybe not. But it seems obvious. Try
redirecting the IO from Java, see if the native lib follows. You may
need to redirect before you even load the library (the DLL might contain
initialization code, which could be called on load.

If not, I guess you'll have to dig into how standard IO is allocated on
your system. There's also a thing on Unix called a co-process which
allocates all IO back to the parent process. It's a form inter-process
communication (IPC). It's really old too, so I don't think it's much
used anymore, but you might look into it. Many windows IPC and process
spawn calls mimic common Unix functions.

After a quick Goggle search, coprocesses seem possible but tricky under
windows. Use with care...

 

[Josef Svitak]

Have you tried the obvious:
* create a new DLL/SO
* have your Java code:
- call the new DLL/SO and redirect stdout
- call the real DLL/SO
- call the new DLL/SO and reenable stdout
?

Maybe this is too obvious. I tried:
....
piOut = new PipedInputStream();
poOut = new PipedOutputStream(piOut);
System.setOut(new PrintStream(poOut));
....

Then I make a reader thread to read from piOut to display the results in
a JTextArea.

Not sure what you mean by _new_ DLL/SO.

Thanks,
josef

 

[Josef Svitak]

shouldn't any native code called from a Java process use the same
standard IO as the Java process? That is, shouldn't the IO descriptors
be inherited from the Java process by the native C++ lib?

I mean, I don't know, so maybe not. But it seems obvious. Try
redirecting the IO from Java, see if the native lib follows. You may
need to redirect before you even load the library (the DLL might contain
initialization code, which could be called on load.

If not, I guess you'll have to dig into how standard IO is allocated on
your system. There's also a thing on Unix called a co-process which
allocates all IO back to the parent process. It's a form inter-process
communication (IPC). It's really old too, so I don't think it's much
used anymore, but you might look into it. Many windows IPC and process
spawn calls mimic common Unix functions.

After a quick Goggle search, coprocesses seem possible but tricky under
windows. Use with care...

Everything is in one process AFAIK. I've tried redirecting before load,
after load and both with no luck. To simplify:

try {
System.setOut(
new PrintStream(new java.io.FileOutputStream("goo.out")));
System.setErr(
new PrintStream(new java.io.FileOutputStream("goo.err")));
} catch (java.io.IOException e) {
System.out.println("Couldn't redirect STDOUT: \n"+e.getMessage());
}

System.loadLibrary("nativelib");

 

[Mark Space]

Everything is in one process AFAIK. I've tried redirecting before load,
after load and both with no luck. To simplify:

try {
System.setOut(
new PrintStream(new java.io.FileOutputStream("goo.out")));
System.setErr(
new PrintStream(new java.io.FileOutputStream("goo.err")));
} catch (java.io.IOException e) {
System.out.println("Couldn't redirect STDOUT: \n"+e.getMessage());
}

System.loadLibrary("nativelib");

Strange. Sorry I can't help you more, I don't have time to investigate
this. But it seems like it should work, so maybe if you keep asking
you'll find an answer. I'd check with the OS folks (Windows news
group), and ask if a DLL does anything with the IO streams when it's
loaded. Then try to find source for the lib, see if one of the specific
C++ classes also does anything funky.

 

[Chris Uppal]

piOut = new PipedInputStream();
poOut = new PipedOutputStream(piOut);
System.setOut(new PrintStream(poOut));

(Are you the same person who was asking about this for interfacing with CLIPS a
little while ago?)

I doubt if that -- or anything else -- will work.

Consider: when a C program writes something to stdout, the output is put into
a data structure which is local to that program. When the buffer is flushed,
the data is sent to a low-level OS-dependent "place" which is that program's
standard output. (File descriptor 1 on Unix, something more exotic on
Windows). Now if a program consists of two parts, a main program and a DLL it
has loaded, then there is not even any guarantee that they use the same
datastructure to hold/buffer the data (they could be written in different
lanugages), so although output from either half of the program goes to the same
OS-level "place", there is no way that either can replace, or in any way
affect, what the other "thinks" is its standard output. That picture applies
in Java. If you replace System.out with a different stream, that is just
playing around with the objects that the JVM knows about. It doesn't affect
the OS in any way, and it is completely invisible to a DLL which is written in
a different language and which has its own IO mechanisms.

The only way that this sort of thing could be done is if on-the-fly redirection
were built into the OS itself (and the JVM knew how to use it), or if the DLL
itself provides a way to tell it to use a different output stream. The first
possibility isn't available on any OS/JVM combination that I know of. The
second is quite common in well-designed libraries.

-- chris

 

[Josef Svitak]

Thanks for your considered reply!

(Are you the same person who was asking about this for interfacing with CLIPS a
little while ago?)
Nope.


I doubt if that -- or anything else -- will work.

Right. I think I've come around to that view...

Consider: when a C program writes something to stdout, the output is put into
a data structure which is local to that program. When the buffer is flushed,
the data is sent to a low-level OS-dependent "place" which is that program's
standard output. (File descriptor 1 on Unix, something more exotic on
Windows). Now if a program consists of two parts, a main program and a DLL it
has loaded, then there is not even any guarantee that they use the same
datastructure to hold/buffer the data (they could be written in different
lanugages), so although output from either half of the program goes to the same
OS-level "place", there is no way that either can replace, or in any way
affect, what the other "thinks" is its standard output. That picture applies
in Java. If you replace System.out with a different stream, that is just
playing around with the objects that the JVM knows about. It doesn't affect
the OS in any way, and it is completely invisible to a DLL which is written in
a different language and which has its own IO mechanisms.

But that doesn't explain why it is possible to redirect from any
_process_, regardless of what it's implemented in. The
Process.get<In|Output|Error>Stream functions work fine. It seems only
sensible that they would try to redirect at the OS level rather than
tinker with the abstractions introduced for any given language.

The only way that this sort of thing could be done is if on-the-fly redirection
were built into the OS itself (and the JVM knew how to use it), or if the DLL
itself provides a way to tell it to use a different output stream. The first
possibility isn't available on any OS/JVM combination that I know of. The
second is quite common in well-designed libraries.

Ah, yes. If only we all were able to deal with just well-designed
libraries (sigh).

Thanks again,
josef

 

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Right. I think I've come around to that view...

If you have access to the same C/C++ compiler
as the DLL you want to capture from, then there
may be some possibilities.

Look at this example. I am using Windows and
Mingw C compiler.

First we emulate the current DLL you have.

Nat1.java:

public class Nat1 {
public native void hello();
static {
System.loadLibrary("Nat1");
}
}

Nat1.c:

#include <stdio.h>

#include <jni.h>

#include "Nat1.h"

JNIEXPORT void JNICALL Java_Nat1_hello(JNIEnv *cntx, jobject me)
{
printf("Hello world\n");
}

A true classic.

Now to capture the output from this DLL we create a new DLL.

Nat2.java:

public class Nat2 {
public native void init();
public native String done();
static {
System.loadLibrary("Nat2");
}
}

Nat2.c:

#include <stdio.h>

#include <fcntl.h>
#include <io.h>

#include <jni.h>

#include "Nat1.h"

#define MAXBUF 20480

static int oldstdout;
static int pipehandles[2];
static char buf[MAXBUF];

JNIEXPORT void JNICALL Java_Nat2_init(JNIEnv *cntx, jobject me)
{
oldstdout = _dup(_fileno(stdout));
_pipe(pipehandles,MAXBUF,_O_BINARY);
_dup2(pipehandles[1],_fileno(stdout));
}

JNIEXPORT jstring JNICALL Java_Nat2_done(JNIEnv *cntx, jobject me)
{
int n;
fflush(stdout);
_dup2(oldstdout,_fileno(stdout));
n = _read(pipehandles[0],buf,sizeof(buf));
buf[n] = '\0';
return (*cntx)->NewStringUTF(cntx,buf);
}

Now we just need a test program.

TestProgram.java:

public class TestProgram {
public static void main(String[] args) throws Exception {
Nat2 n2 = new Nat2();
n2.init();
Nat1 n1 = new Nat1();
n1.hello();
n1.hello();
n1.hello();
System.out.println("#" + n2.done() + "#");
}
}

Building and running:

javac -classpath . Nat1.java
javah -classpath . -jni Nat1
gcc -c -I\sunjava\jdk1.5.0\include -I\sunjava\jdk1.5.0\include\win32
Nat1.c -o Nat1.obj
gcc -s -shared -Wl,--export-all,--kill-at Nat1.obj -o Nat1.dll
javac -classpath . Nat2.java
javah -classpath . -jni Nat2
gcc -c -I\sunjava\jdk1.5.0\include -I\sunjava\jdk1.5.0\include\win32
Nat2.c -o Nat2.obj
gcc -s -shared -Wl,--export-all,--kill-at Nat2.obj -o Nat2.dll
javac -classpath . TestProgram.java
path=.;%PATH%
java -classpath . TestProgram

And the output:

#Hello world
Hello world
Hello world
#

The method should work with C++ IO as well (cout instead of stdout).

This technique requires:
- DLL's being dynamicly linked against the C/C++ RTL
- DLL's being compiled with the same compiler

So it may not be possible for you.

But on the other hand it may be possible !

Arne