Looking for Comments/Thoughts on this approach

[ohaya]

Hi,

I have a program that currently writes to a file using PrintStream, and
I'll be needing to extend the functionality of the program to do some
post-output processing of the file, and if possible, I want to minimize
changes to the existing code.

Initially, I was thinking of just writing a separate program to read the
file that was output by the existing program, but then I started
wondering if there was some way to kind of redirect the existing output
to some kind of in-memory buffer, and then the new part could input from
the buffer.

What I'm thinking about is building a wrapper around PipeOutputStream to
add a 'println()' method. That way, with minimal code changes, the
original could would 'think' it was writing to a file, but it would
actually be writing to a PipeOutputStream.

Then, the new code could read the information created by the original
code by using a PipeInputStream.

I've never used PipeInput/OutputStream before, and so before I go about
this, I was hoping that I could get some comments on this approach? I
also have several concerns:

1) In several articles that I've read, it was also indicated that the
performance of PipeInput/Output was bad, and

2) All the documents I've read seem to indicate that PipeInputStream and
PipeOutputStream are intended to be used "between threads". In my case,
that wouldn't be the case, since the main thread would complete all
writing to the PipeOutputStream, and then start reading from the
PipeInputStream. Is this going to be a problem?

3) It seems like the PipeInputStream blocks on reads if there's no
input. If this is the case, how do I signal an "EOF-like" condition so
that the program doesn't hang when it gets to the end of the
information? Is the only way to do this for the writing end to put some
kind of indicator in the pipe that signals "EOF"?

4) Is there any memory limitation that will prevent this from working
(the 'output file' can be quite large)?


If this isn't a good approach, are there any other mechanisms/approaches
that might be better?

Thanks,
Jim

 

[Roedy Green]

I've never used PipeInput/OutputStream before, and so before I go about
this, I was hoping that I could get some comments on this approach? I
also have several concerns:

ask the file I/O Amanunesis to generate you some code to do just that.

See http://mindprod.com/fileio.html

 

[Michael Borgwardt]

2) All the documents I've read seem to indicate that PipeInputStream and
PipeOutputStream are intended to be used "between threads". In my case,
that wouldn't be the case, since the main thread would complete all
writing to the PipeOutputStream, and then start reading from the
PipeInputStream. Is this going to be a problem?

Yes. Basically, PipedInputStream has a buffer of 1024 bytes in
size. If you write more than that to the connected PipedOutputStream
the call will block until you read from the InputStream. i.e. you'll
have a deadlock.

3) It seems like the PipeInputStream blocks on reads if there's no
input. If this is the case, how do I signal an "EOF-like" condition so
that the program doesn't hang when it gets to the end of the
information?

Call close() on the OutputStream.

4) Is there any memory limitation that will prevent this from working
(the 'output file' can be quite large)?

It won't work at all unless you have two threads working on both ends
of the pipe simultaneously.

 

[Cristiano Sadun]

Hi,

I have a program that currently writes to a file using PrintStream...

If you can afford to keep all the results in memory and don't have to
process 'em as they come (in which case pipes are your best shot), you can
use a simple "tie" approach - a tie stream/writer which is built over two
stream/writers and duplicates the output to both.

If one of the two is a PrintWriter over a StringWriter (or a PrintStream
over a ByteArrayOutputStream), after the processing you'll have a String to
process with textual methods.

You find an example at

http://sadun-util.sourceforge.net/api/org/sadun/util/TieOutputStream.html

with which you can write, for example

ByteArrayOutputStream os = new ByteArrayOutputStream();
PrintStream stream = new PrintStream(new TieOutputStream(System.out, os));
stream.println("Hello");
....
String txt = os.toString();
... do textual processing on txt..

 

[Stephen Ostermiller]

There are two other approaches.

1) Wrap a StringWriter in a PrintWriter (to give it the println()
method) and have your program write to the StringWriter. The use the
toString method of the StringWriter to get the data. Do your post
processing on the String (you might want to consider creating a
StringReader on the data to work with another stream) and then save it
to a file.

2) Use a Circular Buffer:
http://ostermiller.org/utils/CircularBuffer.html
Set the size of the CircularBuffer to INFINITE, have your program
write to the circular buffer (it is just a stream like it would
expect) and then get the other stream that allows you to read from the
circular buffer, do your post processing, and save it to the file.

Bot of these approaches save the entire file in memory, so the file
cannot be huge for this to work.

Stephen