constructing a graph (linking streams)

M

Martijn

Hi,

I want to create a graph where output from one component is input for the
next. The components are invoked serially, so one component needs to be
done before the next one kicks in:

+--------+ +--------+ +--------+
====> | COMP 1 | ====> | COMP 2 | ====> | COMP 3 | ====>
+--------+ +--------+ +--------+

Lets consider all streams to be byte streams to keep it simple (or char
streams if that would help). What I have done now is create an array, e.g.
byte[] buf and simply use a ByteArrayOutputStream and copy the output into
it, and create a new ByteArrayInputStream (I might be able to reset it, but
I figured the overhead would be minimal, as it was using the same buffer
anyway, so I chose to play it safe).

In code (forgive me for any mistakes):

byte[] buf;
ByteArrayInputStream istream = new ByteArrayInputStream(buf);
ByteArrayOutputStream ostream = new ByteArrayOutputStream();

// init the buffer (from file, which happens to be a component with no
input)

Comp1(ostream, istream);

buf = ostream.toByteArray(); // this invalidates the original buffer,
right?
ostream.reset();

Comp2(ostream, istream);

// etc...

But this invalidates my buffer each time and requires an extra copy phase I
would like to get rid of. Any more appropriate solution available for this?
Which classes should I have a look at? Any pointers are greatly
appreciated!

Thanks for the help,
 
J

Joshua Cranmer

Martijn said:
Hi,

I want to create a graph where output from one component is input for the
next. The components are invoked serially, so one component needs to be
done before the next one kicks in:

+--------+ +--------+ +--------+
====> | COMP 1 | ====> | COMP 2 | ====> | COMP 3 | ====>
+--------+ +--------+ +--------+

Lets consider all streams to be byte streams to keep it simple (or char
streams if that would help). What I have done now is create an array, e.g.
byte[] buf and simply use a ByteArrayOutputStream and copy the output into
it, and create a new ByteArrayInputStream (I might be able to reset it, but
I figured the overhead would be minimal, as it was using the same buffer
anyway, so I chose to play it safe).

But this invalidates my buffer each time and requires an extra copy phase I
would like to get rid of. Any more appropriate solution available for this?
Which classes should I have a look at? Any pointers are greatly
appreciated!

Thanks for the help,

What I tend to prefer for chained parsers is to define new InputStreams
that take the previous one as a source, like so:

Step1Transformer s1t = new Step1Transformer(input);
Step2Transformer s2t = new Step2Transformer(s1t);
byte[] data = s2t.getDataHowever(); // Internally invokes s1t's parser.
 
M

Martijn

Joshua said:
Martijn said:
Hi,

I want to create a graph where output from one component is input
for the next. The components are invoked serially, so one component
needs to be done before the next one kicks in:

+--------+ +--------+ +--------+
====> | COMP 1 | ====> | COMP 2 | ====> | COMP 3 | ====>
+--------+ +--------+ +--------+

Lets consider all streams to be byte streams to keep it simple (or
char streams if that would help). What I have done now is create an
array, e.g. byte[] buf and simply use a ByteArrayOutputStream and
copy the output into it, and create a new ByteArrayInputStream (I
might be able to reset it, but I figured the overhead would be
minimal, as it was using the same buffer anyway, so I chose to play
it safe). But this invalidates my buffer each time and requires an extra
copy
phase I would like to get rid of. Any more appropriate solution
available for this? Which classes should I have a look at? Any
pointers are greatly appreciated!

Thanks for the help,

What I tend to prefer for chained parsers is to define new
InputStreams that take the previous one as a source, like so:

Step1Transformer s1t = new Step1Transformer(input);
Step2Transformer s2t = new Step2Transformer(s1t);
byte[] data = s2t.getDataHowever(); // Internally invokes s1t's
parser.

Hi Joshua,

that sounds like a really good idea. It is similar to two approaches I had
taken, but much simpeler!

1) Each component had a reader as an argument and returned a
reader which could be fed to the next component (this is
what I have finally chosen, but I might replace it with
your approach at a later time)

2) Create a callback-wrapper class that has an abstract
method in the form of comp1(reader, writer), then chain
the graph in a graph object using pipes. This was only
tested, but not fully implemented.

Thanks for the tip!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top