constructing a graph (linking streams)

Discussion in 'Java' started by Martijn, Apr 26, 2007.

  1. Martijn

    Martijn Guest

    Hi,

    I want to create a graph where output from one component is input for the
    next. The components are invoked serially, so one component needs to be
    done before the next one kicks in:

    +--------+ +--------+ +--------+
    ====> | COMP 1 | ====> | COMP 2 | ====> | COMP 3 | ====>
    +--------+ +--------+ +--------+

    Lets consider all streams to be byte streams to keep it simple (or char
    streams if that would help). What I have done now is create an array, e.g.
    byte[] buf and simply use a ByteArrayOutputStream and copy the output into
    it, and create a new ByteArrayInputStream (I might be able to reset it, but
    I figured the overhead would be minimal, as it was using the same buffer
    anyway, so I chose to play it safe).

    In code (forgive me for any mistakes):

    byte[] buf;
    ByteArrayInputStream istream = new ByteArrayInputStream(buf);
    ByteArrayOutputStream ostream = new ByteArrayOutputStream();

    // init the buffer (from file, which happens to be a component with no
    input)

    Comp1(ostream, istream);

    buf = ostream.toByteArray(); // this invalidates the original buffer,
    right?
    ostream.reset();

    Comp2(ostream, istream);

    // etc...

    But this invalidates my buffer each time and requires an extra copy phase I
    would like to get rid of. Any more appropriate solution available for this?
    Which classes should I have a look at? Any pointers are greatly
    appreciated!

    Thanks for the help,

    --
    Martijn
    http://www.sereneconcepts.nl
     
    Martijn, Apr 26, 2007
    #1
    1. Advertising

  2. Martijn wrote:
    > Hi,
    >
    > I want to create a graph where output from one component is input for the
    > next. The components are invoked serially, so one component needs to be
    > done before the next one kicks in:
    >
    > +--------+ +--------+ +--------+
    > ====> | COMP 1 | ====> | COMP 2 | ====> | COMP 3 | ====>
    > +--------+ +--------+ +--------+
    >
    > Lets consider all streams to be byte streams to keep it simple (or char
    > streams if that would help). What I have done now is create an array, e.g.
    > byte[] buf and simply use a ByteArrayOutputStream and copy the output into
    > it, and create a new ByteArrayInputStream (I might be able to reset it, but
    > I figured the overhead would be minimal, as it was using the same buffer
    > anyway, so I chose to play it safe).
    >
    > But this invalidates my buffer each time and requires an extra copy phase I
    > would like to get rid of. Any more appropriate solution available for this?
    > Which classes should I have a look at? Any pointers are greatly
    > appreciated!
    >
    > Thanks for the help,
    >


    What I tend to prefer for chained parsers is to define new InputStreams
    that take the previous one as a source, like so:

    Step1Transformer s1t = new Step1Transformer(input);
    Step2Transformer s2t = new Step2Transformer(s1t);
    byte[] data = s2t.getDataHowever(); // Internally invokes s1t's parser.
     
    Joshua Cranmer, Apr 28, 2007
    #2
    1. Advertising

  3. Martijn

    Martijn Guest

    Joshua Cranmer wrote:
    > Martijn wrote:
    >> Hi,
    >>
    >> I want to create a graph where output from one component is input
    >> for the next. The components are invoked serially, so one component
    >> needs to be done before the next one kicks in:
    >>
    >> +--------+ +--------+ +--------+
    >> ====> | COMP 1 | ====> | COMP 2 | ====> | COMP 3 | ====>
    >> +--------+ +--------+ +--------+
    >>
    >> Lets consider all streams to be byte streams to keep it simple (or
    >> char streams if that would help). What I have done now is create an
    >> array, e.g. byte[] buf and simply use a ByteArrayOutputStream and
    >> copy the output into it, and create a new ByteArrayInputStream (I
    >> might be able to reset it, but I figured the overhead would be
    >> minimal, as it was using the same buffer anyway, so I chose to play
    >> it safe). But this invalidates my buffer each time and requires an extra
    >> copy
    >> phase I would like to get rid of. Any more appropriate solution
    >> available for this? Which classes should I have a look at? Any
    >> pointers are greatly appreciated!
    >>
    >> Thanks for the help,
    >>

    >
    > What I tend to prefer for chained parsers is to define new
    > InputStreams that take the previous one as a source, like so:
    >
    > Step1Transformer s1t = new Step1Transformer(input);
    > Step2Transformer s2t = new Step2Transformer(s1t);
    > byte[] data = s2t.getDataHowever(); // Internally invokes s1t's
    > parser.


    Hi Joshua,

    that sounds like a really good idea. It is similar to two approaches I had
    taken, but much simpeler!

    1) Each component had a reader as an argument and returned a
    reader which could be fed to the next component (this is
    what I have finally chosen, but I might replace it with
    your approach at a later time)

    2) Create a callback-wrapper class that has an abstract
    method in the form of comp1(reader, writer), then chain
    the graph in a graph object using pipes. This was only
    tested, but not fully implemented.

    Thanks for the tip!

    --
    Martijn
    http://www.sereneconcepts.nl
     
    Martijn, Apr 29, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. George Sakkis
    Replies:
    1
    Views:
    473
    Szabolcs Nagy
    Jan 29, 2007
  2. Dr Ann Huxtable

    Missing Graph.h and (Graph.lib) woes - any help

    Dr Ann Huxtable, Dec 21, 2004, in forum: C Programming
    Replies:
    6
    Views:
    669
    Dr Ann Huxtable
    Dec 21, 2004
  3. Jef Driesen
    Replies:
    3
    Views:
    2,609
    mlimber
    Jan 24, 2006
  4. Almoni
    Replies:
    0
    Views:
    3,136
    Almoni
    Jan 17, 2010
  5. Emilio Mayorga
    Replies:
    6
    Views:
    375
    Martien Verbruggen
    Oct 8, 2003
Loading...

Share This Page