Forking Inputstream: Am I missing something

Discussion in 'Java' started by Dennis, Mar 17, 2010.

  1. Dennis

    Dennis Guest

    Dear all,

    This has been puzzling me all morning: There is reasonable elegant way (see example below) to log InputStreams and OutputStreams, without consuming the streams. However, I cannot find any reference to it or a good implementation of it anywhere. I tested it and it seems to work, but I have the nagging idea that I'm missing something, or I did miss all the references to it on the internet.

    public class ForkedInputStream extends InputStream {

    private InputStream m_source;
    private OutputStream m_fork;

    public ForkedInputStream(InputStream source, OutputStream fork) {
    m_source = source;
    m_fork = fork;
    }

    public int read() throws IOException {
    int b = m_source.read();
    if (b==-1) {
    m_fork.flush();
    } else {
    m_fork.write(b);
    }

    return b;
    }

    public int available() throws IOException {
    return m_source.available();
    }

    public void close() throws IOException {
    m_fork.close();
    }
    }

    You can feed in your original InputStream to ForkedInputStream and use the instance of ForkedInputStream as you would the original stream. The OutputStream will now contain the same information as read from the InputStream.

    Off course you could enhance it with Threading, Buffers and a check to only write at certain (log) levels
    This would take care of the only two issues I can think of, but the principle stays the same:
    - A blocked output would block the read() operation.
    - Writing to the OutputStream takes time, so you might not have it 'on' in every situation.
    Or as the title suggests: Am I missing something?

    Kind regards,
    Dennis
    Brains2B.org
     
    Dennis, Mar 17, 2010
    #1
    1. Advertising

  2. Dennis

    markspace Guest

    Dennis wrote:

    > public void close() throws IOException {
    > m_fork.close();


    You probably should close the input stream here too.

    > }



    Aside from that the only comment I have is that this will be very
    inefficient, since you don't override read( byte[], int, int ) and
    therefore can't read more than one byte (via the method you do override,
    read()) at a time.

    Next step: override read(byte[]) and read(byte[],int,int).
     
    markspace, Mar 17, 2010
    #2
    1. Advertising

  3. Dennis

    EJP Guest

    On 17/03/2010 11:59 PM, Dennis wrote:
    >
    > public class ForkedInputStream extends InputStream {


    == java.io.SequenceInputStream, with more bugs
     
    EJP, Mar 18, 2010
    #3
  4. Dennis

    dennis Guest

    Hi markspace,

    >Aside from that the only comment I have is that this will be very
    >inefficient, since you don't override read( byte[], int, int ) and
    >therefore can't read more than one byte (via the method you do override,
    >read()) at a time.
    >Next step: override read(byte[]) and read(byte[],int,int).


    Thanks for your comments. I left the other read implementations out in the example for clarity but you are right they should be there. The close was just me being to hasty.

    Kind regards,
    Dennis
     
    dennis, Mar 18, 2010
    #4
  5. Dennis

    dennis Guest

    >On 17/03/2010 11:59 PM, Dennis wrote:
    >>
    >> public class ForkedInputStream extends InputStream {

    >== java.io.SequenceInputStream, with more bugs


    Hi,

    Thanks for the pointer to SequenceInputStream. However SequenceInputStream just takes InputStream and there is no SequenceOutputStream. Which gets met back to the original question: what am I missing if 'forking' OutputStreams is not done.

    What bugs did you find?

    Kind regards,
    Dennis
     
    dennis, Mar 18, 2010
    #5
  6. Dennis

    Ian Shef Guest

    EJP <> wrote in news:TGmon.13637$pv.10376
    @news-server.bigpond.net.au:

    > On 17/03/2010 11:59 PM, Dennis wrote:
    >>
    >> public class ForkedInputStream extends InputStream {

    >
    > == java.io.SequenceInputStream, with more bugs


    I don't think so.
    You might be correct about the bugs, but not about the equivalence.

    From the Javadocs:
    "A SequenceInputStream represents the logical concatenation of other input
    streams."

    A SequenceInputStream performs concatenation. A ForkedInputStream provides
    what I would call a Tee. It provides input, and also echoes that input to
    the designated OutputStream.

    --------

    The original poster might want to look at
    http://commons.apache.org/io/apidocs/org/apache/commons/io/input/TeeInputStre
    am.html

    for another implementation of this idea. It has more features and possibly
    fewer bugs.

    The sources are open, for some definition of "open".

    See
    http://svn.apache.org/viewvc/commons/proper/io/trunk/src/java/org/apache/comm
    ons/io/input/TeeInputStream.java?view=markup
     
    Ian Shef, Mar 19, 2010
    #6
  7. Dennis wrote:
    > Dear all,
    >
    > This has been puzzling me all morning: There is reasonable elegant
    > way (see example below) to log InputStreams and OutputStreams,
    > without consuming the streams. However, I cannot find any reference
    > to it or a good implementation of it anywhere. I tested it and it
    > seems to work, but I have the nagging idea that I'm missing
    > something, or I did miss all the references to it on the internet.


    I'd probably extend FilterInputStream and just call super() for all the
    processing of the "main" stream e,g,

    public class TeeInputStream extends FilterInputStream
    {
    private OutputStream m_tee;

    public TeeInputStream(InputStream in, OutputStream fork)
    {
    super(in);
    }

    public int read() throws IOException
    {
    int c = super.read();
    if (b == -1)
    m_tee.flush();
    else
    m_tee.write(c);
    }

    public void close() throws IOException
    {
    super.close();
    m_tee.close();
    }

    // etc.
    }
     
    Mike Schilling, Mar 19, 2010
    #7
  8. Dennis

    EJP Guest

    On 18/03/2010 8:49 PM, EJP wrote:
    > == java.io.SequenceInputStream, with more bugs


    Sorry, I am crazy, it is actually more like tee(1). I wrote a
    TeeInputStream at some point about 10 years ago, wonder where it is.
     
    EJP, Mar 19, 2010
    #8
  9. Dennis

    dennis Guest

    >The original poster might want to look at
    >http://commons.apache.org/io/apidocs/org/apache/commons/io/input/TeeInputStre
    >am.html
    >for another implementation of this idea. It has more features and possibly
    >fewer bugs.


    Hi Ian,

    Thanks for the reference. This was helpfull. At least now I know it has been done before (and with no unexpected side effects).
    I'm not sure yet if I want to use that one directly, or use my own implementation:
    - One they are using ProxyInputStream which possibly allows for seek(),mark() and reset() of the underlying stream and this could mean part of the output would not be written multiple times or not at all. You either have to handle that or at least mark the output that there was a part not written or written multiple times. Especially when you use it for logging or storing xml-requests
    - The license issue. If I use mine it will be done under the modified BSD-license so no restrictions to future users.

    Kind regards,
    Dennis
     
    dennis, Mar 20, 2010
    #9
  10. Dennis

    dennis Guest

    Hi Mike,

    Thanks for the feedback. I looked into FilterInputStream, but did not want to use it while the underlying stream might support seek(), mark() and reset() and this will prevent data being written to the output or data being written multiple times. You either have to handle that or at least mark the output that there was a part not written or writtenmore then once. Especially when you use it for logging or storing xml-requests.

    Kind regards,
    Dennis..
     
    dennis, Mar 20, 2010
    #10
  11. dennis wrote:
    > Hi Mike,
    >
    > Thanks for the feedback. I looked into FilterInputStream, but did not
    > want to use it while the underlying stream might support seek(),
    > mark() and reset() and this will prevent data being written to the
    > output or data being written multiple times. You either have to
    > handle that or at least mark the output that there was a part not
    > written or writtenmore then once. Especially when you use it for
    > logging or storing xml-requests.



    I must be missing something. Whether you override FilterInpoutStrewam or
    not, you still have the ability to override mark(), markSupported(), skip(),
    and reset() yourself and do whatever you think is best.
     
    Mike Schilling, Mar 20, 2010
    #11
  12. Dennis

    Lew Guest

    dennis wrote:
    > - The license issue. If I use mine it will be done under the modified
    > BSD-license so no restrictions to future users.


    Of what restrictions do you speak? The point of open-source licenses is that
    they *remove* restrictions from future users.

    The BSD license and the Apache license aren't very far apart in their
    consequences.

    I really am interested in what negative consequences you anticipate from the
    Apache license. What are they?

    --
    Lew
     
    Lew, Mar 20, 2010
    #12
  13. In article <ho33il$g7e$>, Lew <>
    wrote:

    > dennis wrote:
    > > - The license issue. If I use mine it will be done under the modified
    > > BSD-license so no restrictions to future users.

    >
    > Of what restrictions do you speak? The point of open-source licenses
    > is that they *remove* restrictions from future users.
    >
    > The BSD license and the Apache license aren't very far apart in their
    > consequences.
    >
    > I really am interested in what negative consequences you anticipate from the
    > Apache license. What are they?


    I would like to hear more on this, too. In particular, which modified
    BSD-license?

    <http://en.wikipedia.org/wiki/BSD_licenses>

    --
    John B. Matthews
    trashgod at gmail dot com
    <http://sites.google.com/site/drjohnbmatthews>
     
    John B. Matthews, Mar 20, 2010
    #13
  14. Dennis

    dennis Guest

    Hi Lew,

    >Of what restrictions do you speak? The point of open-source licenses is that
    >they *remove* restrictions from future users.


    >The BSD license and the Apache license aren't very far apart in their
    >consequences.


    >I really am interested in what negative consequences you anticipate from the
    >Apache license. What are they?
    >--
    >Lew


    First let me clarify something. With users in this case I mean people or customers using the sources and not only the compiled code. For the later no real restrictions do apply. For the people using and deriving of the sources restrictions do apply.
    The most notable is from the GPL: All derived works should also be open source. http://www.opensource.org/licenses/gpl-2.0.php
    The Apache license has just one restriction that has some of my customers worried: If you have a NOTICE in your software and made a derivitive work from any Apache code you have to mention Apache in that notice. http://www.apache.org/licenses/LICENSE-2.0.html
    A comparable advertising restriction was in the original license from BSD as well, but was deleted in 1999. Hence the name 'modified' BSD license. Although me stil referring to 'modified' 11 years after it was done, might indicate I'm getting old ;-) http://www.opensource.org/licenses/bsd-license.php

    I use some open source software in products I create for my customers. After I leave they should be able to extend or alter the work I have done or the open source software it relies on, without being required to release it as open source or advertise other organisations, not even my own.

    Don't get me wrong. I have nothing against Apache, they have some great stuff out there and I use it regularly. If I need to fix something in their code I will supply it back to them for possible commitment in future releases. For my customers it is a different matter. They cannot derive works of it, without complying to the notice part of the license, or supplying it back to Apache. Something they might not be willing to do.

    I hope this answers your questions.

    Kind regards,
    Dennis Groenendijk
     
    dennis, Mar 22, 2010
    #14
  15. Dennis

    dennis Guest

    Hi Mike,

    >I must be missing something. Whether you override FilterInpoutStrewam or
    >not, you still have the ability to override mark(), markSupported(), skip(),
    >and reset() yourself and do whatever you think is best.


    Of course I can overwrite FilterInputStreams methods to do whatever I want. InputStream itself however had all the functionality in these methods I needed to make sure all bytes are read and written just once. It was thus more efficient to just use InputStream.

    Kind regards,
    Dennis
     
    dennis, Mar 22, 2010
    #15
  16. Dennis

    Lew Guest

    Lew wrote:
    >> Of what restrictions do you speak?  The point of open-source licenses is that
    >> they *remove* restrictions from future users.
    >> The BSD license and the Apache license aren't very far apart in their
    >> consequences.
    >> I really am interested in what negative consequences you anticipate from the
    >> Apache license.  What are they?


    dennis wrote:
    > First let me clarify something. With users in this case I mean people or customers using the sources
    > and not only the compiled code. For the later no real restrictions do apply. For the people using and
    > deriving of the sources restrictions do apply.
    > The most notable is from the GPL: All derived works should also be open source.http://www.opensource.org/licenses/gpl-2.0.php


    I asked about the Apache license.

    > The Apache license has just one restriction that has some of my customers worried:


    Your customers are Nervous Nellies.

    Um, IMHO.

    > If you have a NOTICE in your software and made a derivitive work from any Apache code you have
    > to mention Apache in that notice.http://www.apache.org/licenses/LICENSE-2..0.html
    > A comparable advertising restriction was in the original license from BSD as well,
    > but was deleted in 1999. Hence the name


    They're not comparable at all. The BSD license referred to
    advertising, as you mention; the Apache license does not. There is no
    requirement in the Apache license to mention Apache in your
    advertising. All you have to do is put copyright attributions (*not*
    advertisements) in the NOTICE file and source code of your
    application. They don't have to appear in the application interface,
    splash screen or anywhere else but in the copyright notices. The BSD
    license is similar in this respect - "Redistributions of source code
    must retain the above copyright notice, this list of conditions and
    the following disclaimer." Tomayto, tomahto.

    <http://www.apache.org/licenses/>
    <http://www.opensource.org/licenses/bsd-license.php>

    > I use some open source software in products I create for my customers. After I leave
    > they should be able to extend or alter the work I have done or the open source software it relies on,
    > without being required to release it as open source or advertise other organisations, not even my own.


    Again, the Apache license does not say anything whatsoever about
    advertising; it does not impose any restrictions or requirements that
    the copyright holder be mentioned in advertising.

    > Don't get me wrong. I have nothing against Apache, they have some great stuff out there and I use it regularly.
    > If I need to fix something in their code I will supply it back to them for possible commitment in future releases.
    > For my customers it is a different matter. They cannot derive works of it, without complying to the notice
    > part of the license, or supplying it back to Apache. Something they might not be willing to do.


    There is absolutely no requirement in the Apache license to submit
    work back to the copyright holder, much less Apache. Wherever did you
    get that notion?

    It's your choice, but personally I don't see the problem with
    requiring that copyright notices include the copyright information
    from all copyright holders. It's hardly a marketing question, and
    barely a restriction. After all, you're requiring that with the BSD
    license, too. With respect to copyright attribution, there's hardly
    any difference between the Apache license and the BSD or MIT licenses.

    Why are you even open-sourcing your software given your concerns?

    --
    Lew
     
    Lew, Mar 22, 2010
    #16
  17. Dennis

    dennis Guest

    Hi Lew,

    I found your response a bit strange. This were all answers to questions you yourself asked ?

    >Your customers are Nervous Nellies.
    >Um, IMHO.

    No actually they are not. Some are actually very good at taking risks. The ones that make them a profit. Some of them might have legal departments that spell any license agreement. I even have customers who contractually bind you to the fact that they will not accept any liability coming from licenses.

    >They're not comparable at all. The BSD license referred to
    >advertising, as you mention; the Apache license does not. There is no
    >requirement in the Apache license to mention Apache in your
    >advertising.


    Sorry, I created a bit of confusion here. I used advertise in the sense of make public. Both Apache and the old BSD license require you to make public that you use a derivitive of their work. Granted the old BSD license went much further.

    >They don't have to appear in the application interface,
    >splash screen or anywhere else but in the copyright notices.


    This is what 4.4 of the Apache license says: "within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear." How would you interpret this, knowing most of my customers will not release the source code?

    >The BSD license is similar in this respect - "Redistributions of source code
    >must retain the above copyright notice, this list of conditions and
    >the following disclaimer." Tomayto, tomahto.

    It is not that similar, because the modified BSD only refers to source code. The Apache license extends this towards generated code as well.

    >There is absolutely no requirement in the Apache license to submit
    >work back to the copyright holder, much less Apache. Wherever did you
    >get that notion?

    I don't have the notion that this is required. I just described what I do when I alter open source code of any license. I try to give it back, so it might be included in future releases for everyone to enjoy.

    >Why are you even open-sourcing your software given your concerns?

    I don't have any concerns. I answered your question on why I use one license instead of another. I use open source while it helps to build more efficient applications, having more time to focus on what is specific for that project. It helps to improve the more general code those projects rely on by having it used more widely, scrutenized, adapted and improved by everyone who uses it and will share.
    From www.brains2b.org: "By supplying this software as open source I hope it will prevent some 'reinventing of the wheel' that still seems very customary in software development. You will find different components you can use in your own projects that will hopefully save time and effort and can be adapted for your own needs. Next to this there are some applications I have written and are just handy, or fun and would otherwise be collecting dust."

    Regards,
    Dennis..
     
    dennis, Mar 23, 2010
    #17
  18. Dennis

    Lew Guest

    dennis wrote:
    > I found your response a bit strange. This were all answers to questions you yourself asked ?
    >


    I cannot parse that question, if question it is. What are you asking?

    My response was "strange" only insofar as it corrected misinformation
    from your post.

    Lew wrote:
    >> They don't have to appear in the application interface,
    >> splash screen or anywhere else but in the copyright notices.  

    >


    dennis wrote:
    > This is what 4.4 of the Apache license says: "within the Source form or documentation,
    > if provided along with the Derivative Works; or, within a display generated by the
    > Derivative Works, if and wherever such third-party notices normally appear." How would
    > you interpret this, knowing most of my customers will not release the source code?
    >


    I "interpret this" as saying exactly what it says, "... if and
    wherever such third-party notices normally appear." If your splash
    screen doesn't display copyright information, Apache license doesn't
    require to put all the upstream copyright notices.

    This is not different from what I said, "They don't have to appear ...
    anywhere else but in the copyright notices." I don't know why anyone
    objects to displaying copyright notices from the copyright holder(s);
    that's what copyright is for. There's a word for using copyrighted
    material without acknowledging the copyright: plagiarism.

    Lew wrote:
    >> The BSD license is similar in this respect - "Redistributions of source code
    >> must retain the above copyright notice, this list of conditions and
    >> the following disclaimer."  Tomayto, tomahto.

    >


    dennis wrote:
    > It is not that similar, because the modified BSD only refers to source code.
    > The Apache license extends this towards generated code as well.
    >


    No, it extends it to generated copyright notices. Nowhere does it
    discuss placing copyright notices in generated code. You are
    spreading misinformation about the Apache license.

    Lew wrote:
    >> There is absolutely no requirement in the Apache license to submit
    >> work back to the copyright holder, much less Apache.  Wherever did you
    >> get that notion?

    >


    dennis wrote:
    > I don't have the notion that this is required.
    >


    That contradicts what you said. You said,
    > They [your customers] cannot derive works of it, without complying to the notice
    > part of the license, or supplying it back to Apache.
    > Something they might not be willing to do.
    >


    Naturally I wondered why you thought the Apache license required that
    "they ... supply... it back to Apache."

    Lew wrote:
    >> Why are you even open-sourcing your software given your concerns?

    >


    dennis wrote:
    > I don't have any concerns. I answered your question on why I use one license instead of another.
    >


    So despite what you said, you are not concerned that your customers
    have to display the copyright notices of copyright holders?

    It's hard to discuss a subject with someone who shifts their ground on
    every point when a counterpoint or question is raised.

    I see no purpose in discussing this with you further.

    --
    Lew
     
    Lew, Mar 23, 2010
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. R
    Replies:
    5
    Views:
    2,139
    Kevin McMurtrie
    Mar 13, 2005
  2. Guest
    Replies:
    4
    Views:
    502
    Guest
    Oct 13, 2004
  3. Pekka Järvinen
    Replies:
    2
    Views:
    692
    Richard Tobin
    Apr 29, 2008
  4. Replies:
    4
    Views:
    236
    Tad McClellan
    Jun 1, 2007
  5. Replies:
    9
    Views:
    176
Loading...

Share This Page