How to really get the output stream of an unix process?

Discussion in 'Java' started by Carfield Yim, Jul 11, 2003.

  1. Carfield Yim

    Carfield Yim Guest

    I have some code like follow, which get some data from URL pipe to a
    unix command, and get the result:

    String line = "";
    final Runtime rt = Runtime.getRuntime();
    final Process p = rt.exec("somecommand");

    final BufferedReader urlBR = new BufferedReader(new
    InputStreamReader(new URL(url).openStream()));
    final Writer os = new BufferedWriter(new
    OutputStreamWriter(p.getOutputStream()));
    while((line = urlBR.readLine()) != null) os.write(line);
    os.close();
    urlBR.close();

    final BufferedReader din = new BufferedReader(new
    InputStreamReader(p.getInputStream()));
    while ( (line = din.readLine()) != null ) out.write(line);
    din.close();

    However it fail with "broken pipe" exception, I believe that as I don't
    really pipe the data to the command, so java execute the command as
    nothing output to the command.

    Thus I try to use
    final Process p = rt.exec("|somecommand");

    But java complain command not find. How can I tell java that I have
    something needed to pipe to the command?
    --
    \\\|///
    \\- - -//
    ( @ @ )
    -----------oOOo-(_)-oOOo------------------------------------------------
    Visit my homepage at http://www.carfield.com.hk

    Programming discussion groups
    Software design: news://news.carfield.com.hk/programming.design
    Design Pattern: news://news.carfield.com.hk/programming.design.pattern
    java: news://news.carfield.com.hk/programming.java
    linux: news://news.carfield.com.hk/programming.linux
    ------------------------------------------------------------------------
    Carfield Yim, Jul 11, 2003
    #1
    1. Advertising

  2. On Sat, 12 Jul 2003 02:47:30 +0800, Carfield Yim wrote:
    > I have some code like follow, which get some data from URL pipe to a
    > unix command, and get the result:
    >
    > String line = "";
    > final Runtime rt = Runtime.getRuntime();
    > final Process p = rt.exec("somecommand");
    >
    > final BufferedReader urlBR = new BufferedReader(new InputStreamReader(new URL(url).openStream()));
    > final Writer os = new BufferedWriter(new OutputStreamWriter(p.getOutputStream()));
    > while((line = urlBR.readLine()) != null) os.write(line);
    > os.close();
    > urlBR.close();
    >
    > final BufferedReader din = new BufferedReader(new InputStreamReader(p.getInputStream()));
    > while ( (line = din.readLine()) != null ) out.write(line);
    > din.close();
    >


    > However it fail with "broken pipe" exception, I believe that as I
    > don't really pipe the data to the command, so java execute the
    > command as nothing output to the command.


    What do you mean you "don't really pipe the data"? You've got calls to
    os.write() in the example.

    You get broken pipe if you attempt to write to a pipe after the reader
    has closed his end of the connection. Did you manage to write any of
    the data, or do you get this exception as soon as you try to write
    anything?

    Do you have the same problem with other commands, or just one
    particular "somecommand"?

    > Thus I try to use
    > final Process p = rt.exec("|somecommand");


    Without another command on the left of the pipe symbol, it would have
    been nonsense in a shell that understands redirection. As it is, Java
    doesn't even execute the external program in a shell.

    > But java complain command not find. How can I tell java that I have
    > something needed to pipe to the command?


    Java already assumes that you do and provides p.getOutputStream() for
    that purpose. Don't close it if you still need it (you can't get it
    back), and make sure that "somecommand" isn't closing its stdin.

    /gordon

    --
    [ do not send me private copies of your followups ]
    g o r d o n . b e a t o n @ e r i c s s o n . c o m
    Gordon Beaton, Jul 11, 2003
    #2
    1. Advertising

  3. On Sat, 12 Jul 2003 02:47:30 +0800, Carfield Yim wrote:

    > I have some code like follow, which get some data from URL pipe to a
    > unix command, and get the result:
    >
    > String line = "";
    > final Runtime rt = Runtime.getRuntime();
    > final Process p = rt.exec("somecommand");
    >
    > final BufferedReader urlBR = new BufferedReader(new
    > InputStreamReader(new URL(url).openStream()));
    > final Writer os = new BufferedWriter(new
    > OutputStreamWriter(p.getOutputStream()));
    > while((line = urlBR.readLine()) != null) os.write(line);
    > os.close();
    > urlBR.close();
    >
    > final BufferedReader din = new BufferedReader(new
    > InputStreamReader(p.getInputStream()));
    > while ( (line = din.readLine()) != null ) out.write(line);
    > din.close();
    >
    > However it fail with "broken pipe" exception, I believe that as I don't
    > really pipe the data to the command, so java execute the command as
    > nothing output to the command.
    >
    > Thus I try to use
    > final Process p = rt.exec("|somecommand");
    >
    > But java complain command not find. How can I tell java that I have
    > something needed to pipe to the command?



    A few points I can think of:

    BufferedReader.readLine() strips the \r and \n characters from the line.
    If you feed the lines direct to the Process, you are giving it a single
    long line to work on (and no line ending before you close the pipe). This
    may well be why the process produces no discernible output. I suggest that
    either re-insert line endings.

    Does the URL return binary data? If so, better to deal purely in bytes
    than go through a bytes->text->bytes double conversion, with its attendant
    characterset conversion foibles. I see no reason at all in the above code
    to convert to an intermediate text format.

    The code tries to feed the URL contents to the process _completely_ before
    attempting any read of the process's output stream. This is unsafe. The
    process might block because it's stdout stream buffers are full, in which
    cease it will cease reading its input stream, and you are deadlocked. I
    suggest that you use multithreading so that you can keep both its input
    and output serviced concurrently.

    Also, you make no attempt to read the process's error stream. The process
    may deadlock with a blocked stderr stream if it has problems, and you will
    never know why. I suggest another Thread reading the error stream, even
    though you hope it will never have any work to do.

    Always close these streams after use, or you may get "too many open file"
    errors after a while.

    Steve
    Steve Horsley, Jul 11, 2003
    #3
  4. Carfield Yim

    Carfield Yim Guest

    > A few points I can think of:
    >
    > BufferedReader.readLine() strips the \r and \n characters from the line.
    > If you feed the lines direct to the Process, you are giving it a single
    > long line to work on (and no line ending before you close the pipe). This
    > may well be why the process produces no discernible output. I suggest that
    > either re-insert line endings.
    >
    > Does the URL return binary data? If so, better to deal purely in bytes
    > than go through a bytes->text->bytes double conversion, with its attendant
    > characterset conversion foibles. I see no reason at all in the above code
    > to convert to an intermediate text format.
    >
    > The code tries to feed the URL contents to the process _completely_ before
    > attempting any read of the process's output stream. This is unsafe. The
    > process might block because it's stdout stream buffers are full, in which
    > cease it will cease reading its input stream, and you are deadlocked. I
    > suggest that you use multithreading so that you can keep both its input
    > and output serviced concurrently.
    >
    > Also, you make no attempt to read the process's error stream. The process
    > may deadlock with a blocked stderr stream if it has problems, and you will
    > never know why. I suggest another Thread reading the error stream, even
    > though you hope it will never have any work to do.
    >
    > Always close these streams after use, or you may get "too many open file"
    > errors after a while.
    >
    > Steve


    Thx, it solved when output the stderr and stdout in thread
    Carfield Yim, Jul 12, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    6
    Views:
    4,718
  2. Replies:
    9
    Views:
    615
    Alex Buell
    Apr 27, 2006
  3. Alexander Korsunsky

    get stream mode flags from an opened stream

    Alexander Korsunsky, Feb 17, 2007, in forum: C++
    Replies:
    1
    Views:
    444
    John Harrison
    Feb 17, 2007
  4. Kashif Ur Rehman
    Replies:
    2
    Views:
    855
    Tom Hawtin
    May 17, 2007
  5. Replies:
    14
    Views:
    240
    Martijn Lievaart
    Aug 15, 2009
Loading...

Share This Page