exec() and sending to STDIO and reading from STDIO

Discussion in 'Java' started by WinstonSmith_101@hotmail.com, Oct 17, 2006.

  1. Guest

    I have an external program I need to execute which takes an argument.
    So I have the code:

    String command = "...";
    String argument = "...";
    process = Runtime.getRuntime().exec(command);

    And then for sending arguments:

    java.io_OutputStream out = process.getOutputStream();
    out.write(argument.getBytes());
    out.close();


    And the for reading output from the external command:

    java.io.InputStream in = encryptProcess.getInputStream();
    int c;
    while ((c = in.read()) != -1)
    {
    System.out.print("OUT: " + (char)c);
    }
    in.close();

    = = = = = = = = = = = = = = = = = =

    But the problem is that the output is not actually read. It's still
    being outputted when I close the InputStream (in.close();)

    Is there not a way to but execute an external program, send arguments
    to it and get it's output?


    I have tried to insert the command "echo argument|command" e.g.
    "echo test|ls" in the command, but this merely echos out the whole
    line ("test|ls") rather than sending the argument "test" to
    "ls".


    Thanks
    /Rune
     
    , Oct 17, 2006
    #1
    1. Advertising

  2. On 17 Oct 2006 09:44:37 -0700, wrote:
    > I have tried to insert the command "echo argument|command" e.g.
    > "echo test|ls" in the command, but this merely echos out the whole
    > line ("test|ls") rather than sending the argument "test" to
    > "ls".


    It's hardly meaningful to send input to ls that way, but it fails
    because you are attempting to use shell features where there is no
    shell. You need to do it like this:

    String[] cmd = { "/bin/sh", "-c", "echo foo | gurka" };
    Process p = System.getRuntime().exec(cmd);

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Oct 17, 2006
    #2
    1. Advertising

  3. Guest

    Gordon Beaton wrote:
    > On 17 Oct 2006 09:44:37 -0700, wrote:
    > > I have tried to insert the command "echo argument|command" e.g.
    > > "echo test|ls" in the command, but this merely echos out the whole
    > > line ("test|ls") rather than sending the argument "test" to
    > > "ls".

    >
    > It's hardly meaningful to send input to ls that way, but it fails
    > because you are attempting to use shell features where there is no
    > shell. You need to do it like this:
    >
    > String[] cmd = { "/bin/sh", "-c", "echo foo | gurka" };
    > Process p = System.getRuntime().exec(cmd);


    Hello thanks for the feedback, but this merely echos out "foo | gurka".
     
    , Oct 18, 2006
    #3
  4. On 18 Oct 2006 00:46:36 -0700, wrote:
    > Hello thanks for the feedback, but this merely echos out "foo | gurka".


    No, it doesn't. Post your *actual* code. You are not running a shell
    as I suggested.

    This is what happens when I run it with real commands:

    bash$ cat Test.java
    import java.io.*;

    public class Test {
    public static void main(String[] args) throws Exception {
    String[] cmd = { "/bin/sh", "-c", "echo this is a test | wc" };
    Process p = Runtime.getRuntime().exec(cmd);

    BufferedReader br = new BufferedReader(new InputStreamReader (p.getInputStream()));

    String line;
    while ((line = br.readLine()) != null) {
    System.out.println(line);
    }
    br.close();
    }
    }

    base$ javac Test.java
    bash$ java Test
    1 4 15
    bash$

    The example is complete and compilable. Try it.

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Oct 18, 2006
    #4
  5. Guest

    Gordon Beaton wrote:
    > On 18 Oct 2006 00:46:36 -0700, wrote:
    > > Hello thanks for the feedback, but this merely echos out "foo | gurka".

    >
    > No, it doesn't. Post your *actual* code. You are not running a shell
    > as I suggested.
    >
    > This is what happens when I run it with real commands:
    >
    > bash$ cat Test.java
    > import java.io.*;
    >
    > public class Test {
    > public static void main(String[] args) throws Exception {
    > String[] cmd = { "/bin/sh", "-c", "echo this is a test | wc" };
    > Process p = Runtime.getRuntime().exec(cmd);
    >
    > BufferedReader br = new BufferedReader(new InputStreamReader (p.getInputStream()));
    >
    > String line;
    > while ((line = br.readLine()) != null) {
    > System.out.println(line);
    > }
    > br.close();
    > }
    > }
    >
    > base$ javac Test.java
    > bash$ java Test
    > 1 4 15
    > bash$
    >
    > The example is complete and compilable. Try it.


    - your example works. But when I substitude the program gpg for wc, I
    stop getting the output. Can it be that it sends on STDERR or
    something?

    import java.io.*;



    public class Test1
    {
    public static void main(String[] args) throws Exception
    {
    String[] cmd = { "/bin/sh", "-c", "echo mypasswd | gpg
    --passphrase-fd 0 -o tobe.zip -q --decrypt tobe.zip.gpg" };



    Process p = Runtime.getRuntime().exec(cmd);



    BufferedReader br = new BufferedReader(new InputStreamReader
    (p.getInputStream()));



    String line;
    while ((line = br.readLine()) != null)
    {
    System.out.println("XXX: " + line);
    }



    br.close();
    }
    }

    - it's the same. I just changed the "String[] cmd =" line
     
    , Oct 18, 2006
    #5
  6. On 18 Oct 2006 04:30:31 -0700, wrote:
    > But when I substitude the program gpg for wc, I stop getting the
    > output. Can it be that it sends on STDERR or something?


    It's quite possible that gpg is sending output to stderr, but you
    should be reading from both streams anyway (which my simple example
    doesn't do). To do this correctly you need to read from the error
    stream from a separate thread.

    An alternative is to to use ProcessBuilder, which can combine both
    streams into one and you don't need any extra thread. But since you're
    using a shell anyway, you can make a simple change to the command and
    it will combine the output streams for you:

    String[] cmd = {
    "/bin/sh",
    "-c",
    "echo mypasswd | gpg --passphrase-fd 0 -o tobe.zip -q --decrypt tobe.zip.gpg 2>&1"
    };

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Oct 18, 2006
    #6
  7. aeonsun Guest

    how can i reading from stdio use this command:
    type mytest.txt (win32)

    use String[] cmd = {"cmd","/c","type mytest.txt"} ,it dosn't work.
     
    aeonsun, Oct 20, 2006
    #7
  8. On 20 Oct 2006 00:45:15 -0700, aeonsun wrote:
    > how can i reading from stdio use this command:
    > type mytest.txt (win32)
    >
    > use String[] cmd = {"cmd","/c","type mytest.txt"} ,it dosn't work.


    You don't need cmd.exe for this, other than that I have no idea.

    (in fact you don't need Runtime.exec() for this at all).

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Oct 20, 2006
    #8
  9. Gordon Beaton wrote:
    > On 20 Oct 2006 00:45:15 -0700, aeonsun wrote:
    >> how can i reading from stdio use this command:
    >> type mytest.txt (win32)
    >>
    >> use String[] cmd = {"cmd","/c","type mytest.txt"} ,it dosn't work.

    >
    > You don't need cmd.exe for this, other than that I have no idea.


    Since type is an internal command in cmd and not a type.exe, then
    I would think cmd is needed.

    Arne
     
    =?ISO-8859-1?Q?Arne_Vajh=F8j?=, Oct 21, 2006
    #9
  10. aeonsun wrote:
    > how can i reading from stdio use this command:
    > type mytest.txt (win32)
    >
    > use String[] cmd = {"cmd","/c","type mytest.txt"} ,it dosn't work.


    Try:

    String[] cmd = {"cmd","/c", "type", "mytest.txt"};

    Arne
     
    =?ISO-8859-1?Q?Arne_Vajh=F8j?=, Oct 21, 2006
    #10
  11. crazzybugger Guest

    wrote:
    > I have an external program I need to execute which takes an argument.
    > So I have the code:
    >
    > String command = "...";
    > String argument = "...";
    > process = Runtime.getRuntime().exec(command);
    >
    > And then for sending arguments:
    >
    > java.io_OutputStream out = process.getOutputStream();
    > out.write(argument.getBytes());
    > out.close();
    >
    >
    > And the for reading output from the external command:
    >
    > java.io.InputStream in = encryptProcess.getInputStream();
    > int c;
    > while ((c = in.read()) != -1)
    > {
    > System.out.print("OUT: " + (char)c);
    > }
    > in.close();
    >
    > = = = = = = = = = = = = = = = = = =
    >
    > But the problem is that the output is not actually read. It's still
    > being outputted when I close the InputStream (in.close();)
    >
    > Is there not a way to but execute an external program, send arguments
    > to it and get it's output?
    >
    >
    > I have tried to insert the command "echo argument|command" e.g.
    > "echo test|ls" in the command, but this merely echos out the whole
    > line ("test|ls") rather than sending the argument "test" to
    > "ls".
    >
    >
    > Thanks
    > /Rune


    I think you are doing a mistake here........If i am right you are
    supposed to wait for the process to end .
    you can try out this sniplet.
    try{
    Process p=Runtime.getRuntime().exec(yourCommand);
    int result=p.waitFor();
    if(result==0){

    /*this means that the program ran successfully.
    *now use your buffered stream and read from the
    process using the
    *process.getInputStream() call*/

    }else{

    /*program did not run successfully!
    *use your stream and read from the process using the

    * process.getErrorStream()*/

    }
    }catch(Exception e){
    /*you reach here incase the process did not execute
    properly or wrong arguements */
    }
     
    crazzybugger, Oct 21, 2006
    #11
  12. crazzybugger Guest

    incase you get error paste your result here...........
     
    crazzybugger, Oct 21, 2006
    #12
  13. On 20 Oct 2006 16:41:09 -0700, crazzybugger wrote:
    > I think you are doing a mistake here........If i am right you are
    > supposed to wait for the process to end .


    > Process p=Runtime.getRuntime().exec(yourCommand);
    > int result=p.waitFor();
    > if(result==0){
    > /*this means that the program ran successfully.
    > *now use your buffered stream and read from the process using the
    > *process.getInputStream() call*/
    > }else{
    > /*program did not run successfully!
    > *use your stream and read from the process using the
    > * process.getErrorStream()*/
    > }


    This is a poor suggestion. You must read from both output and error
    streams while the process is still running, or it will very likely
    deadlock. Your method can only work for programs with very little
    output (a few lines at most), and it certainly doesn't work for
    programs that should run continuously.

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Oct 21, 2006
    #13
  14. On Fri, 20 Oct 2006 19:04:27 -0400, Arne Vajhøj wrote:
    > Since type is an internal command in cmd and not a type.exe, then
    > I would think cmd is needed.


    I was assuming that type was a "real" program, but if that's not the
    case then of course you're right.

    /gordon (not a windows user)

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Oct 21, 2006
    #14
  15. crazzybugger Guest

    Gordon Beaton wrote:
    > On 20 Oct 2006 16:41:09 -0700, crazzybugger wrote:
    > > I think you are doing a mistake here........If i am right you are
    > > supposed to wait for the process to end .

    >
    > > Process p=Runtime.getRuntime().exec(yourCommand);
    > > int result=p.waitFor();
    > > if(result==0){
    > > /*this means that the program ran successfully.
    > > *now use your buffered stream and read from the process using the
    > > *process.getInputStream() call*/
    > > }else{
    > > /*program did not run successfully!
    > > *use your stream and read from the process using the
    > > * process.getErrorStream()*/
    > > }

    >
    > This is a poor suggestion. You must read from both output and error
    > streams while the process is still running, or it will very likely
    > deadlock. Your method can only work for programs with very little
    > output (a few lines at most), and it certainly doesn't work for
    > programs that should run continuously.
    >
    > /gordon
    >
    > --
    > [ don't email me support questions or followups ]
    > g o r d o n + n e w s @ b a l d e r 1 3 . s e


    Yes my suggestion was for a code which will end quick enough . hence
    there is no point of the suggestion being poor since it is well
    understood from the sniplet that the code is expecting the process to
    finish running quickly/or after a time interval.
    Incase you want the process to finish its execution within a
    time limit , you could always spawn a thread which will count the time
    since the process's execution start and it can be used to kill it in
    case it exceeds the time limit .(used by programming contests i
    suppose) .
    So dont use it if your code is going to run continuously!
     
    crazzybugger, Oct 21, 2006
    #15
  16. On 21 Oct 2006 04:51:08 -0700, crazzybugger wrote:
    > Yes my suggestion was for a code which will end quick enough . hence
    > there is no point of the suggestion being poor since it is well
    > understood from the sniplet that the code is expecting the process to
    > finish running quickly/or after a time interval.


    My comment had really nothing to do with the length of time a process
    runs. It has everything to do with the amount of output it produces.

    Any program that produces more than a few lines of output will *hang*
    if you run it using your example. Yes it was necessary to point this
    out since many posters to this group make that mistake anyway, even
    without you telling them that this is a good way to use Runtime.exec()
    (it isn't).

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Oct 21, 2006
    #16
  17. crazzybugger Guest

    Gordon Beaton wrote:
    > On 21 Oct 2006 04:51:08 -0700, crazzybugger wrote:
    > > Yes my suggestion was for a code which will end quick enough . hence
    > > there is no point of the suggestion being poor since it is well
    > > understood from the sniplet that the code is expecting the process to
    > > finish running quickly/or after a time interval.

    >
    > My comment had really nothing to do with the length of time a process
    > runs. It has everything to do with the amount of output it produces.
    >
    > Any program that produces more than a few lines of output will *hang*
    > if you run it using your example. Yes it was necessary to point this
    > out since many posters to this group make that mistake anyway, even
    > without you telling them that this is a good way to use Runtime.exec()
    > (it isn't).
    >
    > /gordon
    >
    > --
    > [ don't email me support questions or followups ]
    > g o r d o n + n e w s @ b a l d e r 1 3 . s e

    you must have more experience to say so.........anyway i
    have never tried out with programs giving larger output.........
    probably it would.......but i wonder why ?
     
    crazzybugger, Oct 21, 2006
    #17
  18. Chris Uppal Guest

    crazzybugger wrote:

    > Gordon Beaton wrote:
    > > Any program that produces more than a few lines of output will *hang*
    > > if you run it using your example. Yes it was necessary to point this
    > > out since many posters to this group make that mistake anyway, even
    > > without you telling them that this is a good way to use Runtime.exec()
    > > (it isn't).

    [...]
    > you must have more experience to say so.........anyway i
    > have never tried out with programs giving larger output.........
    > probably it would.......but i wonder why ?


    No need for experience -- just a little knowledge will do ;-)

    If the program attempts to write more to its stdout, or stderr, than the OS is
    willing to buffer-up on its behalf (i.e. data that is sitting in the pipes
    between that process and the parent Java process); then the OS will block the
    writer until the reader has consumed some of the data. Since the reader (Java)
    is waiting for the writer to exit before it will read /anything/ from the pipe,
    and since the writer is blocked waiting for the reader to read something before
    it can exit, they both end up waiting for each other.

    Deadlock.

    Note that you have no control over how much data the OS will allow the writer
    to put in the pipe before blocking it. It is certainly OS-dependent, and might
    be installation-dependent.

    Don't ever use code like that in a production environment, and be very cautious
    about using it even in throwaway code.

    -- chris
     
    Chris Uppal, Oct 22, 2006
    #18
  19. crazzybugger Guest

    Chris Uppal wrote:
    > crazzybugger wrote:
    >
    > > Gordon Beaton wrote:
    > > > Any program that produces more than a few lines of output will *hang*
    > > > if you run it using your example. Yes it was necessary to point this
    > > > out since many posters to this group make that mistake anyway, even
    > > > without you telling them that this is a good way to use Runtime.exec()
    > > > (it isn't).

    > [...]
    > > you must have more experience to say so.........anyway i
    > > have never tried out with programs giving larger output.........
    > > probably it would.......but i wonder why ?

    >
    > No need for experience -- just a little knowledge will do ;-)
    >
    > If the program attempts to write more to its stdout, or stderr, than the OS is
    > willing to buffer-up on its behalf (i.e. data that is sitting in the pipes
    > between that process and the parent Java process); then the OS will block the
    > writer until the reader has consumed some of the data. Since the reader (Java)
    > is waiting for the writer to exit before it will read /anything/ from the pipe,
    > and since the writer is blocked waiting for the reader to read something before
    > it can exit, they both end up waiting for each other.
    >
    > Deadlock.
    >
    > Note that you have no control over how much data the OS will allow the writer
    > to put in the pipe before blocking it. It is certainly OS-dependent, and might
    > be installation-dependent.
    >
    > Don't ever use code like that in a production environment, and be very cautious
    > about using it even in throwaway code.
    >
    > -- chris


    Excellent explanation!!!
     
    crazzybugger, Oct 22, 2006
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hal Vaughan
    Replies:
    11
    Views:
    1,138
    Gordon Beaton
    May 22, 2006
  2. tedsuzman
    Replies:
    2
    Views:
    7,097
    Michel Claveau, résurectionné d'outre-bombe inform
    Jul 21, 2004
  3. Ted
    Replies:
    1
    Views:
    472
    Duncan Booth
    Jul 22, 2004
  4. Guillermo Riojas
    Replies:
    0
    Views:
    175
    Guillermo Riojas
    Nov 26, 2010
  5. Random Task
    Replies:
    12
    Views:
    660
    Joe Smith
    Dec 4, 2005
Loading...

Share This Page