Issue implementing Runtime.exec() with StreamGobbler

Discussion in 'Java' started by Vic, Mar 6, 2009.

  1. Vic

    Vic Guest

    I am not very well conversant with java and have a very basic and
    limited java experience being a QA engg. I am currently looking at a
    code which isn't very consistent in our testing atuomation framework.
    Here is the code ->
    // executes system commands and captures STDOUT and STDERR
    import java.io.*;
    import java.util.*;

    class StreamGobbler extends Thread {
    InputStream is;
    String type;
    String result;

    StreamGobbler(InputStream is, String type){
    this.is = is;
    this.type = type;
    }

    public void run(){
    StringBuffer buffer = new StringBuffer();

    try{
    InputStreamReader isr = new InputStreamReader(is);
    BufferedReader br = new BufferedReader(isr);
    String line = null;
    while((line = br.readLine()) != null){
    buffer.append(line);
    buffer.append("\n");
    }
    }
    catch(IOException ioe){
    ioe.printStackTrace();
    }
    finally{
    result = buffer.toString();
    }
    }

    public String getResult(){
    return result;
    }
    }

    class NewCommandExecuter {
    public String stdout = null;
    public String stderr = null;
    private String line;

    public NewCommandExecuter (String cmd) {
    try{
    //VikP - This needed to be modified to incorporate the change in
    syntax for executing getData.pl by Mark Silverman for PR#5026342
    System.out.println("New Command Executer: EXECUTING COMMAND:
    "+cmd);
    String [] execmd = {"bash","-c",cmd};
    //Process proc = Runtime.getRuntime().exec(new String[]{"bash","-
    c",cmd});
    Process proc = Runtime.getRuntime().exec(execmd);
    //Process proc = Runtime.getRuntime().exec(cmd);
    StreamGobbler errorGobbler = new StreamGobbler(proc.getErrorStream
    (),"ERROR");
    StreamGobbler outputGobbler = new StreamGobbler
    (proc.getInputStream(),"OUTPUT");

    errorGobbler.start();
    outputGobbler.start();

    int val = proc.waitFor();
    System.out.println("ExitValue for Command "+cmd+" is: "+val);

    stdout = outputGobbler.getResult();
    stderr = errorGobbler.getResult();

    //VikP - Waits for these threads to die
    errorGobbler.join();
    outputGobbler.join();
    //
    }
    catch (Throwable t){
    t.printStackTrace();
    }

    if (stdout.equals(""))
    stdout = null;

    if (stderr.equals(""))
    stderr = null;
    }
    }

    I call the the above in a loop (of about 2000 cases) as ->
    NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);

    // verity xml output
    if (xmlOut.stdout == null)
    throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    any results. STDERR returned:\n"+xmlOut.stderr);

    // save output to a temp file
    writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    to a file

    The problem that I am facing is it works fine but it doesn't go
    through the entire loop (all the test cases we have) and almost always
    gets terminated prematurely completing only a partial list of cases.
    So that makes me wonder if there is anything wrong in the
    StreamGobbler and/or NewCommandExecuter. I tried a bunch of things in
    those classes but definitely I am missing something here. Please help.
    Thanks
     
    Vic, Mar 6, 2009
    #1
    1. Advertising

  2. Vic

    Vic Guest

    On Mar 6, 9:43 am, Vic <> wrote:
    > I am not very well conversant with java and have a very basic and
    > limited java experience being a QA engg. I am currently looking at a
    > code which isn't very consistent in our testing atuomation framework.
    > Here is the code ->
    > // executes system commands and captures STDOUT and STDERR
    > import java.io.*;
    > import java.util.*;
    >
    > class StreamGobbler extends Thread {
    >     InputStream is;
    >     String type;
    >     String result;
    >
    >     StreamGobbler(InputStream is, String type){
    >         this.is = is;
    >         this.type = type;
    >     }
    >
    >     public void run(){
    >         StringBuffer buffer = new StringBuffer();
    >
    >         try{
    >             InputStreamReader isr = new InputStreamReader(is);
    >             BufferedReader br = new BufferedReader(isr);
    >             String line = null;
    >             while((line = br.readLine()) != null){
    >                 buffer.append(line);
    >                 buffer.append("\n");
    >             }
    >         }
    >         catch(IOException ioe){
    >             ioe.printStackTrace();
    >         }
    >         finally{
    >             result = buffer.toString();
    >         }
    >     }
    >
    >     public String getResult(){
    >         return result;
    >     }
    >
    > }
    >
    > class NewCommandExecuter {
    >     public String stdout = null;
    >     public String stderr = null;
    >     private String line;
    >
    >     public NewCommandExecuter (String cmd) {
    >         try{
    >             //VikP - This needed to be modified to incorporate the change in
    > syntax for executing getData.pl by Mark Silverman for PR#5026342
    >             System.out.println("New Command Executer: EXECUTING COMMAND:
    > "+cmd);
    >             String [] execmd = {"bash","-c",cmd};
    >             //Process proc = Runtime.getRuntime().exec(new String[]{"bash","-
    > c",cmd});
    >             Process proc = Runtime.getRuntime().exec(execmd);
    >             //Process proc = Runtime.getRuntime().exec(cmd);
    >             StreamGobbler errorGobbler = new StreamGobbler(proc.getErrorStream
    > (),"ERROR");
    >             StreamGobbler outputGobbler = new StreamGobbler
    > (proc.getInputStream(),"OUTPUT");
    >
    >             errorGobbler.start();
    >             outputGobbler.start();
    >
    >             int val = proc.waitFor();
    >             System.out.println("ExitValue for Command "+cmd+" is: "+val);
    >
    >             stdout = outputGobbler.getResult();
    >             stderr = errorGobbler.getResult();
    >
    >             //VikP - Waits for these threads to die
    >             errorGobbler.join();
    >             outputGobbler.join();
    >             //
    >         }
    >         catch (Throwable t){
    >                 t.printStackTrace();
    >         }
    >
    >         if (stdout.equals(""))
    >             stdout = null;
    >
    >         if (stderr.equals(""))
    >             stderr = null;
    >     }
    >
    > }
    >
    > I call the the above in a loop (of about 2000 cases) as ->
    > NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);
    >
    > // verity xml output
    >             if (xmlOut.stdout == null)
    >                 throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    > any results. STDERR returned:\n"+xmlOut.stderr);
    >
    >             // save output to a temp file
    >             writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    > to a file
    >
    > The problem that I am facing is it works fine but  it doesn't go
    > through the entire loop (all the test cases we have) and almost always
    > gets terminated prematurely completing only a partial list of cases.
    > So that makes me wonder if there is anything wrong in the
    > StreamGobbler and/or NewCommandExecuter. I tried a bunch of things in
    > those classes but definitely I am missing something here. Please help.
    > Thanks


    xmlOutCmd looks something like this -> /home/qauser/770currentLOADER/
    bin/loader -c -u genevaman -w qa -f Transaction917.0.lrd -o
    Transaction917.0.lrd.xml

    Also the implementation of Runtime.exec() was developed with the help
    from
    http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=4

    Thanks in advance
     
    Vic, Mar 6, 2009
    #2
    1. Advertising

  3. Vic wrote:
    > I am not very well conversant with java and have a very basic and
    > limited java experience being a QA engg. I am currently looking at a
    > code which isn't very consistent in our testing atuomation framework.
    > Here is the code ->
    > // executes system commands and captures STDOUT and STDERR
    > import java.io.*;
    > import java.util.*;
    >
    > class StreamGobbler extends Thread {
    > InputStream is;
    > String type;
    > String result;
    >
    > StreamGobbler(InputStream is, String type){
    > this.is = is;
    > this.type = type;
    > }
    >
    > public void run(){
    > StringBuffer buffer = new StringBuffer();
    >
    > try{
    > InputStreamReader isr = new InputStreamReader(is);
    > BufferedReader br = new BufferedReader(isr);
    > String line = null;
    > while((line = br.readLine()) != null){
    > buffer.append(line);
    > buffer.append("\n");
    > }
    > }
    > catch(IOException ioe){
    > ioe.printStackTrace();
    > }
    > finally{
    > result = buffer.toString();
    > }
    > }
    >
    > public String getResult(){
    > return result;
    > }
    > }
    >
    > class NewCommandExecuter {
    > public String stdout = null;
    > public String stderr = null;
    > private String line;
    >
    > public NewCommandExecuter (String cmd) {
    > try{
    > //VikP - This needed to be modified to incorporate the change in
    > syntax for executing getData.pl by Mark Silverman for PR#5026342
    > System.out.println("New Command Executer: EXECUTING COMMAND:
    > "+cmd);
    > String [] execmd = {"bash","-c",cmd};
    > //Process proc = Runtime.getRuntime().exec(new String[]{"bash","-
    > c",cmd});
    > Process proc = Runtime.getRuntime().exec(execmd);
    > //Process proc = Runtime.getRuntime().exec(cmd);
    > StreamGobbler errorGobbler = new StreamGobbler(proc.getErrorStream
    > (),"ERROR");
    > StreamGobbler outputGobbler = new StreamGobbler
    > (proc.getInputStream(),"OUTPUT");
    >
    > errorGobbler.start();
    > outputGobbler.start();
    >
    > int val = proc.waitFor();
    > System.out.println("ExitValue for Command "+cmd+" is: "+val);
    >
    > stdout = outputGobbler.getResult();
    > stderr = errorGobbler.getResult();
    >
    > //VikP - Waits for these threads to die
    > errorGobbler.join();
    > outputGobbler.join();
    > //
    > }
    > catch (Throwable t){
    > t.printStackTrace();
    > }
    >
    > if (stdout.equals(""))
    > stdout = null;
    >
    > if (stderr.equals(""))
    > stderr = null;
    > }
    > }
    >
    > I call the the above in a loop (of about 2000 cases) as ->
    > NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);
    >
    > // verity xml output
    > if (xmlOut.stdout == null)
    > throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    > any results. STDERR returned:\n"+xmlOut.stderr);
    >
    > // save output to a temp file
    > writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    > to a file
    >
    > The problem that I am facing is it works fine but it doesn't go
    > through the entire loop (all the test cases we have) and almost always
    > gets terminated prematurely completing only a partial list of cases.
    > So that makes me wonder if there is anything wrong in the
    > StreamGobbler and/or NewCommandExecuter. I tried a bunch of things in
    > those classes but definitely I am missing something here. Please help.
    > Thanks


    By 'terminated', do you mean it throws an exception or just ends without
    running all the tests?

    --

    Knute Johnson
    email s/nospam/knute2009/

    --
    Posted via NewsDemon.com - Premium Uncensored Newsgroup Service
    ------->>>>>>http://www.NewsDemon.com<<<<<<------
    Unlimited Access, Anonymous Accounts, Uncensored Broadband Access
     
    Knute Johnson, Mar 6, 2009
    #3
  4. Vic

    Vic Guest

    On Mar 6, 9:56 am, Knute Johnson <>
    wrote:
    > Vic wrote:
    > > I am not very well conversant with java and have a very basic and
    > > limited java experience being a QA engg. I am currently looking at a
    > > code which isn't very consistent in our testing atuomation framework.
    > > Here is the code ->
    > > // executes system commands and captures STDOUT and STDERR
    > > import java.io.*;
    > > import java.util.*;

    >
    > > class StreamGobbler extends Thread {
    > >     InputStream is;
    > >     String type;
    > >     String result;

    >
    > >     StreamGobbler(InputStream is, String type){
    > >    this.is = is;
    > >    this.type = type;
    > >     }

    >
    > >     public void run(){
    > >    StringBuffer buffer = new StringBuffer();

    >
    > >    try{
    > >        InputStreamReader isr = new InputStreamReader(is);
    > >        BufferedReader br = new BufferedReader(isr);
    > >        String line = null;
    > >        while((line = br.readLine()) != null){
    > >            buffer.append(line);
    > >            buffer.append("\n");
    > >        }
    > >    }
    > >    catch(IOException ioe){
    > >        ioe.printStackTrace();
    > >    }
    > >    finally{
    > >        result = buffer.toString();
    > >    }
    > >     }

    >
    > >     public String getResult(){
    > >    return result;
    > >     }
    > > }

    >
    > > class NewCommandExecuter {
    > >     public String stdout = null;
    > >     public String stderr = null;
    > >     private String line;

    >
    > >     public NewCommandExecuter (String cmd) {
    > >    try{
    > >        //VikP - This needed to be modified to incorporate the change in
    > > syntax for executing getData.pl by Mark Silverman for PR#5026342
    > >        System.out.println("New Command Executer: EXECUTING COMMAND:
    > > "+cmd);
    > >        String [] execmd = {"bash","-c",cmd};
    > >        //Process proc = Runtime.getRuntime().exec(new String[]{"bash","-
    > > c",cmd});
    > >        Process proc = Runtime.getRuntime().exec(execmd);
    > >        //Process proc = Runtime.getRuntime().exec(cmd);
    > >        StreamGobbler errorGobbler = new StreamGobbler(proc.getErrorStream
    > > (),"ERROR");
    > >        StreamGobbler outputGobbler = new StreamGobbler
    > > (proc.getInputStream(),"OUTPUT");

    >
    > >        errorGobbler.start();
    > >        outputGobbler.start();

    >
    > >        int val = proc.waitFor();
    > >        System.out.println("ExitValue for Command "+cmd+" is: "+val);

    >
    > >        stdout = outputGobbler.getResult();
    > >        stderr = errorGobbler.getResult();

    >
    > >        //VikP - Waits for these threads to die
    > >        errorGobbler.join();
    > >        outputGobbler.join();
    > >        //
    > >    }
    > >    catch (Throwable t){
    > >            t.printStackTrace();
    > >    }

    >
    > >    if (stdout.equals(""))
    > >        stdout = null;

    >
    > >    if (stderr.equals(""))
    > >        stderr = null;
    > >     }
    > > }

    >
    > > I call the the above in a loop (of about 2000 cases) as ->
    > > NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);

    >
    > > // verity xml output
    > >        if (xmlOut.stdout == null)
    > >            throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    > > any results. STDERR returned:\n"+xmlOut.stderr);

    >
    > >        // save output to a temp file
    > >        writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    > > to a file

    >
    > > The problem that I am facing is it works fine but  it doesn't go
    > > through the entire loop (all the test cases we have) and almost always
    > > gets terminated prematurely completing only a partial list of cases.
    > > So that makes me wonder if there is anything wrong in the
    > > StreamGobbler and/or NewCommandExecuter. I tried a bunch of things in
    > > those classes but definitely I am missing something here. Please help.
    > > Thanks

    >
    > By 'terminated', do you mean it throws an exception or just ends without
    > running all the tests?
    >
    > --
    >
    > Knute Johnson
    > email s/nospam/knute2009/
    >
    > --
    > Posted via NewsDemon.com - Premium Uncensored Newsgroup Service
    >              ------->>>>>>http://www.NewsDemon.com<<<<<<------
    > Unlimited Access, Anonymous Accounts, Uncensored Broadband Access


    Thanks for the reply. it doesn't throw any exception or anything just
    ends without running all the tests and its not like the loop gets
    terminated at one specific point its random and could be anywhere in
    the entire loop. Sometimes it runs 100 cases n will stop whereas
    sometimes it'll run 150x cases n will stop there
     
    Vic, Mar 6, 2009
    #4
  5. Vic

    Vic Guest

    On Mar 6, 10:00 am, Vic <> wrote:
    > On Mar 6, 9:56 am, Knute Johnson <>
    > wrote:
    >
    >
    >
    > > Vic wrote:
    > > > I am not very well conversant with java and have a very basic and
    > > > limited java experience being a QA engg. I am currently looking at a
    > > > code which isn't very consistent in our testing atuomation framework.
    > > > Here is the code ->
    > > > // executes system commands and captures STDOUT and STDERR
    > > > import java.io.*;
    > > > import java.util.*;

    >
    > > > class StreamGobbler extends Thread {
    > > >     InputStream is;
    > > >     String type;
    > > >     String result;

    >
    > > >     StreamGobbler(InputStream is, String type){
    > > >    this.is = is;
    > > >    this.type = type;
    > > >     }

    >
    > > >     public void run(){
    > > >    StringBuffer buffer = new StringBuffer();

    >
    > > >    try{
    > > >        InputStreamReader isr = new InputStreamReader(is);
    > > >        BufferedReader br = new BufferedReader(isr);
    > > >        String line = null;
    > > >        while((line = br.readLine()) != null){
    > > >            buffer.append(line);
    > > >            buffer.append("\n");
    > > >        }
    > > >    }
    > > >    catch(IOException ioe){
    > > >        ioe.printStackTrace();
    > > >    }
    > > >    finally{
    > > >        result = buffer.toString();
    > > >    }
    > > >     }

    >
    > > >     public String getResult(){
    > > >    return result;
    > > >     }
    > > > }

    >
    > > > class NewCommandExecuter {
    > > >     public String stdout = null;
    > > >     public String stderr = null;
    > > >     private String line;

    >
    > > >     public NewCommandExecuter (String cmd) {
    > > >    try{
    > > >        //VikP - This needed to be modified to incorporate the change in
    > > > syntax for executing getData.pl by Mark Silverman for PR#5026342
    > > >        System.out.println("New Command Executer: EXECUTING COMMAND:
    > > > "+cmd);
    > > >        String [] execmd = {"bash","-c",cmd};
    > > >        //Process proc = Runtime.getRuntime().exec(new String[]{"bash","-
    > > > c",cmd});
    > > >        Process proc = Runtime.getRuntime().exec(execmd);
    > > >        //Process proc = Runtime.getRuntime().exec(cmd);
    > > >        StreamGobbler errorGobbler = new StreamGobbler(proc.getErrorStream
    > > > (),"ERROR");
    > > >        StreamGobbler outputGobbler = new StreamGobbler
    > > > (proc.getInputStream(),"OUTPUT");

    >
    > > >        errorGobbler.start();
    > > >        outputGobbler.start();

    >
    > > >        int val = proc.waitFor();
    > > >        System.out.println("ExitValue for Command "+cmd+" is: "+val);

    >
    > > >        stdout = outputGobbler.getResult();
    > > >        stderr = errorGobbler.getResult();

    >
    > > >        //VikP - Waits for these threads to die
    > > >        errorGobbler.join();
    > > >        outputGobbler.join();
    > > >        //
    > > >    }
    > > >    catch (Throwable t){
    > > >            t.printStackTrace();
    > > >    }

    >
    > > >    if (stdout.equals(""))
    > > >        stdout = null;

    >
    > > >    if (stderr.equals(""))
    > > >        stderr = null;
    > > >     }
    > > > }

    >
    > > > I call the the above in a loop (of about 2000 cases) as ->
    > > > NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);

    >
    > > > // verity xml output
    > > >        if (xmlOut.stdout == null)
    > > >            throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    > > > any results. STDERR returned:\n"+xmlOut.stderr);

    >
    > > >        // save output to a temp file
    > > >        writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    > > > to a file

    >
    > > > The problem that I am facing is it works fine but  it doesn't go
    > > > through the entire loop (all the test cases we have) and almost always
    > > > gets terminated prematurely completing only a partial list of cases.
    > > > So that makes me wonder if there is anything wrong in the
    > > > StreamGobbler and/or NewCommandExecuter. I tried a bunch of things in
    > > > those classes but definitely I am missing something here. Please help..
    > > > Thanks

    >
    > > By 'terminated', do you mean it throws an exception or just ends without
    > > running all the tests?

    >
    > > --

    >
    > > Knute Johnson
    > > email s/nospam/knute2009/

    >
    > > --
    > > Posted via NewsDemon.com - Premium Uncensored Newsgroup Service
    > >              ------->>>>>>http://www.NewsDemon.com<<<<<<------
    > > Unlimited Access, Anonymous Accounts, Uncensored Broadband Access

    >
    > Thanks for the reply. it doesn't throw any exception or anything just
    > ends without running all the tests and its not like the loop gets
    > terminated at one specific point its random and could be anywhere in
    > the entire loop. Sometimes it runs 100 cases n will stop whereas
    > sometimes it'll run 150x cases n will stop there


    I am running this automation on
    Solaris 10 3/05 s10_74L2a SPARC with JDK_HOME set as /home/qa/
    jdk1.5.0_02 (so I assume the jdk version in 1.5)
    I don't know if it matters but thought of mentioning. Thanks
     
    Vic, Mar 6, 2009
    #5
  6. Vic

    Vic Guest

    On Mar 6, 10:40 am, "Matt Humphrey" <> wrote:
    > "Vic" <> wrote in message
    >
    > news:...
    > On Mar 6, 9:56 am, Knute Johnson <>
    > wrote:
    >
    > <snip code>
    >
    >
    >
    > > > int val = proc.waitFor();
    > > > System.out.println("ExitValue for Command "+cmd+" is: "+val);

    >
    > > > stdout = outputGobbler.getResult();
    > > > stderr = errorGobbler.getResult();

    >
    > > > //VikP - Waits for these threads to die
    > > > errorGobbler.join();
    > > > outputGobbler.join();

    > > Thanks for the reply. it doesn't throw any exception or anything just
    > > ends without running all the tests and its not like the loop gets
    > > terminated at one specific point its random and could be anywhere in
    > > the entire loop. Sometimes it runs 100 cases n will stop whereas
    > > sometimes it'll run 150x cases n will stop there

    >
    > There is a small opportunity for error in that you are retrieving the stream
    > gobbler results before the threads have finished. That is, the test process
    > may have finished, but some output in the pipeline is still being processed
    > by the stream gobbler.  getResult will return null because the results are
    > not transferred until the reading loop has finished.
    >
    > The answer to your problem is in the loop code (please show it) because the
    > call to the executer must either return or throw an exception.   What are
    > the loop exit conditions? What happens to any exceptions (e.g. Xml-out
    > query... exception) ? What did the JVM do after the loop exited?
    >
    > Matt Humphreyhttp://www.iviz.com/


    Let me try moving the
    stdout = outputGobbler.getResult();
    stderr = errorGobbler.getResult();
    after the threads have finished.

    I just checked and there was no try catch around
    NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);
    so I've put that now and will try to rerun the automation and see if
    what I get. I'll give an update soon. Thanks for your reply
     
    Vic, Mar 6, 2009
    #6
  7. Vic

    Vic Guest

    On Mar 6, 11:04 am, Vic <> wrote:
    > On Mar 6, 10:40 am, "Matt Humphrey" <> wrote:
    >
    >
    >
    > > "Vic" <> wrote in message

    >
    > >news:....
    > > On Mar 6, 9:56 am, Knute Johnson <>
    > > wrote:

    >
    > > <snip code>

    >
    > > > > int val = proc.waitFor();
    > > > > System.out.println("ExitValue for Command "+cmd+" is: "+val);

    >
    > > > > stdout = outputGobbler.getResult();
    > > > > stderr = errorGobbler.getResult();

    >
    > > > > //VikP - Waits for these threads to die
    > > > > errorGobbler.join();
    > > > > outputGobbler.join();
    > > > Thanks for the reply. it doesn't throw any exception or anything just
    > > > ends without running all the tests and its not like the loop gets
    > > > terminated at one specific point its random and could be anywhere in
    > > > the entire loop. Sometimes it runs 100 cases n will stop whereas
    > > > sometimes it'll run 150x cases n will stop there

    >
    > > There is a small opportunity for error in that you are retrieving the stream
    > > gobbler results before the threads have finished. That is, the test process
    > > may have finished, but some output in the pipeline is still being processed
    > > by the stream gobbler.  getResult will return null because the results are
    > > not transferred until the reading loop has finished.

    >
    > > The answer to your problem is in the loop code (please show it) because the
    > > call to the executer must either return or throw an exception.   What are
    > > the loop exit conditions? What happens to any exceptions (e.g. Xml-out
    > > query... exception) ? What did the JVM do after the loop exited?

    >
    > > Matt Humphreyhttp://www.iviz.com/

    >
    > Let me try moving the
    >  stdout = outputGobbler.getResult();
    >  stderr = errorGobbler.getResult();
    > after the threads have finished.
    >
    > I just checked and there was no try catch around
    > NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);
    > so I've put that now and will try to rerun the automation and see if
    > what I get. I'll give an update soon. Thanks for your reply


    I just reran the automation again after moving the getResult() after
    the thread join() and putting the NewCommandExecuter in a try, catch
    block and I got the results for all the test cases in the loop and the
    automation finished in entirety successfully. I'll keep an eye on the
    behavior for the next couple of days and see if its all working and
    will let you know if its working. Thank you so much for the help so
    far on this topic; really appreciate it.
     
    Vic, Mar 6, 2009
    #7
  8. On 06.03.2009 18:43, Vic wrote:
    > I am not very well conversant with java and have a very basic and
    > limited java experience being a QA engg. I am currently looking at a
    > code which isn't very consistent in our testing atuomation framework.
    > Here is the code ->
    > // executes system commands and captures STDOUT and STDERR
    > import java.io.*;
    > import java.util.*;
    >
    > class StreamGobbler extends Thread {
    > InputStream is;
    > String type;
    > String result;
    >
    > StreamGobbler(InputStream is, String type){
    > this.is = is;
    > this.type = type;
    > }
    >
    > public void run(){
    > StringBuffer buffer = new StringBuffer();
    >
    > try{
    > InputStreamReader isr = new InputStreamReader(is);
    > BufferedReader br = new BufferedReader(isr);
    > String line = null;
    > while((line = br.readLine()) != null){
    > buffer.append(line);
    > buffer.append("\n");
    > }
    > }
    > catch(IOException ioe){


    I'd replace that with a catch(Throwable) in order to not loose any error
    conditions.

    > ioe.printStackTrace();
    > }
    > finally{
    > result = buffer.toString();
    > }
    > }
    >
    > public String getResult(){
    > return result;
    > }
    > }
    >
    > class NewCommandExecuter {
    > public String stdout = null;
    > public String stderr = null;
    > private String line;
    >
    > public NewCommandExecuter (String cmd) {
    > try{
    > //VikP - This needed to be modified to incorporate the change in
    > syntax for executing getData.pl by Mark Silverman for PR#5026342
    > System.out.println("New Command Executer: EXECUTING COMMAND:
    > "+cmd);
    > String [] execmd = {"bash","-c",cmd};
    > //Process proc = Runtime.getRuntime().exec(new String[]{"bash","-
    > c",cmd});
    > Process proc = Runtime.getRuntime().exec(execmd);
    > //Process proc = Runtime.getRuntime().exec(cmd);
    > StreamGobbler errorGobbler = new StreamGobbler(proc.getErrorStream
    > (),"ERROR");
    > StreamGobbler outputGobbler = new StreamGobbler
    > (proc.getInputStream(),"OUTPUT");
    >
    > errorGobbler.start();
    > outputGobbler.start();
    >
    > int val = proc.waitFor();
    > System.out.println("ExitValue for Command "+cmd+" is: "+val);
    >
    > stdout = outputGobbler.getResult();
    > stderr = errorGobbler.getResult();
    >
    > //VikP - Waits for these threads to die
    > errorGobbler.join();
    > outputGobbler.join();
    > //


    You must first join and then fetch results. Otherwise you cannot be
    sure that you got all the output properly.

    > }
    > catch (Throwable t){
    > t.printStackTrace();
    > }
    >
    > if (stdout.equals(""))
    > stdout = null;
    >
    > if (stderr.equals(""))
    > stderr = null;
    > }
    > }
    >
    > I call the the above in a loop (of about 2000 cases) as ->
    > NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);
    >
    > // verity xml output
    > if (xmlOut.stdout == null)
    > throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    > any results. STDERR returned:\n"+xmlOut.stderr);
    >
    > // save output to a temp file
    > writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    > to a file
    >
    > The problem that I am facing is it works fine but it doesn't go
    > through the entire loop (all the test cases we have) and almost always
    > gets terminated prematurely completing only a partial list of cases.
    > So that makes me wonder if there is anything wrong in the
    > StreamGobbler and/or NewCommandExecuter. I tried a bunch of things in
    > those classes but definitely I am missing something here. Please help.
    > Thanks


    No errors? No exceptions? Are you sure it's caused by the exeternal
    command execution? What if you replace that with a dummy which simply
    prints command lines to stdout?

    Cheers

    robert
     
    Robert Klemme, Mar 7, 2009
    #8
  9. Vic

    Lew Guest

    Vic wrote:
    >> try{
    >> InputStreamReader isr = new InputStreamReader(is);
    >> BufferedReader br = new BufferedReader(isr);
    >> String line = null;
    >> while((line = br.readLine()) != null){
    >> buffer.append(line);
    >> buffer.append("\n");
    >> }
    >> }
    >> catch(IOException ioe){


    Robert Klemme wrote:
    > I'd replace that with a catch(Throwable) in order to not loose [sic] any error
    > conditions.


    That's usually terrible advice.

    >> ioe.printStackTrace();


    The problem here is that the exception isn't handled, and is therefore loosed
    (actual meaning) on the rest of the program.

    >> }


    Both of y'all need to study exception-handling strategies.

    The basic tenet is "catch, log, and gracefully exit."

    Not all Throwables are equally catchable, nor as easy to escape from.

    --
    Lew
     
    Lew, Mar 7, 2009
    #9
  10. On 07.03.2009 18:07, Lew wrote:
    > Vic wrote:
    >>> try{
    >>> InputStreamReader isr = new InputStreamReader(is);
    >>> BufferedReader br = new BufferedReader(isr);
    >>> String line = null;
    >>> while((line = br.readLine()) != null){
    >>> buffer.append(line);
    >>> buffer.append("\n");
    >>> }
    >>> }
    >>> catch(IOException ioe){

    >
    > Robert Klemme wrote:
    >> I'd replace that with a catch(Throwable) in order to not loose [sic]
    >> any error conditions.

    >
    > That's usually terrible advice.


    Lew, while I generally appreciate your comments I do not find this one
    especially helpful as you do not provide any reasoning. I'll present my
    reasoning: with a catch all on a thread's main level (method run) and
    output of stack trace the code does exactly what you request (catch,
    log, exit gracefully).

    Having said that there are many other things that can be improved but I
    did not want start rewriting the whole piece.

    >>> ioe.printStackTrace();

    >
    > The problem here is that the exception isn't handled, and is therefore
    > loosed (actual meaning) on the rest of the program.


    You got me stumped - is "loosed" proper English? Is this some form of
    colloquial English that I'm missing?

    > The basic tenet is "catch, log, and gracefully exit."


    See above.

    > Not all Throwables are equally catchable, nor as easy to escape from.


    Well, yes. Could be that there is an Error which would prevent
    subsequent output of the Exception. But at least the single catch block
    for Throwable makes sure that everything that can be handled is caught
    here. You could of course have separate handlers for different Thrable
    subclasses but since we're in run() there's little other to do than to
    output the exception and terminate. Of course, when using a proper
    logging framework it would make more sense to distinguish logging levels
    etc.

    Cheers

    robert
     
    Robert Klemme, Mar 8, 2009
    #10
  11. Vic

    Vic Guest

    On Mar 7, 6:47 am, Robert Klemme <> wrote:
    > On 06.03.2009 18:43, Vic wrote:
    >
    >
    >
    > > I am not very well conversant with java and have a very basic and
    > > limited java experience being a QA engg. I am currently looking at a
    > > code which isn't very consistent in our testing atuomation framework.
    > > Here is the code ->
    > > // executes system commands and captures STDOUT and STDERR
    > > import java.io.*;
    > > import java.util.*;

    >
    > > class StreamGobbler extends Thread {
    > >     InputStream is;
    > >     String type;
    > >     String result;

    >
    > >     StreamGobbler(InputStream is, String type){
    > >    this.is = is;
    > >    this.type = type;
    > >     }

    >
    > >     public void run(){
    > >    StringBuffer buffer = new StringBuffer();

    >
    > >    try{
    > >        InputStreamReader isr = new InputStreamReader(is);
    > >        BufferedReader br = new BufferedReader(isr);
    > >        String line = null;
    > >        while((line = br.readLine()) != null){
    > >            buffer.append(line);
    > >            buffer.append("\n");
    > >        }
    > >    }
    > >    catch(IOException ioe){

    >
    > I'd replace that with a catch(Throwable) in order to not loose any error
    > conditions.
    >
    >
    >
    > >        ioe.printStackTrace();
    > >    }
    > >    finally{
    > >        result = buffer.toString();
    > >    }
    > >     }

    >
    > >     public String getResult(){
    > >    return result;
    > >     }
    > > }

    >
    > > class NewCommandExecuter {
    > >     public String stdout = null;
    > >     public String stderr = null;
    > >     private String line;

    >
    > >     public NewCommandExecuter (String cmd) {
    > >    try{
    > >        //VikP - This needed to be modified to incorporate the change in
    > > syntax for executing getData.pl by Mark Silverman for PR#5026342
    > >        System.out.println("New Command Executer: EXECUTING COMMAND:
    > > "+cmd);
    > >        String [] execmd = {"bash","-c",cmd};
    > >        //Process proc = Runtime.getRuntime().exec(new String[]{"bash","-
    > > c",cmd});
    > >        Process proc = Runtime.getRuntime().exec(execmd);
    > >        //Process proc = Runtime.getRuntime().exec(cmd);
    > >        StreamGobbler errorGobbler = new StreamGobbler(proc.getErrorStream
    > > (),"ERROR");
    > >        StreamGobbler outputGobbler = new StreamGobbler
    > > (proc.getInputStream(),"OUTPUT");

    >
    > >        errorGobbler.start();
    > >        outputGobbler.start();

    >
    > >        int val = proc.waitFor();
    > >        System.out.println("ExitValue for Command "+cmd+" is: "+val);

    >
    > >        stdout = outputGobbler.getResult();
    > >        stderr = errorGobbler.getResult();

    >
    > >        //VikP - Waits for these threads to die
    > >        errorGobbler.join();
    > >        outputGobbler.join();
    > >        //

    >
    > You must first join and then fetch results.  Otherwise you cannot be
    > sure that you got all the output properly.
    >
    >
    >
    > >    }
    > >    catch (Throwable t){
    > >            t.printStackTrace();
    > >    }

    >
    > >    if (stdout.equals(""))
    > >        stdout = null;

    >
    > >    if (stderr.equals(""))
    > >        stderr = null;
    > >     }
    > > }

    >
    > > I call the the above in a loop (of about 2000 cases) as ->
    > > NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);

    >
    > > // verity xml output
    > >        if (xmlOut.stdout == null)
    > >            throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    > > any results. STDERR returned:\n"+xmlOut.stderr);

    >
    > >        // save output to a temp file
    > >        writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    > > to a file

    >
    > > The problem that I am facing is it works fine but  it doesn't go
    > > through the entire loop (all the test cases we have) and almost always
    > > gets terminated prematurely completing only a partial list of cases.
    > > So that makes me wonder if there is anything wrong in the
    > > StreamGobbler and/or NewCommandExecuter. I tried a bunch of things in
    > > those classes but definitely I am missing something here. Please help.
    > > Thanks

    >
    > No errors?  No exceptions?  Are you sure it's caused by the exeternal
    > command execution?  What if you replace that with a dummy which simply
    > prints command lines to stdout?
    >
    > Cheers
    >
    >         robert


    Sorry to bother you guys and I am still not able to get it running in
    entirety. I have been following the automation results and its still
    not running 100% (stops in the middle). I moved getResult() after the
    join and used Throwable as follows -l
    errorGobbler.start();
    outputGobbler.start();

    int val = proc.waitFor();
    System.out.println("ExitValue for Command "+cmd+" is: "+val);

    //VikP - Waits for these threads to die
    errorGobbler.join();
    outputGobbler.join();
    //

    stdout = outputGobbler.getResult();
    stderr = errorGobbler.getResult();
    }
    catch (Throwable t){
    t.printStackTrace();
    }

    Also put a Try catch with Throwable in the java file from where
    NewCommandExecuter gets called as
    try{
    NewCommandExecuter xmlOut = new NewCommandExecuter(xmlOutCmd);

    /*catch (Exception e){
    System.out.println("The xmlOut command didn't get executed
    properly");
    this.getStackTrace(e);
    }*/

    // verity xml output
    if (xmlOut.stdout == null)
    throw new Exception ("Xml-out query '"+xmlOutCmd+"' didn't return
    any results. STDERR returned:\n"+xmlOut.stderr);

    // save output to a temp file
    writeStringToFile(tempXmlFile, xmlOut.stdout); // save xml output
    to a file
    }
    catch (Throwable e){
    System.out.println("Error executing xmlOutCmd");
    e.printStackTrace();
    }

    To give you guys an idea about the way we run automation is
    we do jloader -v <version> >&! out/<log file>
    So jloader basically calls LoaderAutomationRun.java (starts the loop)
    and calls LoaderTestcase.java in the loop for each test case and
    inside LoaderTestcase.java it runs the commands using
    NewCommandExecuter.

    I just looked a thte out/<log file> and it just shows something like
    this ->
    TESTCASE Transaction836
    Start generation all loader files
    DEBUG: validating(transaction/*
    DEBUG: NEW load order transaction/1,
    DEBUG: FINAL NEW load order transaction/1,
    Executing generateLoaderFile(transaction/1, 0)
    dTrans: null
    Executing generateLoaderFile, adding Transaction836.0.lrd
    LOADER FILES CREATED 1
    DEBUG: Created 1 loader files
    New Command Executer: EXECUTING COMMAND: /home/qauser/620currentLOADER/
    bin/loader -u genevaman -w qa -f Transaction836.0.lrd

    As you can see the last line, the New Command Executer never returned.
    Sorry to bother you guys but I really want to get this solved soon so
    any help on this would be appreciated. Thanks in advance
     
    Vic, Mar 9, 2009
    #11
  12. Vic

    Vic Guest

    On Mar 9, 4:49 pm, "Matt Humphrey" <> wrote:
    > "Vic" <> wrote in message
    >
    > news:...
    >
    >
    >
    > > To give you guys an idea about the way we run automation is
    > > we do jloader -v <version> >&! out/<log file>
    > > So jloader basically calls LoaderAutomationRun.java (starts the loop)
    > > and calls LoaderTestcase.java in the loop for each test case and
    > > inside LoaderTestcase.java it runs the commands using
    > > NewCommandExecuter.

    >
    > > I just looked a thte out/<log file> and it just shows something like
    > > this ->
    > > TESTCASE Transaction836
    > > Start generation all loader files
    > > DEBUG: validating(transaction/*
    > > DEBUG: NEW load order transaction/1,
    > > DEBUG: FINAL NEW load order transaction/1,
    > > Executing generateLoaderFile(transaction/1, 0)
    > > dTrans: null
    > > Executing generateLoaderFile, adding Transaction836.0.lrd
    > > LOADER FILES CREATED 1
    > > DEBUG: Created 1 loader files
    > > New Command Executer: EXECUTING
    > > COMMAND: /home/qauser/620currentLOADER/
    > > bin/loader  -u genevaman -w qa  -f Transaction836.0.lrd

    >
    > > As you can see the last line, the New Command Executer never returned.
    > > Sorry to bother you guys but I really want to get this solved soon so
    > > any help on this would be appreciated. Thanks in advance

    >
    > Ok, so what you're observing is that diagnostic line "ExitValue for Command"
    > does not appear in the output, but you still haven't shown the loop code or
    > said what you mean by "never returned"  Now that we understand the previous
    > problem, we know that if the test returns null the loop will exit and your
    > main program will still stop without any messages.  This new case, however,
    > looks like the test process itself never finishes and the test driver does
    > not stop.  Is that the case?  You need to say more about what that loop is
    > and does.
    >
    > If your test driver is hanging, I'd guess that the test process itself is
    > faulty.  Perhaps it's requesting input or just plain stuck.  If your test
    > driver is halting, the answer will be in how it halts.  It may fail if New
    > Command Executer throws an exception prior to waitFor.  It's not clear that
    > stderr exceptions are appearing in your output and we know that the null
    > will cause your main to halt with no messages.
    >
    > Lew also mentioned one possible case by which the StreamGobbler's assignment
    > of the result field in its own thread is not visible to the driver thread
    > because of the lack of synchronization.  Put the keyword "volatile" on the
    > String result declaration in StreamGobbler to fix this.
    >
    > Matt Humphreyhttp://www.iviz.com/


    Thanks for the reply as I mentioned earlier jloader.java calls ->
    LoaderAutomationRun.java

    constructor -> public jloader(Vector vTestcasesList, int count) {
    LoaderAutomationRun myLoaderAutomationRun = new
    LoaderAutomationRun("Loader",count);
    myLoaderAutomationRun.run (vTestcasesList,count);
    System.exit(0);
    }
    public static void main (String args[]) {
    Automation setupAutomation = new Automation();
    jloader myAutomation;
    parseCommandLineArguments (args, "loader");


    else {
    //0 is specified for the Standard Loader
    myAutomation = new jloader(vTestcasesList, 0);
    }
    }

    and the loop inside LoaderAutomationRun.java which calls
    LoaderTestcase.java and looks like this ->
    for (int i = 0; i < vTestcasesList.size(); i++) {

    sTestcaseName = vTestcasesList.elementAt(i).toString();
    iExpected++;
    iRan++;
    System.out.println("\nTESTCASE "+sTestcaseName);

    //VicP- copying the testcase name
    LoaderTestcase.sTestcaseNameTemp = sTestcaseName;
    //

    //VicP - Transaction324 is working now to commenting out :(
    if(sTestcaseName.equals("Transaction324"))
    continue;

    try {
    myTC = new LoaderTestcase(sTestcaseName);
    myTC.run();
    bPassed = true;
    } catch (Exception e) {
    iDiffs++;
    String exceptionFileName = results_log_dir + sTestcaseName +
    ".exception";
    // save exception to log file
    try {
    Automation.writeStringToFile (exceptionFileName,
    Automation.getStackTrace(e));
    System.err.println ("\n\nTESTCASE "+sTestcaseName+" FAILED
    \n"+Automation.getStackTrace(e)+"Exception saved to file
    '"+exceptionFileName+"'\n*** TESTCASE END\n");
    } catch (Exception ex) {
    System.err.println("Exception occured while trying to write
    to file '"+exceptionFileName+"'\n"+Automation.getStackTrace(ex));
    }
    bPassed = false;
    }
    System.err.println("EXP:" +iExpected+" RAN:"+iRan+"
    DIFF:"+iDiffs);
    System.out.println("TESTCASE "+sTestcaseName
    +"\n----------------------");
    if (bPassed)
    System.err.println("->PASSED");
    else
    System.err.println("->FAILED");
    System.err.println("\n\n\n\n\n\n\n");
    }

    myTC.run() funcation in LoaderTestcase.java above calls
    NewCommandExecuter.java to execute the command
    what I meant by never returns is that the automation stops/gets killed
    either in jloader.java, LoaderAutomationRun.java or while executing
    NewcommandExecuter (not sure which one is causing the automation to
    quit in the middle of the run). As you mentioned there could be
    something wrong with jloader.java which gets killed somehow
    thanks in advance
     
    Vic, Mar 10, 2009
    #12
  13. Vic

    Vic Guest

    On Mar 10, 8:41 am, Vic <> wrote:
    > On Mar 9, 4:49 pm, "Matt Humphrey" <> wrote:
    >
    >
    >
    > > "Vic" <> wrote in message

    >
    > >news:....

    >
    > > > To give you guys an idea about the way we run automation is
    > > > we do jloader -v <version> >&! out/<log file>
    > > > So jloader basically calls LoaderAutomationRun.java (starts the loop)
    > > > and calls LoaderTestcase.java in the loop for each test case and
    > > > inside LoaderTestcase.java it runs the commands using
    > > > NewCommandExecuter.

    >
    > > > I just looked a thte out/<log file> and it just shows something like
    > > > this ->
    > > > TESTCASE Transaction836
    > > > Start generation all loader files
    > > > DEBUG: validating(transaction/*
    > > > DEBUG: NEW load order transaction/1,
    > > > DEBUG: FINAL NEW load order transaction/1,
    > > > Executing generateLoaderFile(transaction/1, 0)
    > > > dTrans: null
    > > > Executing generateLoaderFile, adding Transaction836.0.lrd
    > > > LOADER FILES CREATED 1
    > > > DEBUG: Created 1 loader files
    > > > New Command Executer: EXECUTING
    > > > COMMAND: /home/qauser/620currentLOADER/
    > > > bin/loader  -u genevaman -w qa  -f Transaction836.0.lrd

    >
    > > > As you can see the last line, the New Command Executer never returned..
    > > > Sorry to bother you guys but I really want to get this solved soon so
    > > > any help on this would be appreciated. Thanks in advance

    >
    > > Ok, so what you're observing is that diagnostic line "ExitValue for Command"
    > > does not appear in the output, but you still haven't shown the loop code or
    > > said what you mean by "never returned"  Now that we understand the previous
    > > problem, we know that if the test returns null the loop will exit and your
    > > main program will still stop without any messages.  This new case, however,
    > > looks like the test process itself never finishes and the test driver does
    > > not stop.  Is that the case?  You need to say more about what that loop is
    > > and does.

    >
    > > If your test driver is hanging, I'd guess that the test process itself is
    > > faulty.  Perhaps it's requesting input or just plain stuck.  If your test
    > > driver is halting, the answer will be in how it halts.  It may fail if New
    > > Command Executer throws an exception prior to waitFor.  It's not clear that
    > > stderr exceptions are appearing in your output and we know that the null
    > > will cause your main to halt with no messages.

    >
    > > Lew also mentioned one possible case by which the StreamGobbler's assignment
    > > of the result field in its own thread is not visible to the driver thread
    > > because of the lack of synchronization.  Put the keyword "volatile" on the
    > > String result declaration in StreamGobbler to fix this.

    >
    > > Matt Humphreyhttp://www.iviz.com/

    >
    > Thanks for the reply as I mentioned earlier jloader.java calls ->
    > LoaderAutomationRun.java
    >
    > constructor -> public jloader(Vector vTestcasesList, int count) {
    >         LoaderAutomationRun myLoaderAutomationRun = new
    > LoaderAutomationRun("Loader",count);
    >         myLoaderAutomationRun.run (vTestcasesList,count);
    >         System.exit(0);
    >     }
    > public static void main (String args[]) {
    >        Automation setupAutomation = new Automation();
    >        jloader myAutomation;
    >        parseCommandLineArguments (args, "loader");
    >
    >       else {
    >            //0 is specified for the Standard Loader
    >            myAutomation = new jloader(vTestcasesList, 0);
    >        }
    >     }
    >
    > and the loop inside LoaderAutomationRun.java which calls
    > LoaderTestcase.java and looks like this ->
    > for (int i = 0; i < vTestcasesList.size(); i++) {
    >
    >             sTestcaseName = vTestcasesList.elementAt(i).toString();
    >             iExpected++;
    >             iRan++;
    >             System.out.println("\nTESTCASE "+sTestcaseName);
    >
    >             //VicP- copying the testcase name
    >             LoaderTestcase.sTestcaseNameTemp = sTestcaseName;
    >             //
    >
    >             //VicP - Transaction324 is working now to commenting out :(
    >             if(sTestcaseName.equals("Transaction324"))
    >                 continue;
    >
    >             try {
    >                 myTC = new LoaderTestcase(sTestcaseName);
    >                 myTC.run();
    >                 bPassed = true;
    >             } catch (Exception e) {
    >                 iDiffs++;
    >                 String exceptionFileName = results_log_dir + sTestcaseName +
    > ".exception";
    >                 // save exception to log file
    >                 try {
    >                     Automation.writeStringToFile (exceptionFileName,
    > Automation.getStackTrace(e));
    >                     System.err.println ("\n\nTESTCASE "+sTestcaseName+" FAILED
    > \n"+Automation.getStackTrace(e)+"Exception saved to file
    > '"+exceptionFileName+"'\n*** TESTCASE END\n");
    >                 } catch (Exception ex) {
    >                     System.err.println("Exception occured while trying to write
    > to file '"+exceptionFileName+"'\n"+Automation.getStackTrace(ex));
    >                     }
    >                 bPassed = false;
    >             }
    >             System.err.println("EXP:" +iExpected+" RAN:"+iRan+"
    > DIFF:"+iDiffs);
    >             System.out.println("TESTCASE "+sTestcaseName
    > +"\n----------------------");
    >             if (bPassed)
    >                 System.err.println("->PASSED");
    >             else
    >                 System.err.println("->FAILED");
    >             System.err.println("\n\n\n\n\n\n\n");
    >         }
    >
    > myTC.run() funcation in LoaderTestcase.java above calls
    > NewCommandExecuter.java to execute the command
    > what I meant by never returns is that the automation stops/gets killed
    > either in jloader.java, LoaderAutomationRun.java or while executing
    > NewcommandExecuter (not sure which one is causing the automation to
    > quit in the middle of the run). As you mentioned there could be
    > something wrong with jloader.java which gets killed somehow
    > thanks in advance


    o forgot to mention, myTC.run() looks like this ->
    public void run () throws java.io.IOException, Exception {
    try {
    LoaderFile[] aLoaderFilesObjects = generateAllLoaderFiles();
    logDebugMsg("Created "+aLoaderFilesObjects.length+" loader
    files", 1);
    if (aLoaderFilesObjects.length == 0)
    throw new Exception("No loader files have been created");

    for (int j = 0; j < aLoaderFilesObjects.length; j++) {
    FileLoader currentFileLoader = new FileLoader (aLoaderFilesObjects
    [j], (j==aLoaderFilesObjects.length-1));
    if ( j == (aLoaderFilesObjects.length-1) ){
    if((sTestcaseType.intern().equals("Positive") ||
    sTestcaseType.intern().equals("Negative")) &&
    LoaderAutomationRun.counter == 0)
    verify(currentFileLoader);
    }
    //loadCleanUpRecords ();
    }
    } catch (Exception e) {
    //loadCleanUpRecords ();
    throw e;
    }
    }
     
    Vic, Mar 10, 2009
    #13
  14. Vic

    Vic Guest

    On Mar 10, 10:18 am, Vic <> wrote:
    > On Mar 10, 8:41 am, Vic <> wrote:
    >
    >
    >
    > > On Mar 9, 4:49 pm, "Matt Humphrey" <> wrote:

    >
    > > > "Vic" <> wrote in message

    >
    > > >news:....

    >
    > > > > To give you guys an idea about the way we run automation is
    > > > > we do jloader -v <version> >&! out/<log file>
    > > > > So jloader basically calls LoaderAutomationRun.java (starts the loop)
    > > > > and calls LoaderTestcase.java in the loop for each test case and
    > > > > inside LoaderTestcase.java it runs the commands using
    > > > > NewCommandExecuter.

    >
    > > > > I just looked a thte out/<log file> and it just shows something like
    > > > > this ->
    > > > > TESTCASE Transaction836
    > > > > Start generation all loader files
    > > > > DEBUG: validating(transaction/*
    > > > > DEBUG: NEW load order transaction/1,
    > > > > DEBUG: FINAL NEW load order transaction/1,
    > > > > Executing generateLoaderFile(transaction/1, 0)
    > > > > dTrans: null
    > > > > Executing generateLoaderFile, adding Transaction836.0.lrd
    > > > > LOADER FILES CREATED 1
    > > > > DEBUG: Created 1 loader files
    > > > > New Command Executer: EXECUTING
    > > > > COMMAND: /home/qauser/620currentLOADER/
    > > > > bin/loader  -u genevaman -w qa  -f Transaction836.0.lrd

    >
    > > > > As you can see the last line, the New Command Executer never returned.
    > > > > Sorry to bother you guys but I really want to get this solved soon so
    > > > > any help on this would be appreciated. Thanks in advance

    >
    > > > Ok, so what you're observing is that diagnostic line "ExitValue for Command"
    > > > does not appear in the output, but you still haven't shown the loop code or
    > > > said what you mean by "never returned"  Now that we understand the previous
    > > > problem, we know that if the test returns null the loop will exit and your
    > > > main program will still stop without any messages.  This new case, however,
    > > > looks like the test process itself never finishes and the test driver does
    > > > not stop.  Is that the case?  You need to say more about what that loop is
    > > > and does.

    >
    > > > If your test driver is hanging, I'd guess that the test process itself is
    > > > faulty.  Perhaps it's requesting input or just plain stuck.  If your test
    > > > driver is halting, the answer will be in how it halts.  It may fail if New
    > > > Command Executer throws an exception prior to waitFor.  It's not clear that
    > > > stderr exceptions are appearing in your output and we know that the null
    > > > will cause your main to halt with no messages.

    >
    > > > Lew also mentioned one possible case by which the StreamGobbler's assignment
    > > > of the result field in its own thread is not visible to the driver thread
    > > > because of the lack of synchronization.  Put the keyword "volatile" on the
    > > > String result declaration in StreamGobbler to fix this.

    >
    > > > Matt Humphreyhttp://www.iviz.com/

    >
    > > Thanks for the reply as I mentioned earlier jloader.java calls ->
    > > LoaderAutomationRun.java

    >
    > > constructor -> public jloader(Vector vTestcasesList, int count) {
    > >         LoaderAutomationRun myLoaderAutomationRun = new
    > > LoaderAutomationRun("Loader",count);
    > >         myLoaderAutomationRun.run (vTestcasesList,count);
    > >         System.exit(0);
    > >     }
    > > public static void main (String args[]) {
    > >        Automation setupAutomation = new Automation();
    > >        jloader myAutomation;
    > >        parseCommandLineArguments (args, "loader");

    >
    > >       else {
    > >            //0 is specified for the Standard Loader
    > >            myAutomation = new jloader(vTestcasesList, 0);
    > >        }
    > >     }

    >
    > > and the loop inside LoaderAutomationRun.java which calls
    > > LoaderTestcase.java and looks like this ->
    > > for (int i = 0; i < vTestcasesList.size(); i++) {

    >
    > >             sTestcaseName = vTestcasesList.elementAt(i).toString();
    > >             iExpected++;
    > >             iRan++;
    > >             System.out.println("\nTESTCASE "+sTestcaseName);

    >
    > >             //VicP- copying the testcase name
    > >             LoaderTestcase.sTestcaseNameTemp = sTestcaseName;
    > >             //

    >
    > >             //VicP - Transaction324 is working now to commenting out :(
    > >             if(sTestcaseName.equals("Transaction324"))
    > >                 continue;

    >
    > >             try {
    > >                 myTC = new LoaderTestcase(sTestcaseName);
    > >                 myTC.run();
    > >                 bPassed = true;
    > >             } catch (Exception e) {
    > >                 iDiffs++;
    > >                 String exceptionFileName = results_log_dir + sTestcaseName +
    > > ".exception";
    > >                 // save exception to log file
    > >                 try {
    > >                     Automation.writeStringToFile (exceptionFileName,
    > > Automation.getStackTrace(e));
    > >                     System.err.println ("\n\nTESTCASE "+sTestcaseName+" FAILED
    > > \n"+Automation.getStackTrace(e)+"Exception saved to file
    > > '"+exceptionFileName+"'\n*** TESTCASE END\n");
    > >                 } catch (Exception ex) {
    > >                     System.err.println("Exception occured while trying to write
    > > to file '"+exceptionFileName+"'\n"+Automation.getStackTrace(ex));
    > >                     }
    > >                 bPassed = false;
    > >             }
    > >             System.err.println("EXP:" +iExpected+" RAN:"+iRan+"
    > > DIFF:"+iDiffs);
    > >             System.out.println("TESTCASE "+sTestcaseName
    > > +"\n----------------------");
    > >             if (bPassed)
    > >                 System.err.println("->PASSED");
    > >             else
    > >                 System.err.println("->FAILED");
    > >             System.err.println("\n\n\n\n\n\n\n");
    > >         }

    >
    > > myTC.run() funcation in LoaderTestcase.java above calls
    > > NewCommandExecuter.java to execute the command
    > > what I meant by never returns is that the automation stops/gets killed
    > > either in jloader.java, LoaderAutomationRun.java or while executing
    > > NewcommandExecuter (not sure which one is causing the automation to
    > > quit in the middle of the run). As you mentioned there could be
    > > something wrong with jloader.java which gets killed somehow
    > > thanks in advance

    >
    > o forgot to mention, myTC.run() looks like this ->
    > public void run () throws java.io.IOException, Exception {
    >         try {
    >             LoaderFile[] aLoaderFilesObjects = generateAllLoaderFiles();
    >             logDebugMsg("Created "+aLoaderFilesObjects.length+" loader
    > files", 1);
    >             if (aLoaderFilesObjects.length == 0)
    >                 throw new Exception("No loader files have been created");
    >
    >             for (int j = 0; j < aLoaderFilesObjects.length; j++) {
    >                 FileLoader currentFileLoader = new FileLoader (aLoaderFilesObjects
    > [j], (j==aLoaderFilesObjects.length-1));
    >                 if ( j == (aLoaderFilesObjects.length-1) ){
    >                     if((sTestcaseType.intern().equals("Positive") ||
    > sTestcaseType.intern().equals("Negative")) &&
    > LoaderAutomationRun.counter == 0)
    >                         verify(currentFileLoader);
    >                 }
    >                 //loadCleanUpRecords ();
    >             }
    >         } catch (Exception e) {
    >             //loadCleanUpRecords ();
    >             throw e;
    >         }
    >     }


    I think the NewCommandExecuter is all good as mentioned by someone
    here earlier and the problem is I think in the program where it gets
    called. I had sent some code snippets earlier. Any idea how to make it
    better? Also I was wondering many of our methods that calls
    NewCommandExecuter specify throws java.io.Exception, Exception and
    then also has a try catch block around the code where
    NewCommandExecuter.java gets executed and then there is catch
    (Thrwoable t) in the NewCommandExecuter.java as well. Is it doing too
    much of stuff with the exception handling and inspite of all this the
    program is getting terminated in the middle which is probably because
    the exception handling isn't correctly written or something. I am
    going to try to rework some of the stuff and this code has been around
    for a long time and nobody really looked at it till now.
     
    Vic, Mar 13, 2009
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hal Vaughan
    Replies:
    11
    Views:
    1,119
    Gordon Beaton
    May 22, 2006
  2. tedsuzman
    Replies:
    2
    Views:
    7,092
    Michel Claveau, résurectionné d'outre-bombe inform
    Jul 21, 2004
  3. Ted
    Replies:
    1
    Views:
    468
    Duncan Booth
    Jul 22, 2004
  4. Guillermo Riojas
    Replies:
    0
    Views:
    172
    Guillermo Riojas
    Nov 26, 2010
  5. Random Task
    Replies:
    12
    Views:
    652
    Joe Smith
    Dec 4, 2005
Loading...

Share This Page