Executing command with Runtime.getRuntime.exec() fails

[Lionel]

I wrote this test to show what's happening:

public class TestClass {

public static void main(String args[]) {
testMySQLService();
}

public static void testMySQLService() {
MySQLInterfaceManager mysqlInterface =
MySQLInterfaceManager.getInstance();
String mysqlServiceCommand =
"cmd /c \"C:\\Program Files\\MySQL\\MySQL " +
"Server 5.0\\bin\\mysqld-nt.exe --install MySQL1 " +
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL
Server 5.0\\my-large.ini\"";
System.out.println(mysqlServiceCommand);
Process isRunningProcess = null;

try {
isRunningProcess =
Runtime.getRuntime().exec(mysqlServiceCommand);
readStandardError(isRunningProcess);
readStandardOut(isRunningProcess);
isRunningProcess.waitFor();
} catch(java.io.IOException ioe) {
ioe.printStackTrace();
return;
} catch(InterruptedException ie) {
//Assume processing has finished.
ie.printStackTrace();
return;
}
}

/**
* Reads the standard ouput stream of the provided Process one
character at
* a time until the stream is closed or there is no more data.
Stores the
* result in standardOut.
*/
private static void readStandardOut(final Process process) {
Thread inputStreamThread = new Thread() {
@Override
public void run() {
try {
InputStreamReader inputStreamReader =
new
InputStreamReader(process.getInputStream());
int currentChar;
while ((currentChar = inputStreamReader.read()) !=
-1) {
System.out.print((char) currentChar);
}
inputStreamReader.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
};
inputStreamThread.start();
}

/**
* Reads the error stream of the provided Process one character at
* a time until the stream is closed or there is no more data.
Stores the
* result in standardError.
*/
private static void readStandardError(final Process process) {
Thread errorStreamThread = new Thread() {
@Override
public void run() {
try {
InputStreamReader inputStreamReader =
new
InputStreamReader(process.getErrorStream());
int currentChar;
while ((currentChar = inputStreamReader.read()) !=
-1) {
System.out.print((char) currentChar);
}
inputStreamReader.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
};
errorStreamThread.start();
}
}

The output of the above is:

cmd /c "C:\Program Files\MySQL\MySQL Server 5.0\bin\mysqld-nt.exe
--install MySQL1 --defaults-file="C:\Program Files\MySQL\MySQL Server
5.0\my-large.ini"
'C:\Program' is not recognized as an internal or external command,
operable program or batch file


If I change mysqlServiceCommand to
"cmd /c \"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe
--install MySQL1";

It works as expected.

Does anyone have any idea what's going on?

Thanks

Lionel.

 

[Patrick McNicol]

I wrote this test to show what's happening:

public class TestClass {

public static void main(String args[]) {
testMySQLService();
}

public static void testMySQLService() {
MySQLInterfaceManager mysqlInterface =
MySQLInterfaceManager.getInstance();
String mysqlServiceCommand =
"cmd /c \"C:\\Program Files\\MySQL\\MySQL " +
"Server 5.0\\bin\\mysqld-nt.exe --install MySQL1 " +
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL
Server 5.0\\my-large.ini\"";
System.out.println(mysqlServiceCommand);
Process isRunningProcess = null;

try {
isRunningProcess =
Runtime.getRuntime().exec(mysqlServiceCommand);
readStandardError(isRunningProcess);
readStandardOut(isRunningProcess);
isRunningProcess.waitFor();
} catch(java.io.IOException ioe) {
ioe.printStackTrace();
return;
} catch(InterruptedException ie) {
//Assume processing has finished.
ie.printStackTrace();
return;
}
}

/**
* Reads the standard ouput stream of the provided Process one
character at
* a time until the stream is closed or there is no more data.
Stores the
* result in standardOut.
*/
private static void readStandardOut(final Process process) {
Thread inputStreamThread = new Thread() {
@Override
public void run() {
try {
InputStreamReader inputStreamReader =
new
InputStreamReader(process.getInputStream());
int currentChar;
while ((currentChar = inputStreamReader.read()) !=
-1) {
System.out.print((char) currentChar);
}
inputStreamReader.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
};
inputStreamThread.start();
}

/**
* Reads the error stream of the provided Process one character at
* a time until the stream is closed or there is no more data.
Stores the
* result in standardError.
*/
private static void readStandardError(final Process process) {
Thread errorStreamThread = new Thread() {
@Override
public void run() {
try {
InputStreamReader inputStreamReader =
new
InputStreamReader(process.getErrorStream());
int currentChar;
while ((currentChar = inputStreamReader.read()) !=
-1) {
System.out.print((char) currentChar);
}
inputStreamReader.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
};
errorStreamThread.start();
}

}

The output of the above is:

cmd /c "C:\Program Files\MySQL\MySQL Server 5.0\bin\mysqld-nt.exe
--install MySQL1 --defaults-file="C:\Program Files\MySQL\MySQL Server
5.0\my-large.ini"
'C:\Program' is not recognized as an internal or external command,
operable program or batch file

If I change mysqlServiceCommand to
"cmd /c \"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe
--install MySQL1";

It works as expected.

Does anyone have any idea what's going on?

Thanks

Lionel.

Have you tried using Runtime.exec(String[])? That's the one to use if
you have CL arguments.

 

[Gordon Beaton]

If I change mysqlServiceCommand to
"cmd /c \"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe
--install MySQL1";

It works as expected.

Does anyone have any idea what's going on?

The string you pass to exec(String) gets tokenized before it's passed
to exec(String[]). The tokenizer breaks your command into whitespace
delimited tokens without regard to escaping, quoting or other special
characters.

In your particular example, you pass your arguments to a command shell
which does further processing to the command line (and probably
respects escapes, quotes etc) before the command itself gets executed
by the shell you specified.

Exactly how the two interact is not always clear, as you've
discovered. Try running a program that simply prints its arguments to
get a better understanding of the situation. Try it both with and
without "cmd /c".

If you want a greater degree of control over this, you can avoid the
initial tokenization done by exec(String) by instead calling
exec(String[]) directly. Break the command into tokens yourself
(exactly one per array element) without using extra quotation marks.

And just for fun, IIRC the actual behaviour is slightly different on
windows and unix platforms, and depends highly on the actual choice of
command shell specified.

/gordon

--

 

[Lionel]

If I change mysqlServiceCommand to
"cmd /c \"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe
--install MySQL1";

It works as expected.

Does anyone have any idea what's going on?

The string you pass to exec(String) gets tokenized before it's passed
to exec(String[]). The tokenizer breaks your command into whitespace
delimited tokens without regard to escaping, quoting or other special
characters.

In your particular example, you pass your arguments to a command shell
which does further processing to the command line (and probably
respects escapes, quotes etc) before the command itself gets executed
by the shell you specified.

Exactly how the two interact is not always clear, as you've
discovered. Try running a program that simply prints its arguments to
get a better understanding of the situation. Try it both with and
without "cmd /c".

If you want a greater degree of control over this, you can avoid the
initial tokenization done by exec(String) by instead calling
exec(String[]) directly. Break the command into tokens yourself
(exactly one per array element) without using extra quotation marks.

More detailed response of what Patrick also said, so I'll answer both here.

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:

String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

This give the same error as before!

Do I need to tokenise at spaces?

thanks

Lionel.

 

[Ingo Menger]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:

String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

What is the "cmd /c" for, except to make the program less portable?
I think one can trust that Runtime.exec will choose the right command
interpreter.
Actually, this might be the whole problem, since Runtime.exec will
pass your command string to the system default command interpreter
(which does tokenization). So it ends up like
cmd /c "cmd /c \"your program\""
when you want
cmd /c "your program" (on Windows)
and
sh -c "your prog" (under Linux)

This give the same error as before!

Do I need to tokenise at spaces?

No.
Consider how you type this command at the command prompt:

"C:\program files\foo.exe" baz

IMPORTANT: Try it out so that you're sure it works!
Then make a java string that contains *exactly* what you typed. This
means you have to quote backslashes and quotation marks:
String command = "\"C:\\program files\\foo.exe\" baz";
Now pass command to Runtime.exec().

 

[Gordon Beaton]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:

String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

This give the same error as before!

First, you've got an extra, escaped quote after mysqld-nt.exe. Maybe
you meant to put one at the start of the line too?

Second, I don't think you need cmd /c at all here. Why can't you run
mysqld-nt.exe + arguments directly?

Third, if the shell really is necessary for some reason that isn't
obvious here, I would do it like this:

{ "cmd", "/c", "rest of command, using escaped quotes as necessary" }

I don't use windows (or cmd.exe), but that's the way e.g. /bin/sh
would require it on unix. The entire "rest of command" gets passed
unaltered to the shell, which then does any necessary expansion etc
and then executes the command.

Do I need to tokenise at spaces?

Not unless your arguments are delimited by spaces (which they don't
seem to be here).

/gordon

--

 

[Patricia Shanahan]

Ingo Menger wrote:
....

Consider how you type this command at the command prompt:

"C:\program files\foo.exe" baz

IMPORTANT: Try it out so that you're sure it works!
Then make a java string that contains *exactly* what you typed. This
means you have to quote backslashes and quotation marks:
String command = "\"C:\\program files\\foo.exe\" baz";
Now pass command to Runtime.exec().

....

Until it works, I suggest printing, or viewing in a debugger, the
command exactly as it will be passed to exec. Make sure that works at a
command prompt.

Patricia

 

[Lionel van den Berg]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:

String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

What is the "cmd /c" for, except to make the program less portable?
I think one can trust that Runtime.exec will choose the right command
interpreter.

I was having problems getting it to work in other areas without "cmd /c".

This particular part of the application will only be run if the host is
windows.

No.
Consider how you type this command at the command prompt:

"C:\program files\foo.exe" baz

IMPORTANT: Try it out so that you're sure it works!
Then make a java string that contains *exactly* what you typed. This
means you have to quote backslashes and quotation marks:
String command = "\"C:\\program files\\foo.exe\" baz";
Now pass command to Runtime.exec().

I've done exactly this. I printed out the string from the program and
pasted the command into the dos prompt and it's done what I expected.

As you would have seen I did quote back slashes and quotation marks.

thanks

Lionel.

 

[Lionel van den Berg]

Ingo Menger wrote:
...
...

Until it works, I suggest printing, or viewing in a debugger, the
command exactly as it will be passed to exec. Make sure that works at a
command prompt.

I answered this in another reply, I've done exactly this. I printed the
command and copy pasted, it worked.

thanks

Lionel

 

[Lionel van den Berg]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:

String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

This give the same error as before!

First, you've got an extra, escaped quote after mysqld-nt.exe. Maybe
you meant to put one at the start of the line too?

Yeah, my bad on that one, shouldn't have been any on that one.

Second, I don't think you need cmd /c at all here. Why can't you run
mysqld-nt.exe + arguments directly?

I think I got myself confused at some stage because it wasn't working. I
was sure that another command that I was running with MySQL required the
"cmd /c" to work, maybe it was the "net start mysql". Anyway, I ended up
under a false impression. So I tried again without "cmd /c" and it worked.

Sometimes I just get too deeply involved, I think I have proven
something that in fact I have confused myself, and so I don't take that
avenue again.

I've got responses below, but at this point it is now working - I should
have listened earlier to everyone saying don't use "cmd /c" .

Third, if the shell really is necessary for some reason that isn't
obvious here, I would do it like this:

{ "cmd", "/c", "rest of command, using escaped quotes as necessary" }

Same result doing this.

I don't use windows (or cmd.exe),

You've got my immediate respect . I wish I could say the same, but
alas, I am forced into some things.


Thanks

Lionel.

 

[Ingo Menger]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:
String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

What is the "cmd /c" for, except to make the program less portable?
I think one can trust that Runtime.exec will choose the right command
interpreter.

I was having problems getting it to work in other areas without "cmd /c".

This particular part of the application will only be run if the host is
windows.

No.
Consider how you type this command at the command prompt:

"C:\program files\foo.exe" baz

IMPORTANT: Try it out so that you're sure it works!
Then make a java string that contains *exactly* what you typed. This
means you have to quote backslashes and quotation marks:
String command = "\"C:\\program files\\foo.exe\" baz";
Now pass command to Runtime.exec().

I've done exactly this.

No, you didn't.
You are actually confused about who does tokenization/interpretation
of the command string. It's the command shell. Every "cmd /c" adds
ANOTHER level of interpretation, so that'd require ANOTHER level of
quotation on your site.
Consider "Tell John: \"Tell Mary: \\\"Tell Joe: \\\\\\\"Jack will give
a party tonight. ... "
This is silly.

It is also hard, since we have TWO sorts of quotations to do: For
once, the quotation that is needed by the command shell itself. Then,
the JAVA-string constant quotation. For example, in the latter we must
quote backslashes, but not in the former.

I printed out the string from the program and
pasted the command into the dos prompt and it's done what I expected.

Without the extra cmd /c, I guess.

 

[Nigel Wade]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:

String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

What is the "cmd /c" for, except to make the program less portable?
I think one can trust that Runtime.exec will choose the right command
interpreter.

No, it definitely will not. exec() will run exactly what you tell it to run. If
you don't explicitly include cmd.exe then it will not run cmd.exe. It will
execute the first element from the array if you invoke exec(String[]), or the
first whitespace-delimited sub-string if you invoke exec(String).

Actually, this might be the whole problem, since Runtime.exec will
pass your command string to the system default command interpreter

No it won't.

(which does tokenization). So it ends up like
cmd /c "cmd /c \"your program\""
when you want
cmd /c "your program" (on Windows)
and
sh -c "your prog" (under Linux)

Neither cmd.exe [on Windows] or sh [on Linux] will be exec'd unless you tell
exec() explicitly to do so by including it as the first argument in the string
(or string array) passed to exec.

The problem is that an array of strings is getting passed to cmd.exc. The first
argument is the executable you really want to run, and cmd.exe will attempt to
run it. All the other arguments are passed to cmd.exe, not to mysqld-nt.exe.

You can pass a single string to cmd.exe and let it work out how to execute the
string, but you only need to do this if cmd.exe is actually required. If
cmd.exe isn't needed then remove it from the list and invoke mysqld-nt.exe
directly by making it the first element of the array, with the arguments being
passed one per element in the array.

The argument to exec should be either:

String[] { "cmd.exe", "/c", "executable arg1 arg2 ..."}
or
String[] { "executable", "arg1", "arg2" ... }

 

[Ingo Menger]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:
String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};

What is the "cmd /c" for, except to make the program less portable?
I think one can trust that Runtime.exec will choose the right command
interpreter.

No, it definitely will not.

I just read the API docs and find out to my surprise that you're
right.

I didn't also know that Runtime.exec indeed does tokenization which
makes things more, not less, complicated and does not make any sense,
IMHO.

exec() will run exactly what you tell it to run.

Sure. In this case
cmd /c C:\Program Files\... further args ...

If
you don't explicitly include cmd.exe then it will not run cmd.exe.

Too bad for the "run anywhere" approach. Especially when directory
names contain spaces.

 

[Lew]

Too bad for the "run anywhere" approach. Especially when directory
names contain spaces.

Tsk. That's why there's the (String []) overload. The (String) overload is
only meant as a convenience for the simple case.

And you're talking about Runtime.exec(), whose very definition is to break
such portability. To take an intentionally non-portable call and blame it
because what it exec()s isn't portable hardly seems fair, now does it?

 

[Nigel Wade]

Too bad for the "run anywhere" approach. Especially when directory
names contain spaces.

Runtime.exec() is non-portable. If you use it you shouldn't expect your code to
"run anywhere" other than the platform where the executable you are attempting
to run is valid. Even if exec() had a notion of "intelligence" and did run cmd
on Windows and sh on Linux how portable would, for example, Runtime.exec("ls")
be?

Directory and filenames with spaces are only an issue if you use
the .exec(String) variant which splits the string into fields based on
whitespace delimiters. If you have spaces within an argument use
the .exec(String[]) variant. If you need quotes in an argument it gets messy...

 

[Ingo Menger]

Too bad for the "run anywhere" approach. Especially when directory
names contain spaces.

Tsk. That's why there's the (String []) overload. The (String) overload is
only meant as a convenience for the simple case.

I see that. Yet it is at if it were designed to mislead former C/C++/
perl Programmers, to lure them into the deception that
Runtime.exec(String) was something like system(3).

And you're talking about Runtime.exec(), whose very definition is to break
such portability. To take an intentionally non-portable call and blame it
because what it exec()s isn't portable hardly seems fair, now does it?

I don't blame it, I just remark that it would be nice to have a method
that runs a command string without adding another level of
interpretation. The tokenization done by Runtime.exec() is just stupid
IMHO. Now that it is brought to my attention I'll desperately try to
avoid it at all cost.

 

[Roedy Green]

String mysqlServiceCommand =
"cmd /c \"C:\\Program Files\\MySQL\\MySQL " +
"Server 5.0\\bin\\mysqld-nt.exe --install MySQL1 " +
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL
Server 5.0\\my-large.ini\"";

you mean cmd.exe

See http://mindprod.com/jgloss/exec.html
for details.

 

[Roedy Green]

String mysqlServiceCommand =

you mean cmd.exe

See http://mindprod.com/jgloss/exec.html
for details.

or even better:

You don't need a command interpreter to spawn an exe file, but you do
need the explicit .exe.

You also need quotes around filenames with embedded spaces.

Alternatively you could use one of the other exec methods that let you
specify individual parms. Then you don't need the quotes.

 

[Ingo Menger]

Runtime.exec() is non-portable.

That's exactly my point.

Even if exec() had a notion of "intelligence" and did run cmd
on Windows and sh on Linux how portable would, for example, Runtime.exec("ls")
be?

More portable, since on my windows system at least, there is an ls.
But, in fact, the portability does not depend so much on the existance
of a certain executable.
Even if ls is not accessible for some reason (which can happen under
UNIX/LINUX also for other reasons than non-existance), the java
environment still could provide the following abstraction: Execute a
command string in such a way that it works as if one had typed the
command at the command line.
Since there is a lowest common denominator between a unix shell and
cmd.exe about quoting, wildcards, etc. one could go a fairly long way.

Look, I am not saying anything against Runtime.exec(String[]). But the
"convenience" method Runtime.exec(String) is almost useless (since it
does tokenization, but does not provide for escape mechanism and
ignores quotation). It would be much more convenient (and intuitive,
and simple) to let Runtime.exec(cmdstr) be equivalent to
Runtime.exec(new String[] {"cmd.exe", "/c", cmdstr}); // Win
Runtime.exec(new String[] {"sh", "-c", cmdstr}); // Unix
.... // etc.

The appropriate command interpreter for each os could be found through
a system property.

I am curious how many java programs contain incomplete, bug ridden
implementations of basic shell features like tokenization, quotation,
wildcard matching, etc. in order to enhance the useless
Runtime.exec(String).

 

[Lionel van den Berg]

I did in fact try using exec(String[]) but perhaps I tokenized it
incorrectly. This is what I tried:
String mysqlServiceCommand[] = {"cmd", "/c",
"C:\\Program Files\\MySQL\\MySQL Server 5.0\\bin\\mysqld-nt.exe\"",
"--install", "MySQL1",
"--defaults-file=\"C:\\Program Files\\MySQL\\MySQL Server " +
"5.0\\my-large.ini\""};
What is the "cmd /c" for, except to make the program less portable?
I think one can trust that Runtime.exec will choose the right command
interpreter.

I was having problems getting it to work in other areas without "cmd /c".

This particular part of the application will only be run if the host is
windows.

This give the same error as before!
Do I need to tokenise at spaces?
No.
Consider how you type this command at the command prompt:
"C:\program files\foo.exe" baz
IMPORTANT: Try it out so that you're sure it works!
Then make a java string that contains *exactly* what you typed. This
means you have to quote backslashes and quotation marks:
String command = "\"C:\\program files\\foo.exe\" baz";
Now pass command to Runtime.exec().

I've done exactly this.

No, you didn't.

Sorry, but I'm afraid I did. You weren't watching me so you can't make
that statement.

First thing I did was start up my windows box running under qemu. I
typed out the install service command to make sure it worked and didn't
throw several errors as it has previously. Once I had the command
correct, I right clicked on the dos prompt at the top to select text,
copied the appropriate text, saved it to a file and placed it in a
network shared folder. Went to the development machine and put it into
code delimiting the quotes and back slashes as appropriate.

I ran the program, I printed out the command using
System.out.println(command) and then when it failed I copied that
command back into the dos prompt and ran it - it worked at dos.

I repeated these things several times before posting here.

Without the extra cmd /c, I guess.

Yes, sorry for not being explicity.

Thanks

Lionel.