how to invoke the shell command and then get the result in python

[Bin Chen]

Hi,

I want to do following: get a user input regex, then pass this as a
parameter to grep, and then get the result from grep.

Any code snip to implement the similar function? I am a python newbie.

Thanks a lot.
Bin

 

[Fredrik Lundh]

I want to do following: get a user input regex, then pass this as a
parameter to grep, and then get the result from grep.

Any code snip to implement the similar function? I am a python newbie.

import os
for line in os.popen("grep pattern *.txt"):
print line,

also see os.system and subprocess.

note that if you want to write portable code, you can implement your own
"grep" using the "re" module:

import re
p = re.compile(pattern)
for index, line in enumerate(open(filename)):
if p.match(line):
print index, line,

</F>

 

[petercable]

import os
for line in os.popen("grep pattern *.txt"):
print line,

also see os.system and subprocess.

note that if you want to write portable code, you can implement your own
"grep" using the "re" module:

</F>

Also, for a wrapper around popen, try commands:

import commands

pattern = raw_input('pattern to search? ')
print commands.getoutput('grep %s *.txt' % pattern)

Pete

 

[Fredrik Lundh]

Also, for a wrapper around popen, try commands:

import commands

pattern = raw_input('pattern to search? ')
print commands.getoutput('grep %s *.txt' % pattern)

that's not quite as portable as the other alternatives, though. "grep"
is at least available for non-Unix platforms, but "commands" requires a
unix shell.

for Python 2.5 and later, you could use:

def getoutput(cmd):
from subprocess import Popen, PIPE, STDOUT
p = Popen(cmd, stdout=PIPE, stderr=STDOUT,
shell=isinstance(cmd, basestring))
return p.communicate()[0]

print getoutput(["grep", pattern, glob.glob("*.txt")])

which, if given a list instead of a string, passes the arguments
right through to the underlying process, without going through the
shell (consider searching for "-" or ";rm" with the original code).

</F>

 

[Nick Craig-Wood]

Also, for a wrapper around popen, try commands:

import commands

pattern = raw_input('pattern to search? ')
print commands.getoutput('grep %s *.txt' % pattern)

What if I entered "; rm -rf * ;" as my pattern?

Don't ever pass user input (from file/web/raw_input) to the shell if
you want to write a secure program!

If you use subprocess then you can use a sequence of args to bypass
the shell rather than a string to be passed to the shell. That will
get over lots of shell escaping problems too. Eg

from subprocess import Popen, PIPE
from glob import glob
pattern = raw_input('pattern to search? ')
files = glob("*.txt")
output = Popen(["grep", pattern] + files, stdout=PIPE).communicate()[0]
print output

You can also use subprocess to read the return code of the command and
its stderr both of which you'll need if you are programming
defensively!

 

[petercable]

What if I entered "; rm -rf * ;" as my pattern?

Assuming the script isn't setuid, this would do no more damage than the
user could do directly on the command line. I agree, when dealing with
web applications or setuid programs, direct shell access isn't a good
idea.

Pete

 

[Fredrik Lundh]

Assuming the script isn't setuid, this would do no more damage than the
user could do directly on the command line.

except that when the user is typing things into the command line, he
*knows* that he's typing things into the command line.

</F>

 

[Nick Craig-Wood]

except that when the user is typing things into the command line, he
*knows* that he's typing things into the command line.

Aye!

Who is to say that this script won't get re-used innocently in a web
application?

And in this particular example we were talking about typing regular
expressions into the shell, which have many of the same metacharacters
as the shell. So even an innocent use of the above can cause
problems.

Just say no to passing user input (from anywhere at all) via the
shell! That (along with SQL injection attacks which are very similar
in concept) is one of the most common security attacks for scripting
languages like Python when used in a web environment.