Popen in main and subprocess

[pistacchio]

the following code (in the main thread) works well, I grep some files
and the search until the first 100 results are found (writing the
results to a file), then exit:

command = 'grep -F "%s" %s*.txt' % (search_string, DATA_PATH)

p = Popen(['/bin/bash', '-c', command], stdout = PIPE)
f = open(output_file, 'w+')
num_lines = MAX_RESULTS
while True:
line = p.stdout.readline()
print num_lines
if line != '':
f.write(line)
num_lines = num_lines - 1
if num_lines == 0:
break
else:
break
The very same code used into a Process subclass, always returns grep:
writing output: Broken pipe in the console:

class Search(Process):
def __init__(self, search_id, search_string):
self.search_id = search_id
self.search_string = search_string
self.grepped = ''
Process.__init__(self)

def run(self):
output_file = TMP_PATH + self.search_id

# flag if no regex chars
flag = '-F' if re.match(r"^[a-zA-Z0\ ]*$",
self.search_string) else '-P'

command = 'grep %s "%s" %s*.txt' % (flag,
self.search_string, DATA_PATH)

p = Popen(['/bin/bash', '-c', command], stdout = PIPE)
f = open(output_file, 'w+')
num_lines = MAX_RESULTS
while True:
line = p.stdout.readline()
print num_lines
if line != '':
f.write(line)
num_lines = num_lines - 1
if num_lines == 0:
break
else:
break
How come? How to fix this? Thanks

 

[Thomas Rachel]

Am 28.01.2012 11:19 schrieb pistacchio:

the following code (in the main thread) works well, I grep some files
and the search until the first 100 results are found (writing the
results to a file), then exit:

command = 'grep -F "%s" %s*.txt' % (search_string, DATA_PATH)

p = Popen(['/bin/bash', '-c', command], stdout = PIPE)

BTW: That's double weird: You can perfectly call grep without a shell
in-between. And if you really needed a shell, you would use

p = Popen(command, shell=True, stdout=PIPE)

- while the better solution is

import glob
command = ['grep', '-F', search_string] + glob.glob(DATA_PATH +
'*.txt')
p = Popen(command, stdout=PIPE)

writing output: Broken pipe in the console:

It could be that this is an output of grep which tries to write data to
its stdout. But you are only reading the first 100 lines, and
afterwards, you close the pipe (implicitly).

Maybe you should read out everything what is pending before closing the
pipe/dropping the subprocess.

BTW: I don't really understand

if num_lines == 0:
break
else:
break

HTH,


Thomas