portable multiprocessing code

[Eric Frederich]

I have written some code using Python 2.7 but I'd like these scripts
to be able to run on Red Hat 5's 2.4.3 version of Python which doesn't
have multiprocessing.
I can try to import multiprocessing and set a flag as to whether it is
available. Then I can create a Queue.Queue instead of a
multiprocessing.Queue for the arg_queue and result_queue.
Without actually trying this yet it seems like things would work okay
except for the Worker class. It seems I can conditionally replace
multiprocessing.Queue with Queue.Queue, but is there anything to
replace multiprocessing.Process with?

Are there any best practices for doing something like this?
Below is a dumb example that just counts lines in files.
What would be the best way to make this runnable in older (2.4.3)
versions of Python?


#!/usr/bin/env python

import sys
import os
import multiprocessing
import Queue

fnames = sys.argv[1:]

def SimpleWorker(func):
class SimpleWorker_wrapped(multiprocessing.Process):
def __init__(self, arg_queue, result_queue):
super(SimpleWorker_wrapped, self).__init__()
self.arg_queue = arg_queue
self.result_queue = result_queue
def run(self):
while True:
try:
args = self.arg_queue.get_nowait()
except Queue.Empty:
break
self.result_queue.put(func(*args))
return SimpleWorker_wrapped

@SimpleWorker
def line_counter(fname):
lc = len(open(fname).read().splitlines())
return fname, lc

arg_queue = multiprocessing.Queue()
result_queue = multiprocessing.Queue()
for fname in fnames:
arg_queue.put((fname,))

for i in range(multiprocessing.cpu_count()):
w = line_counter(arg_queue, result_queue)
w.start()

results = {}
for fname in sorted(fnames):
while fname not in results:
n, i = result_queue.get()
results[n] = i
print "%-40s %d" % (fname, results[fname])