multiprocessing: child process share changes to global variable

[Alan]

In the code below, child processes see changes to global variables caused by other child processes. That is, pool.map is producing a list where the value of ``lst`` is being (non-deterministically) shared across child processes. Standard advice is that, "Processes have independent memory space" and"each process works with its own copy" of global variables, so I thought this was not supposed to happen. In the end, the globals are not changed, which matches my expectation. Still, clearly I have misunderstood what seemedto be standard advice. Can someone clarify for me?

Expected result from pool.map: [[0],[1],[2]]
Usual result: [[0],[0,1],[0,1,2]]
Occasional result: [[0],[1],[0,2]]

Thanks,
Alan Isaac


#--------------- temp.py -------------------------
#run at Python 2.7 command prompt
import time
import multiprocessing as mp
lst = []
lstlst = []

def alist(x):
lst.append(x)
lstlst.append(lst)
print "a"
return lst

if __name__=='__main__':
pool = mp.Pool(3)
print pool.map(alist,range(3)) #UNEXPECTED RESULTS
print "b"
time.sleep(0.1)

print "c"
print lst
print lstlst

 

[Jason Friedman]

#--------------- temp.py -------------------------

#run at Python 2.7 command prompt
import time
import multiprocessing as mp
lst = []
lstlst = []

def alist(x):
lst.append(x)
lstlst.append(lst)
print "a"
return lst

if __name__=='__main__':
pool = mp.Pool(3)
print pool.map(alist,range(3)) #UNEXPECTED RESULTS
print "b"
time.sleep(0.1)

print "c"
print lst
print lstlst
--

I get several different results:

$ for i in {1..1000}; do

python2.7 temp.py | md5sum >> result
done

$ wc -l result
1000 result

$ sort result | uniq | wc -l
9

 

[Alan]

I received a suggestion off list that my results indicated that the jobs were not being distributed across the pool workers. I used mp.current_process().name to confirm this suggestion.

Alan Isaac