Problem with multiprocessing

[tleeuwenburg]

I have a problem using multiprocessing in a simple way. I created a
file, testmp.py, with the following contents:

---------------------------------------------------
import multiprocessing as mp

p = mp.Pool(5)

def f(x):
return x * x

print map(f, [1,2,3,4,5])
print p.map(f, [1,2,3,4,5])

----------------------------------------------------

I'm using 2.6 r26:66713, so not quite bleeding edge.

If I run 'python2.6 testmp.py' I get errors of exactly the kind
mentioned on docs.python.org for when multiprocessing is run using the
interactive interpreter. However, I'm obviously *not* running the
interactive interpreter.

Any suggestions?

 

[Peter Otten]

I have a problem using multiprocessing in a simple way. I created a
file, testmp.py, with the following contents:

---------------------------------------------------
import multiprocessing as mp

p = mp.Pool(5)

def f(x):
return x * x

print map(f, [1,2,3,4,5])
print p.map(f, [1,2,3,4,5])

----------------------------------------------------

I'm using 2.6 r26:66713, so not quite bleeding edge.

If I run 'python2.6 testmp.py' I get errors of exactly the kind
mentioned on docs.python.org for when multiprocessing is run using the
interactive interpreter. However, I'm obviously *not* running the
interactive interpreter.

Any suggestions?

I'm too lazy to read the docs, so I just tinkered:

import multiprocessing as mp
import testmp

p = mp.Pool(5)

def f(x):
return x * x

if __name__ == "__main__":
print map(f, [1,2,3,4,5])
print p.map(testmp.f, [1,2,3,4,5])

 

[Roy Hyunjin Han]

I have a problem using multiprocessing in a simple way. I created a
file, testmp.py, with the following contents:

---------------------------------------------------
import multiprocessing as mp

p = mp.Pool(5)

def f(x):
return x * x

print map(f, [1,2,3,4,5])
print p.map(f, [1,2,3,4,5])

I'm too lazy to read the docs, so I just tinkered:

import multiprocessing as mp
import testmp

p = mp.Pool(5)

def f(x):
return x * x

if __name__ == "__main__":
print map(f, [1,2,3,4,5])
print p.map(testmp.f, [1,2,3,4,5])

Yes, to use the multiprocessing module, you must make your script
importable, so runtime statements should go into a __main__
conditional. This way, when multiprocessing imports the module for each
of its threads, the actual runtime code only gets executed once in the
parent thread, which has the lock. At least, that is what I think is
happening.


import multiprocessing as mp

def f(x):
return x * x

if __name__ == '__main__':
p = mp.Pool(5)
print map(f, [1,2,3,4,5])
print p.map(f, [1,2,3,4,5])


http://docs.python.org/library/multiprocessing.html

 

[Tennessee]

Yes, to use the multiprocessing module, you must make your script

importable, so runtime statements should go into a __main__
conditional. This way, when multiprocessing imports the module for each
of its threads, the actual runtime code only gets executed once in the
parent thread, which has the lock. At least, that is what I think is
happening.

Oh, that makes total sense. Thanks to both of you!

-T