multiprocessing and Locks

[gvv]

Hi All,

I am trying to understand multiprocessing, but I am getting a Runtime
error on the
code below. What am I missing or doing wrong?
Error is:
RuntimeError: Lock objects should only be shared between processes
through inheritance

I am using:
Python 2.6 (r26:66714, Nov 28 2008, 22:17:21)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2

Thanks in Advance,
George


import multiprocessing
import time

class Base(object):
def __init__(self, id, lock):
self.Id = id
lock.acquire()
self.Sleep()
lock.release()

def Run(self):
pass

def Sleep(self):
time.sleep(5)

class Derived(Base):
def __init__(self, id, lock):
Base.__init__(self, id, lock)

def Run(self):
print self.Id

def RunFunc(id, lock):
obj = Derived(id, lock)
obj.Run()

if __name__ == "__main__":
lock = multiprocessing.Lock()
Pool = multiprocessing.Pool(processes=5)
for i in xrange(100):
Pool.apply_async(func=RunFunc, args=(i,lock))

 

[Piet van Oostrum]

G> Hi All,
G> I am trying to understand multiprocessing, but I am getting a Runtime
G> error on the
G> code below. What am I missing or doing wrong?
G> Error is:
G> RuntimeError: Lock objects should only be shared between processes
G> through inheritance

[code deleted]

I guess you can't share locks (and probably other objects) between
processes from a Pool. Maybe because there is no direct parent-child
relation or so (there is a separate thread involved). There is nothing
in the doc that explicitely forbids it AFAICT but it says that you have
to be careful with sharing. But it could be a bug.

You can do it with a manager, however, but this involves an additional
process under the hood.

if __name__ == "__main__":
manager = multiprocessing.Manager()
lock = manager.Lock()
pool = multiprocessing.Pool(processes=5)
for i in xrange(100):
pool.apply_async(func=RunFunc, args=(i,lock))
pool.close()
pool.join()

 

[gvv]

G> Hi All,
G> I am trying to understand multiprocessing, but I am getting a Runtime
G> error on the
G> code below. What am I missing or doing wrong?
G> Error is:
G> RuntimeError: Lock objects should only be shared between processes
G> through inheritance

[code deleted]

I guess you can't share locks (and probably other objects) between
processes from a Pool. Maybe because there is no direct parent-child
relation or so (there is a separate thread involved). There is nothing
in the doc that explicitely forbids it AFAICT but it says that you have
to be careful with sharing. But it could be a bug.

You can do it with a manager, however, but this involves an additional
process under the hood.

if __name__ == "__main__":
    manager = multiprocessing.Manager()
    lock = manager.Lock()
    pool = multiprocessing.Pool(processes=5)
    for i in xrange(100):
        pool.apply_async(func=RunFunc, args=(i,lock))
    pool.close()
    pool.join()

Hi Piet,

Thanks for your help. It worked.