multiprocessing and Locks

Discussion in 'Python' started by gvv, Apr 11, 2009.

  1. gvv

    gvv Guest

    Hi All,

    I am trying to understand multiprocessing, but I am getting a Runtime
    error on the
    code below. What am I missing or doing wrong?
    Error is:
    RuntimeError: Lock objects should only be shared between processes
    through inheritance

    I am using:
    Python 2.6 (r26:66714, Nov 28 2008, 22:17:21)
    [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2

    Thanks in Advance,
    George


    import multiprocessing
    import time

    class Base(object):
    def __init__(self, id, lock):
    self.Id = id
    lock.acquire()
    self.Sleep()
    lock.release()

    def Run(self):
    pass

    def Sleep(self):
    time.sleep(5)

    class Derived(Base):
    def __init__(self, id, lock):
    Base.__init__(self, id, lock)

    def Run(self):
    print self.Id

    def RunFunc(id, lock):
    obj = Derived(id, lock)
    obj.Run()

    if __name__ == "__main__":
    lock = multiprocessing.Lock()
    Pool = multiprocessing.Pool(processes=5)
    for i in xrange(100):
    Pool.apply_async(func=RunFunc, args=(i,lock))
     
    gvv, Apr 11, 2009
    #1
    1. Advertising

  2. >>>>> gvv <> (G) wrote:

    >G> Hi All,
    >G> I am trying to understand multiprocessing, but I am getting a Runtime
    >G> error on the
    >G> code below. What am I missing or doing wrong?
    >G> Error is:
    >G> RuntimeError: Lock objects should only be shared between processes
    >G> through inheritance

    [code deleted]

    I guess you can't share locks (and probably other objects) between
    processes from a Pool. Maybe because there is no direct parent-child
    relation or so (there is a separate thread involved). There is nothing
    in the doc that explicitely forbids it AFAICT but it says that you have
    to be careful with sharing. But it could be a bug.

    You can do it with a manager, however, but this involves an additional
    process under the hood.

    if __name__ == "__main__":
    manager = multiprocessing.Manager()
    lock = manager.Lock()
    pool = multiprocessing.Pool(processes=5)
    for i in xrange(100):
    pool.apply_async(func=RunFunc, args=(i,lock))
    pool.close()
    pool.join()

    --
    Piet van Oostrum <>
    URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
    Private email:
     
    Piet van Oostrum, Apr 12, 2009
    #2
    1. Advertising

  3. gvv

    gvv Guest

    On Apr 13, 3:30 am, Piet van Oostrum <> wrote:
    > >>>>> gvv <> (G) wrote:

    > >G> Hi All,
    > >G> I am trying to understand multiprocessing, but I am getting a Runtime
    > >G> error on the
    > >G> code below. What am I missing or doing wrong?
    > >G> Error is:
    > >G> RuntimeError: Lock objects should only be shared between processes
    > >G> through inheritance

    >
    > [code deleted]
    >
    > I guess you can't share locks (and probably other objects) between
    > processes from a Pool. Maybe because there is no direct parent-child
    > relation or so (there is a separate thread involved). There is nothing
    > in the doc that explicitely forbids it AFAICT but it says that you have
    > to be careful with sharing. But it could be a bug.
    >
    > You can do it with a manager, however, but this involves an additional
    > process under the hood.
    >
    > if __name__ == "__main__":
    >     manager = multiprocessing.Manager()
    >     lock = manager.Lock()
    >     pool = multiprocessing.Pool(processes=5)
    >     for i in xrange(100):
    >         pool.apply_async(func=RunFunc, args=(i,lock))
    >     pool.close()
    >     pool.join()
    >
    > --
    > Piet van Oostrum <>
    > URL:http://pietvanoostrum.com[PGP 8DAE142BE17999C4]
    > Private email:


    Hi Piet,

    Thanks for your help. It worked.
     
    gvv, Apr 13, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tom Pester

    Migrating to ADO.NET and locks

    Tom Pester, Oct 19, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    404
    Natty Gur
    Oct 20, 2003
  2. Earl Teigrob
    Replies:
    3
    Views:
    351
    Alvin Bruney [MVP]
    Mar 4, 2004
  3. Jane Austine
    Replies:
    0
    Views:
    321
    Jane Austine
    Aug 14, 2003
  4. Ahmad Syukri bin Abdollah

    multiprocessing.sharedctypes and built-in locks

    Ahmad Syukri bin Abdollah, Mar 14, 2009, in forum: Python
    Replies:
    4
    Views:
    836
    Ahmad Syukri b
    Mar 16, 2009
  5. rooster 808
    Replies:
    0
    Views:
    645
    rooster 808
    Sep 28, 2009
Loading...

Share This Page