threading/Queue: join() and task_done() not working?(deadlock)

Discussion in 'Python' started by Gal Aviel, Feb 27, 2008.

  1. Gal Aviel

    Gal Aviel Guest

    Hello All,

    I'm seeing strange behavior where one thread waits on a queue using join() and
    the later (or so I think, according to order of the printing in STDOUT) another
    thread calls task_done() on the very same queue, however the first thread does
    not wake up, it keeps blocking forever.

    I'm using a queue of size 1, my print's:

    # @115000 : DEBUG PY wait(): waiting for task_done
    # on <Queue.Queue instance at 0x416c7a8c>
    # @115000 : DEBUG PY callback_cb() - 2
    # @115000 : DEBUG PY callback_cb() - signaling
    # task_done on <Queue.Queue instance at 0x416c7a8c>
    # @115000 : DEBUG PY callback_cb() - 3

    As you can see, the queue object is the same on both threads.

    Any help would be greatly appreciated, since I've spent days and days on this
    already !!

    Python 2.5.1, SUSE LINUX Enterprise Server 9.

    Thanks in advance, Gal Aviel.
     
    Gal Aviel, Feb 27, 2008
    #1
    1. Advertising

  2. Re: threading/Queue: join() and task_done() not working? (deadlock)

    On Feb 27, 11:06 am, Gal Aviel <> wrote:
    > Hello All,
    >
    > I'm seeing strange behavior where one thread waits on a queue using join() and
    > the later (or so I think, according to order of the printing in STDOUT) another
    > thread calls task_done() on the very same queue, however the first thread does
    > not wake up, it keeps blocking forever.
    >
    > I'm using a queue of size 1, my print's:



    Does the problem persist with a queue size of 2?


    Raymond
     
    Raymond Hettinger, Feb 27, 2008
    #2
    1. Advertising

  3. Gal Aviel

    7stud Guest

    Re: threading/Queue: join() and task_done() not working? (deadlock)

    Gal Aviel wrote:
    > Hello All,
    >
    > I'm seeing strange behavior where one thread waits on a queue using join() and
    > the later (or so I think, according to order of the printing in STDOUT) another
    > thread calls task_done() on the very same queue,


    What is task_done()? The python docs don't list any such function.
     
    7stud, Feb 27, 2008
    #3
  4. Re: threading/Queue: join() and task_done() not working? (deadlock)

    On Wed, 27 Feb 2008 13:14:25 -0800 (PST), 7stud <>
    declaimed the following in comp.lang.python:

    >
    > Gal Aviel wrote:
    > > Hello All,
    > >
    > > I'm seeing strange behavior where one thread waits on a queue using join() and
    > > the later (or so I think, according to order of the printing in STDOUT) another
    > > thread calls task_done() on the very same queue,

    >
    > What is task_done()? The python docs don't list any such function.


    I didn't even know Queue objects /had/ a join() method... The only
    ..join() I'm familiar with is the one that waits for a thread to
    complete... <G>
    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
     
    Dennis Lee Bieber, Feb 28, 2008
    #4
  5. Gal Aviel

    Gal Aviel Guest

    Re: threading/Queue: join() and task_done() notworking? (deadlock)

    7stud <bbxx789_05ss <at> yahoo.com> writes:


    > What is task_done()? The python docs don't list any such function.



    I'm using Python 2.5 and it's in the docs ...
     
    Gal Aviel, Feb 28, 2008
    #5
  6. Gal Aviel

    Gal Aviel Guest

    Re: threading/Queue: join() and task_done() notworking? (deadlock)

    Dennis Lee Bieber <wlfraed <at> ix.netcom.com> writes:

    > I didn't even know Queue objects /had/ a join() method... The only
    > .join() I'm familiar with is the one that waits for a thread to
    > complete... <G>


    Well it's probably new in Python 2.5 although I'm not sure.
     
    Gal Aviel, Feb 28, 2008
    #6
  7. Gal Aviel

    Gal Aviel Guest

    Re: threading/Queue: join() and task_done() notworking? (deadlock)

    Raymond Hettinger <python <at> rcn.com> writes:


    > Does the problem persist with a queue size of 2?
    >
    > Raymond
    >

    Unfortunately yes.
     
    Gal Aviel, Feb 28, 2008
    #7
  8. Gal Aviel

    7stud Guest

    Re: threading/Queue: join() and task_done() not working? (deadlock)

    On Feb 28, 1:47 am, Gal Aviel <> wrote:
    > 7stud <bbxx789_05ss <at> yahoo.com> writes:
    >
    > > What is task_done()?  The python docs don't list any such function.

    >
    > I'm using Python 2.5 and it's in the docs ...


    According to the docs, join() unblocks only after you call task_done()
    for every item that was entered into the Queue. However, just because
    the Queue size is 1 doesn't mean there aren't other items waiting to
    be added to the Queue. With one item in the Queue, presumably the
    task-count is 1. But, if you get() an item out of the Queue and
    another item is waiting to be added to the Queue, then it seems likely
    that the task_count will jump to 2 immediately after calling get().
    Thereafter, a call to task_done() will decrement the task_count to 1,
    which means join() will continue blocking.

    Please recognize that it is very difficult to debug imaginary code.
    If you want some help that is more relevant, then post a short example
    that demonstrates your problem. Do not post some lengthy program with
    cryptic variable names.
     
    7stud, Feb 28, 2008
    #8
  9. Re: threading/Queue: join() and task_done() not working? (deadlock)

    On Thu, 28 Feb 2008 08:48:51 +0000 (UTC), Gal Aviel <>
    declaimed the following in comp.lang.python:

    >
    > Well it's probably new in Python 2.5 although I'm not sure.
    >

    After checking the 2.5 documents online, it is...

    Apparently some means of allowing the main process to /wait/ until
    the daemon thread have nothing left to work on, instead of suddenly
    killing them mid-effort when the main process exits because no other
    means of synchronizing was done.
    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
     
    Dennis Lee Bieber, Feb 29, 2008
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jonathan Amsterdam

    Queue can result in nested monitor deadlock

    Jonathan Amsterdam, Apr 17, 2006, in forum: Python
    Replies:
    14
    Views:
    687
    Ove Svensson
    Apr 19, 2006
  2. Russell Warren

    Is Queue.Queue.queue.clear() thread-safe?

    Russell Warren, Jun 22, 2006, in forum: Python
    Replies:
    4
    Views:
    719
    Russell Warren
    Jun 27, 2006
  3. Michael Bayer

    queue deadlock possibility

    Michael Bayer, Jun 26, 2006, in forum: Python
    Replies:
    0
    Views:
    325
    Michael Bayer
    Jun 26, 2006
  4. Martin DeMello
    Replies:
    9
    Views:
    360
    alex23
    Sep 8, 2008
  5. Kris
    Replies:
    0
    Views:
    529
Loading...

Share This Page