Re: Problem with multithreading

Discussion in 'Python' started by Jeffrey Barish, Jun 25, 2009.

  1. Jeffrey Barish wrote:

    > I have a program that uses multithreading to monitor two loops. When
    > something happens in loop1, it sends a message to loop2 to have it execute
    > a command. loop2 might have to return a result. If it does, it puts the
    > result in a queue. loop1, meanwhile, would have blocked waiting for
    > something to appear in the queue. The program works for a while, but
    > eventually freezes. I know that freezing is a sign of deadlock. However,
    > I put in print statements to localize the problem and discovered something
    > weird. The freeze always occurs at a point in the code with the following
    > statements:
    >
    > print "about to try"
    > try:
    > print "in try"
    > <do something>
    >
    > I get "about to try", but not "in try". Is this observation consistent
    > with
    > the deadlock theory? If not, what could be making the program freeze at
    > the try statement? I wrote a test program using the same techniques to
    > illustrate the problem, but the test program works perfectly. I could
    > post it, though, if it would help to understand what I am doing -- and
    > what might be wrong in the real program.


    As I ponder this problem, I am beginning to believe that the problem is not
    related to multithreading. If the problem were due to a collision between
    the two threads then timing would matter, yet I find that the program
    always freezes at exactly the same statement (which executes perfectly
    hundreds of times before the freeze). Moreover, the test program that I
    wrote to test the multithreading implementation works perfectly. And
    finally, there is nothing going on related to multithreading at this point
    in the code. Why else might the program freeze at a try statement?
    --
    Jeffrey Barish
    Jeffrey Barish, Jun 25, 2009
    #1
    1. Advertising

  2. Jeffrey Barish

    larudwer Guest

    "Jeffrey Barish" <> schrieb im Newsbeitrag
    news:...
    > Jeffrey Barish wrote:
    >
    >> I have a program that uses multithreading to monitor two loops. When
    >> something happens in loop1, it sends a message to loop2 to have it
    >> execute
    >> a command. loop2 might have to return a result. If it does, it puts the
    >> result in a queue. loop1, meanwhile, would have blocked waiting for
    >> something to appear in the queue. The program works for a while, but
    >> eventually freezes. I know that freezing is a sign of deadlock.
    >> However,
    >> I put in print statements to localize the problem and discovered
    >> something
    >> weird. The freeze always occurs at a point in the code with the
    >> following
    >> statements:
    >>
    >> print "about to try"
    >> try:
    >> print "in try"
    >> <do something>
    >>
    >> I get "about to try", but not "in try". Is this observation consistent
    >> with
    >> the deadlock theory? If not, what could be making the program freeze at
    >> the try statement? I wrote a test program using the same techniques to
    >> illustrate the problem, but the test program works perfectly. I could
    >> post it, though, if it would help to understand what I am doing -- and
    >> what might be wrong in the real program.

    >
    > As I ponder this problem, I am beginning to believe that the problem is
    > not
    > related to multithreading. If the problem were due to a collision between
    > the two threads then timing would matter, yet I find that the program
    > always freezes at exactly the same statement (which executes perfectly
    > hundreds of times before the freeze). Moreover, the test program that I
    > wrote to test the multithreading implementation works perfectly. And
    > finally, there is nothing going on related to multithreading at this point
    > in the code. Why else might the program freeze at a try statement?
    > --
    > Jeffrey Barish
    >


    If you have one thread sleeping you need another running thread to waken up
    the sleeping thread.
    If the running thread terminates unexpectedly the other thread will sleep
    forever.

    Though. Since there is a try statement in your example code and the failure
    always happens there,
    there might be the chance that some unexpected exception was thrown and
    cought somewhere else in your program.
    If the Program is terminated, the last print might also have gone lost in
    some I/O buffer.
    There is no guarantee that the print statement really wasn't executed.

    Think about things like
    exception Queue.Empty
    Exception raised when non-blocking get() (or get_nowait()) is called on a
    Queue object which is empty.
    exception Queue.Full
    Exception raised when non-blocking put() (or put_nowait()) is called on a
    Queue object which is full.
    larudwer, Jun 25, 2009
    #2
    1. Advertising

  3. Jeffrey Barish

    Lou Pecora Guest

    In article <h20d7k$nih$03$-online.com>,
    "larudwer" <> wrote:

    > "Jeffrey Barish" <> schrieb im Newsbeitrag
    > news:...
    > > Jeffrey Barish wrote:
    > >
    > >> I have a program that uses multithreading to monitor two loops. When
    > >> something happens in loop1, it sends a message to loop2 to have it
    > >> execute
    > >> a command. loop2 might have to return a result. If it does, it puts the
    > >> result in a queue. loop1, meanwhile, would have blocked waiting for
    > >> something to appear in the queue. The program works for a while, but
    > >> eventually freezes. I know that freezing is a sign of deadlock.
    > >> However,
    > >> I put in print statements to localize the problem and discovered
    > >> something
    > >> weird. The freeze always occurs at a point in the code with the
    > >> following
    > >> statements:
    > >>
    > >> print "about to try"
    > >> try:
    > >> print "in try"
    > >> <do something>
    > >>
    > >> I get "about to try", but not "in try". Is this observation consistent


    Try putting a flush in after the 2nd print statement in case the output
    is left in some I/O buffer when the thing terminates. e.g.

    import sys

    .....

    try:
    print 'in try"
    sys.stdout.flush()
    <do something>

    --
    -- Lou Pecora
    Lou Pecora, Jun 25, 2009
    #3
  4. Lou Pecora wrote:

    > In article <h20d7k$nih$03$-online.com>,
    > "larudwer" <> wrote:
    >
    >> "Jeffrey Barish" <> schrieb im Newsbeitrag
    >> news:...
    >> > Jeffrey Barish wrote:
    >> >
    >> >> I have a program that uses multithreading to monitor two loops. When
    >> >> something happens in loop1, it sends a message to loop2 to have it
    >> >> execute
    >> >> a command. loop2 might have to return a result. If it does, it puts
    >> >> the
    >> >> result in a queue. loop1, meanwhile, would have blocked waiting for
    >> >> something to appear in the queue. The program works for a while, but
    >> >> eventually freezes. I know that freezing is a sign of deadlock.
    >> >> However,
    >> >> I put in print statements to localize the problem and discovered
    >> >> something
    >> >> weird. The freeze always occurs at a point in the code with the
    >> >> following
    >> >> statements:
    >> >>
    >> >> print "about to try"
    >> >> try:
    >> >> print "in try"
    >> >> <do something>
    >> >>
    >> >> I get "about to try", but not "in try". Is this observation
    >> >> consistent

    >
    > Try putting a flush in after the 2nd print statement in case the output
    > is left in some I/O buffer when the thing terminates. e.g.
    >
    > import sys
    >
    > ....
    >
    > try:
    > print 'in try"
    > sys.stdout.flush()
    > <do something>
    >


    I was hoping for some suggestions of things to think about, so thanks
    especially to those who had such suggestions. Believe it or not (and I'm
    having trouble believing it myself), I didn't think to use flush. When I
    did, I found that, indeed, the program did progress past the try statement.
    It made it to a call to GStreamer (playbin2), which has been proving itself
    intractable in my experience. Note that my test program (which works)
    excised GStreamer. The next step will be to try again to compile the
    latest version of PyGST as the version in Ubuntu 9.04 is one generation
    old. The last time I tried, the compile failed. This is the first time in
    days that I have had any hope.
    --
    Jeffrey Barish
    Jeffrey Barish, Jun 25, 2009
    #4
  5. Jeffrey Barish

    MRAB Guest

    Jeffrey Barish wrote:
    [snip]

    > Lou Pecora wrote:
    >
    >> Try putting a flush in after the 2nd print statement in case the output
    >> is left in some I/O buffer when the thing terminates. e.g.
    >>
    >> import sys
    >>
    >> ....
    >>
    >> try:
    >> print 'in try"
    >> sys.stdout.flush()
    >> <do something>
    >>

    >
    > I was hoping for some suggestions of things to think about, so thanks
    > especially to those who had such suggestions. Believe it or not (and I'm
    > having trouble believing it myself), I didn't think to use flush. When I
    > did, I found that, indeed, the program did progress past the try statement.
    > It made it to a call to GStreamer (playbin2), which has been proving itself
    > intractable in my experience. Note that my test program (which works)
    > excised GStreamer. The next step will be to try again to compile the
    > latest version of PyGST as the version in Ubuntu 9.04 is one generation
    > old. The last time I tried, the compile failed. This is the first time in
    > days that I have had any hope.


    On occasion I've needed to debug a program that's crashing, and I've
    found it best to open the log file unbuffered, otherwise I lose the
    final log messages. It saves me from having to flush each message
    explicitly.
    MRAB, Jun 25, 2009
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. insomniac

    ASP.Net 2.0 Multithreading Problem

    insomniac, Apr 27, 2006, in forum: ASP .Net
    Replies:
    2
    Views:
    2,714
    insomniac
    Apr 27, 2006
  2. Lee Garrington

    Multithreading beginner problem

    Lee Garrington, Dec 22, 2003, in forum: C++
    Replies:
    1
    Views:
    559
    Thomas Matthews
    Dec 22, 2003
  3. Replies:
    1
    Views:
    3,146
    bruce barker \(sqlwork.com\)
    May 11, 2006
  4. Diez B. Roggisch

    multithreading-problem

    Diez B. Roggisch, Jul 27, 2003, in forum: Python
    Replies:
    4
    Views:
    334
    Diez B. Roggisch
    Jul 27, 2003
  5. abhinav

    Python multithreading problem

    abhinav, Mar 26, 2006, in forum: Python
    Replies:
    3
    Views:
    378
    abhinav
    Mar 27, 2006
Loading...

Share This Page