How safe is 'timeout' ?

Discussion in 'Ruby' started by Bill Kelly, Aug 24, 2004.

  1. Bill Kelly

    Bill Kelly Guest

    Hi,

    I'm experiencing strange behavior in a Ruby app doing
    multithreaded I/O, and I seem to have narrowed it down
    to my using a timeout { } block around a condition
    variable wait. E.g.

    timeout(1.0) { @condition_variable.wait(@mutex) }

    The whole program is currently still a bit too large
    to post, but now that I'm able to reproduce the problem
    pretty reliably, I think I could narrow it down to just
    a few lines if need be. But first, I wanted to ask if
    it's even *supposed* to be reasonable to use timeout
    in this fashion.

    If it's supposed to be OK to do a timeout around a
    condition variable wait, then, I'll be happy to try
    to narrow it down to a few lines of code that reproduce
    the problem I'm seeing. (I'm seeing threads not being
    awakened when signalled. Note that these are
    completely different threads with different condition
    variables and mutexes than the one the timeout applies to.)

    If doing a timeout isn't OK around a condition variable
    wait, then, I'd like to ask what alternatives are
    recommended? I would very much like to have a timeout
    somehow on a condition variable wait.

    Thanks for your help !

    Regards,

    Bill
    Bill Kelly, Aug 24, 2004
    #1
    1. Advertising

  2. Bill Kelly wrote:
    > Hi,
    >
    > I'm experiencing strange behavior in a Ruby app doing
    > multithreaded I/O, and I seem to have narrowed it down
    > to my using a timeout { } block around a condition
    > variable wait. E.g.
    >
    > timeout(1.0) { @condition_variable.wait(@mutex) }
    >
    > The whole program is currently still a bit too large
    > to post, but now that I'm able to reproduce the problem
    > pretty reliably, I think I could narrow it down to just
    > a few lines if need be. But first, I wanted to ask if
    > it's even *supposed* to be reasonable to use timeout
    > in this fashion.
    >
    > If it's supposed to be OK to do a timeout around a
    > condition variable wait, then, I'll be happy to try
    > to narrow it down to a few lines of code that reproduce
    > the problem I'm seeing. (I'm seeing threads not being
    > awakened when signalled. Note that these are
    > completely different threads with different condition
    > variables and mutexes than the one the timeout applies to.)
    >
    > If doing a timeout isn't OK around a condition variable
    > wait, then, I'd like to ask what alternatives are
    > recommended? I would very much like to have a timeout
    > somehow on a condition variable wait.


    Probably wrong, but... when the timeout happens, a TimeoutError is
    raised in the thread that called #timeout. If you don't rescue this
    error and do something with it, the thread dies (silently, unless
    Thread.abort_on_exception is enabled). This may be why threads that have
    called #timeout later become unresponsive. (Or maybe I'm totally wrong
    and you are catching the TimeoutError somewhere and the problem is
    something else.)

    For example, the following code simply prints "main done". If you
    uncomment the two lines, you also get "thread done".

    require 'timeout'

    Thread.new do
    begin
    timeout(0.2) do
    sleep
    end
    # rescue Exception => e
    # p e
    end
    puts "thread done"
    end

    sleep 0.4
    puts "main done"

    HTH.
    Joel VanderWerf, Aug 24, 2004
    #2
    1. Advertising

  3. Bill Kelly

    Bill Kelly Guest

    Hi Joel,

    From: "Joel VanderWerf" <>
    > Bill Kelly wrote:
    > >
    > > timeout(1.0) { @condition_variable.wait(@mutex) }
    > >

    > Probably wrong, but... when the timeout happens, a TimeoutError is
    > raised in the thread that called #timeout. If you don't rescue this
    > error and do something with it, the thread dies (silently, unless
    > Thread.abort_on_exception is enabled). This may be why threads that have
    > called #timeout later become unresponsive. (Or maybe I'm totally wrong
    > and you are catching the TimeoutError somewhere and the problem is
    > something else.)


    Right, the actual code doing the timeout looks like:

    def timed_wait(timeout_secs)
    timedout = false
    begin
    timeout(timeout_secs) { wait }
    rescue Timeout::Error
    timedout = true
    end
    !timedout
    end

    And, interestingly the above is being called only on the
    main thread, and the main thread keeps running. But other
    thread(s) which have their own mutexes and condition variables,
    start "not being awakened" when signalled, sporradically,
    depending on how often the above routine is called on the
    main thread.


    Regards,

    Bill
    Bill Kelly, Aug 24, 2004
    #3
  4. Bill Kelly

    Bill Kelly Guest

    [BUG?] timeout{wait} (was: Re: How safe is 'timeout' ?)

    Hi again,

    > I'm experiencing strange behavior in a Ruby app doing
    > multithreaded I/O, and I seem to have narrowed it down
    > to my using a timeout { } block around a condition
    > variable wait. E.g.
    >
    > timeout(1.0) { @condition_variable.wait(@mutex) }



    I have gotten the code down to a couple hundred lines,
    and am able to reproduce the problem easily on:

    ruby 1.8.0 (2003-08-04) [i386-mswin32]
    ruby 1.8.2 (2004-07-29) [i386-mswin32]
    ruby 1.8.0 (2003-10-24) [i686-linux]

    My apologies in advance if I turn out to be doing something
    stupid in my own code.
    http://bwk.homeip.net/ftp/buffered-io-test3.rb

    If I comment out line 216
    100.times { global_signal.timed_wait(0.003) }

    and uncomment line 213 instead
    100.times { sleep(0.003) }

    ...I don't experience the problem. But as-is, with line 216
    in place, the problem I experience is that the background_write
    thread will sporradically not awaken, even after being
    signalled repeatedly.

    If anyone is able to help confirm that this really is a
    bug, or, point out that I'm doing something stupid....
    I'd really appreciate it.

    Thanks for your help!

    Regards,

    Bill
    Bill Kelly, Aug 24, 2004
    #4
  5. Bill Kelly

    Bill Kelly Guest

    Re: [BUG?] timeout{wait}

    > > I'm experiencing strange behavior in a Ruby app doing
    > > multithreaded I/O, and I seem to have narrowed it down
    > > to my using a timeout { } block around a condition
    > > variable wait. E.g.
    > >
    > > timeout(1.0) { @condition_variable.wait(@mutex) }

    >
    >
    > I have gotten the code down to a couple hundred lines,
    > and am able to reproduce the problem easily on:
    >
    > ruby 1.8.0 (2003-08-04) [i386-mswin32]
    > ruby 1.8.2 (2004-07-29) [i386-mswin32]
    > ruby 1.8.0 (2003-10-24) [i686-linux]
    >
    > My apologies in advance if I turn out to be doing something
    > stupid in my own code.
    > http://bwk.homeip.net/ftp/buffered-io-test3.rb


    I have a bit more info. The timeout on ConditionVariable#wait
    was causing the current thread to be left in the @waiters
    queue in ConditionVariable. I have added a line to ensure
    Thread.current is removed from @waiters:

    class ConditionVariable
    def wait(mutex)
    begin
    mutex.exclusive_unlock do
    @waiters.push(Thread.current)
    Thread.stop
    end
    ensure
    mutex.lock
    @waiters.delete Thread.current # ADDED THIS LINE
    end
    end
    end

    Adding this line SEEMS to fix the primary problem I've
    been seeing, but I can't explain why. Because although
    my condition variable surrounded by the timeout was
    indeed filling up with hundreds of references to
    Thread.current in @waiters, the problem I'm seeing
    is that a totally different thread with a different
    condition variable, simply starts not waking up when
    signalled. At least, it doesn't wake up promptly,
    consistently. I've tried giving it a high priority,
    I know the other threads are mostly sleeping anyway,
    whatever, ... I'm still trying to make the program
    that reproduces the problem smaller... There's probably
    a stupid near-deadlock situation that's my own fault
    and I'm just not seeing it.

    In any case, I think something like the line I've
    added above to ConditionVariable#wait needs to be
    there. . . .


    Regards,

    Bill
    Bill Kelly, Aug 25, 2004
    #5
  6. Bill Kelly

    Bill Kelly Guest

    Re: [BUG?] timeout{wait}

    > > > I'm experiencing strange behavior in a Ruby app doing
    > > > multithreaded I/O, and I seem to have narrowed it down
    > > > to my using a timeout { } block around a condition
    > > > variable wait. E.g.
    > > >
    > > > timeout(1.0) { @condition_variable.wait(@mutex) }


    For all the avid readers of this thread, (ha ha... :)
    just wanted to follow-up that, based on how timeout itself
    is implemented, I've now switched to a timed_wait that
    takes a similar approach.

    Create an alarm thread, to signal the condition variable
    we're waiting on, after the specified time elapses:

    def timed_wait(timeout_secs)
    begin
    alarm_th = Thread.start {
    sleep timeout_secs
    signal
    }
    wait
    ensure
    alarm_th.kill if alarm_th and alarm_th.alive?
    end
    end


    This avoids having to have the Timeout::Error exception
    be raised while inside #wait itself. (See prior message
    in thread for a patch to #wait to clean up its @waiters.)

    Still have not been able to explain the weird behavior
    seen with threads not waking up using the timeout approach
    without the aforementioned patch to wait...

    Now moving on to the next prob. I'm seeing which seems
    to be an actual hang in the garbage collector (I'm not
    loading any custom extensions or C code.) . . . Will
    start new exciting thread for that.. :)


    Regards,

    Bill
    Bill Kelly, Aug 26, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gabriel Rossetti
    Replies:
    0
    Views:
    1,294
    Gabriel Rossetti
    Aug 29, 2008
  2. Replies:
    1
    Views:
    326
    Brian Candler
    Aug 12, 2003
  3. Aredridel

    Not just $SAFE, but damn $SAFE

    Aredridel, Sep 2, 2004, in forum: Ruby
    Replies:
    19
    Views:
    226
  4. Mark Probert

    Timeout::timeout and Socket timeout

    Mark Probert, Oct 6, 2004, in forum: Ruby
    Replies:
    1
    Views:
    1,269
    Brian Candler
    Oct 6, 2004
  5. Farrel Lifson

    $SAFE =4 safe enough?

    Farrel Lifson, Aug 29, 2006, in forum: Ruby
    Replies:
    7
    Views:
    95
    Eric Hodel
    Aug 31, 2006
Loading...

Share This Page