$stdin, thread blocking and DRb (for lack of a better subject :)

Discussion in 'Ruby' started by Ken Hilton, Apr 27, 2004.

  1. Ken Hilton

    Ken Hilton Guest

    Greetings friends in Ruby-land,



    I've crafted a Ruby application and its giving me a little trouble and I'm
    hoping someone out there can give me some insight into the problem.



    The app is composed of 3 threads (other than the primary thread, which waits
    for the others to finish once its created them): the first thread is bound
    to a code block which reads email via NET::pOP3; the 2nd thread is that of a
    simple DRb server; and the third thread is bound to a code block that
    implements a simple command shell which reads lines from $stdin.



    The application works wonderfully but w/the following problem: threads 1 and
    2 block when the command shell thread reads from $stdin. Adjusting thread
    priorities had no affect on the problem. When the command shell thread is
    stopped all works as expected (email is read in the background and in
    parallel w/the DRb server accepting requests from an test harness running as
    a separate ruby session.) I've checked all the Ruby books and docs I have
    and can't find an explanation as to why this should be happening. My
    suspicion is that because $stdin belongs to the ruby application main thread
    and is being used by the command shell thread, that for some reason the
    blocking read of $stdin is holding off the other threads. This speculation
    is supported by the fact that entering a command to the shell allows all 3
    threads to run in parallel until the command is completed and the next read
    of $stdin occurs.



    Any insight ya'll can lend will be much appreciated.



    Ken.
     
    Ken Hilton, Apr 27, 2004
    #1
    1. Advertising

  2. Ken Hilton

    Ara.T.Howard Guest

    Re: $stdin, thread blocking and DRb (for lack of a better subject

    On Tue, 27 Apr 2004, Ken Hilton wrote:

    > Greetings friends in Ruby-land,
    >
    >
    >
    > I've crafted a Ruby application and its giving me a little trouble and I'm
    > hoping someone out there can give me some insight into the problem.
    >
    >
    > The app is composed of 3 threads (other than the primary thread, which waits
    > for the others to finish once its created them): the first thread is bound
    > to a code block which reads email via NET::pOP3; the 2nd thread is that of a
    > simple DRb server; and the third thread is bound to a code block that
    > implements a simple command shell which reads lines from $stdin.
    >
    >
    > The application works wonderfully but w/the following problem: threads 1 and
    > 2 block when the command shell thread reads from $stdin. Adjusting thread
    > priorities had no affect on the problem. When the command shell thread is
    > stopped all works as expected (email is read in the background and in
    > parallel w/the DRb server accepting requests from an test harness running as
    > a separate ruby session.) I've checked all the Ruby books and docs I have
    > and can't find an explanation as to why this should be happening. My
    > suspicion is that because $stdin belongs to the ruby application main thread
    > and is being used by the command shell thread, that for some reason the
    > blocking read of $stdin is holding off the other threads. This speculation
    > is supported by the fact that entering a command to the shell allows all 3
    > threads to run in parallel until the command is completed and the next read
    > of $stdin occurs.
    >
    > Any insight ya'll can lend will be much appreciated.
    >
    > Ken.


    in ruby, if the process goes to sleep all threads stop because they are
    non-native threads...

    this program, however, seems to work in 1.6.8 or 1.8.0 so perhaps you have
    another problem though?

    ~ > cat a.rb
    #!/usr/bin/env ruby
    require 'thread'

    begin
    require 'readline'
    def prompt s
    Readline::readline s, true
    end
    rescue
    STDOUT.sync = true
    def prompt
    print s
    gets
    end
    end

    threads = []
    data = {}
    mutex = Mutex.new

    3.times do
    threads <<
    Thread.new(threads.size) do |tid|
    loop{mutex.synchronize{(data[tid] ||= []) << [tid, Time.now]}; sleep 1}
    end
    end


    loop do
    cmd = prompt "select a thread #{ (0...threads.size).to_a.join ',' } > "
    case cmd
    when /exit/io
    exit
    when /\d+/io
    values = mutex.synchronize{data[cmd.to_i]}
    (values || []).each{|val| p val}
    end
    end

    ~ > ruby a.rb
    select a thread 0,1,2 > 1
    [1, Tue Apr 27 15:46:46 MDT 2004]
    [1, Tue Apr 27 15:46:47 MDT 2004]
    select a thread 0,1,2 > 1
    [1, Tue Apr 27 15:46:46 MDT 2004]
    [1, Tue Apr 27 15:46:47 MDT 2004]
    [1, Tue Apr 27 15:46:48 MDT 2004]
    [1, Tue Apr 27 15:46:49 MDT 2004]
    select a thread 0,1,2 > exit


    so, how do you KNOW that all three threads are not running, if you run this
    program you'll see that they do seem to - even with the blocking read... i
    think ruby tries very hard not to block the process...

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
     
    Ara.T.Howard, Apr 27, 2004
    #2
    1. Advertising

  3. Ken Hilton

    Ken Hilton Guest

    "Ara.T.Howard" <> wrote in message
    news:p...
    > On Tue, 27 Apr 2004, Ken Hilton wrote:
    >
    > > Greetings friends in Ruby-land,
    > >
    > >
    > >
    > > I've crafted a Ruby application and its giving me a little trouble and

    I'm
    > > hoping someone out there can give me some insight into the problem.
    > >
    > >
    > > The app is composed of 3 threads (other than the primary thread, which

    waits
    > > for the others to finish once its created them): the first thread is

    bound
    > > to a code block which reads email via NET::pOP3; the 2nd thread is that

    of a
    > > simple DRb server; and the third thread is bound to a code block that
    > > implements a simple command shell which reads lines from $stdin.
    > >
    > >
    > > The application works wonderfully but w/the following problem: threads 1

    and
    > > 2 block when the command shell thread reads from $stdin. Adjusting

    thread
    > > priorities had no affect on the problem. When the command shell thread

    is
    > > stopped all works as expected (email is read in the background and in
    > > parallel w/the DRb server accepting requests from an test harness

    running as
    > > a separate ruby session.) I've checked all the Ruby books and docs I

    have
    > > and can't find an explanation as to why this should be happening. My
    > > suspicion is that because $stdin belongs to the ruby application main

    thread
    > > and is being used by the command shell thread, that for some reason the
    > > blocking read of $stdin is holding off the other threads. This

    speculation
    > > is supported by the fact that entering a command to the shell allows all

    3
    > > threads to run in parallel until the command is completed and the next

    read
    > > of $stdin occurs.
    > >
    > > Any insight ya'll can lend will be much appreciated.
    > >
    > > Ken.

    >
    > in ruby, if the process goes to sleep all threads stop because they are
    > non-native threads...
    >
    > this program, however, seems to work in 1.6.8 or 1.8.0 so perhaps you have
    > another problem though?
    >
    > ~ > cat a.rb
    > #!/usr/bin/env ruby
    > require 'thread'
    >
    > begin
    > require 'readline'
    > def prompt s
    > Readline::readline s, true
    > end
    > rescue
    > STDOUT.sync = true
    > def prompt
    > print s
    > gets
    > end
    > end
    >
    > threads = []
    > data = {}
    > mutex = Mutex.new
    >
    > 3.times do
    > threads <<
    > Thread.new(threads.size) do |tid|
    > loop{mutex.synchronize{(data[tid] ||= []) << [tid, Time.now]}; sleep

    1}
    > end
    > end
    >
    >
    > loop do
    > cmd = prompt "select a thread #{ (0...threads.size).to_a.join ',' } > "
    > case cmd
    > when /exit/io
    > exit
    > when /\d+/io
    > values = mutex.synchronize{data[cmd.to_i]}
    > (values || []).each{|val| p val}
    > end
    > end
    >
    > ~ > ruby a.rb
    > select a thread 0,1,2 > 1
    > [1, Tue Apr 27 15:46:46 MDT 2004]
    > [1, Tue Apr 27 15:46:47 MDT 2004]
    > select a thread 0,1,2 > 1
    > [1, Tue Apr 27 15:46:46 MDT 2004]
    > [1, Tue Apr 27 15:46:47 MDT 2004]
    > [1, Tue Apr 27 15:46:48 MDT 2004]
    > [1, Tue Apr 27 15:46:49 MDT 2004]
    > select a thread 0,1,2 > exit
    >
    >
    > so, how do you KNOW that all three threads are not running, if you run

    this
    > program you'll see that they do seem to - even with the blocking read...

    i
    > think ruby tries very hard not to block the process...
    >
    > -a
    > --
    >

    ============================================================================
    ===
    > | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    > | PHONE :: 303.497.6469
    > | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    > | URL :: http://www.ngdc.noaa.gov/stp/
    > | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    >

    ============================================================================
    ===
    >


    Thanks Ara,

    I can't try your program due to not having "readline.rb" I that part of the
    Ruby 1.8.1 distribution? If not could you provide it to me? As for how (I
    believe) I know that my threads are not running, they log events at a very
    fine level to a shared log file (mutex protected.) As soon as the command
    shell calls $stdin.gets the threads stop logging and the DRb thread stops
    responding to remote method calls (they seem to be queued.) As soon as I
    press [Enter] on my command shell and gets() returns the log file gets
    updated, my POP3 mail fetch completes and DRb responds immediately. And, if
    I simply comment out the gets() call and add a hard coded command string
    that is processed repeatedly, all of the threads run in parallel. Hmmm....
     
    Ken Hilton, Apr 27, 2004
    #3
  4. Ken Hilton

    Ara.T.Howard Guest

    Re: $stdin, thread blocking and DRb (for lack of a better subject

    On Tue, 27 Apr 2004, Ken Hilton wrote:

    > Thanks Ara,
    >
    > I can't try your program due to not having "readline.rb" I that part of the
    > Ruby 1.8.1 distribution?


    oops.. should have been

    'rescue LoadError'

    that way it'll work w/o readline....


    > If not could you provide it to me? As for how (I believe) I know that my
    > threads are not running, they log events at a very fine level to a shared
    > log file (mutex protected.) As soon as the command shell calls $stdin.gets
    > the threads stop logging and the DRb thread stops responding to remote
    > method calls (they seem to be queued.) As soon as I press [Enter] on my
    > command shell and gets() returns the log file gets updated, my POP3 mail
    > fetch completes and DRb responds immediately. And, if I simply comment out
    > the gets() call and add a hard coded command string
    > that is processed repeatedly, all of the threads run in parallel. Hmmm....


    sounds tough...

    - does the stdin read have the mutex while trying to gets?

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
     
    Ara.T.Howard, Apr 28, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ken Hilton

    DRb & thread blocking.

    Ken Hilton, Apr 28, 2004, in forum: Ruby
    Replies:
    8
    Views:
    134
    Ken Hilton
    Apr 29, 2004
  2. Miles Keaton
    Replies:
    3
    Views:
    182
    Miles Keaton
    Mar 30, 2005
  3. Kirk Haines

    More DRb; SSL & DRB & errors

    Kirk Haines, Jul 1, 2005, in forum: Ruby
    Replies:
    0
    Views:
    129
    Kirk Haines
    Jul 1, 2005
  4. J. Wook
    Replies:
    16
    Views:
    285
    Robert Klemme
    May 16, 2007
  5. Ittay Dror
    Replies:
    1
    Views:
    137
    Ittay Dror
    Oct 21, 2008
Loading...

Share This Page