Newly created thread blocks all others in Ruby 1.8.7 patchlevel 72on Cygwin

Discussion in 'Ruby' started by Daniel Merk, Aug 19, 2008.

  1. Daniel Merk

    Daniel Merk Guest

    [Note: parts of this message were removed to make it a legal post.]

    Hello together,

    I am using the following Ruby version:

    ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]

    I have problems with threads now that did not occur in Ruby 1.8.6. I execute
    the following code (derived from an
    example in the Ruby core documentation):


    require 'thread'

    count1 = count2 = 0
    puts "starting thread a"
    a = Thread.new do
    loop { count1 += 1 }
    end
    a.priority = -1

    puts "starting thread b"
    b = Thread.new do
    loop { count2 += 1 }
    end
    b.priority = -2
    sleep 1 #=> 1
    Thread.critical = 1

    puts "count1: #{count1}"
    puts "count2: #{count2}"


    This code never starts thread b. It seems that thread a blocks all
    other threads once it runs (though its priority is lowered), even the
    main thread.
    If I put a sleep command into the thread a loop, thread b starts, but
    the problem remains, since then thread b blocks all others from
    running.

    The Ruby 1.8.7 release notes speak of a change in the mutex
    implementation in C. Could that be a problem?

    Thanks for any help
    Daniel
     
    Daniel Merk, Aug 19, 2008
    #1
    1. Advertising

  2. Daniel Merk

    Pit Capitain Guest

    Re: Newly created thread blocks all others in Ruby 1.8.7 patchlevel72 on Cygwin

    2008/8/19 Daniel Merk <>:
    > I am using the following Ruby version:
    >
    > ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]


    Hello Daniel, I don't have 1.8.7 yet, I don't have cygwin, and...

    > I have problems with threads now that did not occur in Ruby 1.8.6.


    ...and I don't think that what you see is a problem. It may be that on
    1.8.6 you saw a different behavior, but I still don't think that the
    new behavior is a problem, because AFAIK Ruby doesn't guarantee that
    threads with equal priority do get equal time to run. If you create a
    thread with

    > a = Thread.new do
    > loop { count1 += 1 }
    > end


    then Ruby starts the thread as documented. I think it's perfectly
    legal that Ruby runs only the new thread without scheduling other
    threads, including the main thread, as long as there's no other thread
    with higher priority.

    > a.priority = -1


    If you finally get there, then it should be the main thread again who
    is blocking thread "a".

    > b = Thread.new do
    > loop { count2 += 1 }
    > end


    Here the same applies as above. Thread "b" could run all the time.

    > b.priority = -2


    Now it should be: main thread blocks "a" blocks "b".

    > sleep 1 #=> 1


    Give "a" some more time to run.

    > Thread.critical = 1
    >
    > puts "count1: #{count1}"
    > puts "count2: #{count2}"
    >
    > This code never starts thread b. It seems that thread a blocks all
    > other threads once it runs (though its priority is lowered), even the
    > main thread.


    See above. I think that's ok. What behavior do you expect? Or better:
    what do you want to achieve?

    Regards,
    Pit
     
    Pit Capitain, Aug 19, 2008
    #2
    1. Advertising

  3. Daniel Merk

    Daniel Merk Guest

    Re: Newly created thread blocks all others in Ruby 1.8.7 patchlevel72 on Cygwin

    [Note: parts of this message were removed to make it a legal post.]

    Hi Pit,

    thanks for your response.

    The behaviour I expect from threads in general is that they virtually run in
    parallel. So I expect each active thread to be at least scheduled, but not
    blocked, even if it has a low priority. If there is a total of two threads
    that have the same priority, I expect the thread scheduler to assign 50% of
    processing resources to each of them. In earlier Ruby versions (e.g. 1.8.6),
    this is exactly what has happened.

    I could reach that behavior in the example before (which is as I stated an
    "official" one, since you can also find it in the only ruby core
    documentation of the thread class:
    http://www.ruby-doc.org/core/classes/Thread.html#M000480) by inserting sleep
    statements into the loop-blocks. I could then control the amount of
    processing time for each thread by balancing the sleep times, therefore
    emulating priority. But then I would mimic the behavior that usually rubys
    thread scheduler should perform for me.


    Best regards
    Daniel
     
    Daniel Merk, Aug 20, 2008
    #3
  4. Daniel Merk

    Pit Capitain Guest

    Re: Newly created thread blocks all others in Ruby 1.8.7 patchlevel72 on Cygwin

    Hi Daniel.

    > The behaviour I expect from threads in general is that they virtually run in
    > parallel. So I expect each active thread to be at least scheduled, but not
    > blocked, even if it has a low priority.


    From looking at the source code of 1.8.6 and from running some tests
    this doesn't seem to be the case. Threads with lower priority are
    blocked as long as there are runnable threads with higher priority.

    > If there is a total of two threads
    > that have the same priority, I expect the thread scheduler to assign 50% of
    > processing resources to each of them. In earlier Ruby versions (e.g. 1.8.6),
    > this is exactly what has happened.


    Indeed this seems to be the case in 1.8.6, again from looking at the
    source and running some tests. If this doesn't work in Ruby 1.8.7, you
    should perhaps report it on ruby-core.

    > I could reach that behavior in the example before (which is as I stated an
    > "official" one, since you can also find it in the only ruby core
    > documentation of the thread class:
    > http://www.ruby-doc.org/core/classes/Thread.html#M000480) by inserting sleep
    > statements into the loop-blocks.


    AFAIK there's no official description of Ruby yet. There are projects
    trying to fill this gap with executable specifications.

    > I could then control the amount of
    > processing time for each thread by balancing the sleep times, therefore
    > emulating priority. But then I would mimic the behavior that usually rubys
    > thread scheduler should perform for me.


    I don't buy this "should". The documentation you've cited clearly
    says, that higher-priority threads will run before lower-priority
    threads.

    Regards,
    Pit
     
    Pit Capitain, Aug 21, 2008
    #4
  5. Daniel Merk

    Daniel Merk Guest

    Re: Newly created thread blocks all others in Ruby 1.8.7 patchlevel72 on Cygwin

    Hi Pit,

    thanks again for the response. I backgraded to Ruby 1.8.6 to make some
    tests too, the behavior is as you have described it. Here the test
    code I have used:


    Thread.main.priority = 0
    count1 = count2 = 0

    a = Thread.new do
    puts "starting thread a"
    Thread.current.priority = -1
    loop { count1 += 1 }
    end

    b = Thread.new do
    puts "starting thread b"
    Thread.current.priority = -1
    loop { count2 += 1}
    end

    puts "stalling main thread for 1 second"
    sleep 1

    a.kill; b.kill
    puts "count1: #{count1}"
    puts "count2: #{count2}"


    On Ruby 1.8.6 I get the following output:

    starting thread a
    starting thread b
    stalling main thread for 1 second
    count1: 953760
    count2: 953753


    So this is fine and what I expected (if I lower thread b priority to
    -2, count2 drop to absolute 0. This is quite awful, but ok, I'll stop
    complaining ;-). On Ruby 1.8.7 however, I just see:

    starting thread a
    starting thread b
    stalling main thread for 1 second

    No more output from here on. It seems that the sleep statement does not return.

    Thanks however for your help.

    Regards
    Daniel


    On 8/21/08, Pit Capitain <> wrote:
    > Hi Daniel.
    >
    >> The behaviour I expect from threads in general is that they virtually run
    >> in
    >> parallel. So I expect each active thread to be at least scheduled, but not
    >> blocked, even if it has a low priority.

    >
    > From looking at the source code of 1.8.6 and from running some tests
    > this doesn't seem to be the case. Threads with lower priority are
    > blocked as long as there are runnable threads with higher priority.
    >
    >> If there is a total of two threads
    >> that have the same priority, I expect the thread scheduler to assign 50%
    >> of
    >> processing resources to each of them. In earlier Ruby versions (e.g.
    >> 1.8.6),
    >> this is exactly what has happened.

    >
    > Indeed this seems to be the case in 1.8.6, again from looking at the
    > source and running some tests. If this doesn't work in Ruby 1.8.7, you
    > should perhaps report it on ruby-core.
    >
    >> I could reach that behavior in the example before (which is as I stated an
    >> "official" one, since you can also find it in the only ruby core
    >> documentation of the thread class:
    >> http://www.ruby-doc.org/core/classes/Thread.html#M000480) by inserting
    >> sleep
    >> statements into the loop-blocks.

    >
    > AFAIK there's no official description of Ruby yet. There are projects
    > trying to fill this gap with executable specifications.
    >
    >> I could then control the amount of
    >> processing time for each thread by balancing the sleep times, therefore
    >> emulating priority. But then I would mimic the behavior that usually rubys
    >> thread scheduler should perform for me.

    >
    > I don't buy this "should". The documentation you've cited clearly
    > says, that higher-priority threads will run before lower-priority
    > threads.
    >
    > Regards,
    > Pit
    >
    >
     
    Daniel Merk, Aug 22, 2008
    #5
  6. Daniel Merk

    ara.t.howard Guest

    Re: Newly created thread blocks all others in Ruby 1.8.7 patchlevel72 on Cygwin

    On Aug 19, 2008, at 9:22 AM, Daniel Merk wrote:

    > require 'thread'
    >
    > count1 = count2 = 0
    > puts "starting thread a"
    > a = Thread.new do
    > loop { count1 += 1 }
    > end
    > a.priority = -1
    >
    > puts "starting thread b"
    > b = Thread.new do
    > loop { count2 += 1 }
    > end
    > b.priority = -2
    > sleep 1 #=> 1
    > Thread.critical = 1
    >
    > puts "count1: #{count1}"
    > puts "count2: #{count2}"
    >
    >
    > This code never starts thread b. It seems that thread a blocks all
    > other threads once it runs (though its priority is lowered), even the
    > main thread.
    > If I put a sleep command into the thread a loop, thread b starts, but
    > the problem remains, since then thread b blocks all others from
    > running.


    that code looks completely non-deterministic to me and i've had
    similar issues many times in the past. i think the fundamental error
    is mistaking the creation of a thread for scheduling it. i've used
    the following pattern many times to force my code to know that a
    thread has started processing before continuing:

    cfp:~ > ruby186 a.rb
    starting thread a
    starting thread b
    count1: 4580083
    count2: 44732



    cfp:~ > ruby187 a.rb
    starting thread a
    starting thread b
    count1: 4394267
    count2: 37032



    cfp:~ > cat a.rb

    require 'thread'

    q = Queue.new
    count1 = count2 = 0

    a =
    Thread.new do
    puts "starting thread a"
    q.push Thread.current
    loop { count1 += 1 }
    end

    a = q.pop # wait for the thread to start
    a.priority = -1


    b =
    Thread.new do
    puts "starting thread b"
    q.push Thread.current
    loop { count2 += 1 }
    end

    b = q.pop # wait for the thread to start
    b.priority = -2

    sleep 1 #=> 1
    Thread.critical = 1

    puts "count1: #{count1}"
    puts "count2: #{count2}"


    i'm sorry that i cannot test this on cygwin so i'm not *sure* it's the
    issue. but to me it simply looks like non-deterministic code. in
    fact, even the code above has a race condition, i only force the
    program to ensure each thread calls puts - there is nothing to make
    sure that stdout has been flushed or that the increments have run in
    the loop. to do that i think you'd need something like this


    cfp:~ > cat a.rb
    require 'thread'

    q = Queue.new
    count1 = count2 = 0

    a =
    Thread.new do
    STDERR.puts "starting thread a"
    loop do
    count1 += 1
    q.push Thread.current if count1 == 1
    end
    end

    a = q.pop # wait for the thread to start
    a.priority = -1


    b =
    Thread.new do
    puts "starting thread b"
    q.push Thread.current
    loop do
    count2 += 1
    q.push Thread.current if count2 == 1
    end
    end

    b = q.pop # wait for the thread to start
    b.priority = -2

    Thread.critical = 1

    puts "count1: #{count1}"
    puts "count2: #{count2}"



    cfp:~ > ruby186 a.rb
    starting thread a
    starting thread b
    count1: 19297
    count2: 28490



    cfp:~ > ruby187 a.rb
    starting thread a
    starting thread b
    count1: 17435
    count2: 24603


    i'd be curious how that fairs on cygwin. notice that the sleep is not
    required with the queues.

    regards.

    a @ http://codeforpeople.com/
    --
    we can deny everything, except that we have the possibility of being
    better. simply reflect on that.
    h.h. the 14th dalai lama
     
    ara.t.howard, Aug 22, 2008
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Nodir Gulyamov

    MainThread blocks all others

    Nodir Gulyamov, Aug 9, 2005, in forum: Python
    Replies:
    17
    Views:
    544
    Nodir Gulyamov
    Aug 16, 2005
  2. matt
    Replies:
    1
    Views:
    274
    George Ogata
    Aug 6, 2004
  3. Rit Li
    Replies:
    3
    Views:
    120
    Rob Saul
    Nov 27, 2007
  4. Urabe Shyouhei

    [ANN] Ruby 1.8.7 patchlevel 330 released

    Urabe Shyouhei, Dec 25, 2010, in forum: Ruby
    Replies:
    0
    Views:
    98
    Urabe Shyouhei
    Dec 25, 2010
  5. Travis
    Replies:
    3
    Views:
    365
Loading...

Share This Page