threading sync

Discussion in 'Ruby' started by Mage, Jan 25, 2006.

  1. Mage

    Mage Guest

    Hello,

    I was thinking of writing my first threaded app in Ruby. One thread
    should fetch data from the database, another should write it to a file.

    I would like to use an array, which has hash elements. I append new
    elements to the end, and I write to the file from the beginning (FIFO).

    My question is that do I really need mutex? Is appending a hash to the
    end of an array atomic, or not? How can I know which method is atomic?

    Mage
    Mage, Jan 25, 2006
    #1
    1. Advertising

  2. Mage wrote:
    > Hello,
    >
    > I was thinking of writing my first threaded app in Ruby. One thread
    > should fetch data from the database, another should write it to a
    > file.
    >
    > I would like to use an array, which has hash elements. I append new
    > elements to the end, and I write to the file from the beginning
    > (FIFO).
    >
    > My question is that do I really need mutex? Is appending a hash to the
    > end of an array atomic, or not? How can I know which method is atomic?


    In your case you probably better use a Queue which saves you the
    synchronization overhead (the Queue does it internally). Something along
    the lines of this (untested):

    require 'thread'

    QUEUE = Queue.new

    threads = [
    Thread.new(QUEUE) do |queue|
    File.open("foo", "w") do |io|
    while x = queue.deq
    io.puts x
    end
    end
    end,
    Thread.new(QUEUE) do |queue|
    while rec = fetch_from_db()
    queue.enq rec
    end
    queue.enc false
    end
    ]

    thread.each {|th|th.join}

    Note, if you like you can increase the number of DB writer and / or file
    reader threads. You can as well leave the db reader in the main loop if
    you need only one. Depending on your situation you may need a different
    end of processing detection.

    Kind regards

    robert
    Robert Klemme, Jan 25, 2006
    #2
    1. Advertising

  3. Mage

    Mage Guest

    Robert Klemme wrote:

    >
    >In your case you probably better use a Queue which saves you the
    >synchronization overhead (the Queue does it internally). Something along
    >the lines of this (untested):
    >
    >

    Thank you. I see.

    However, in cases when there is no existing solution, should I write a
    tester unit that determines about a method its atomicness?

    Mage
    Mage, Jan 25, 2006
    #3
  4. Mage wrote:
    > Robert Klemme wrote:
    >
    >>
    >> In your case you probably better use a Queue which saves you the
    >> synchronization overhead (the Queue does it internally). Something
    >> along the lines of this (untested):
    >>
    >>

    > Thank you. I see.
    >
    > However, in cases when there is no existing solution, should I write a
    > tester unit that determines about a method its atomicness?


    IMHO it's generally a bad idea to determine thread safety properties of
    code by testing. The test may fail, in which case you got a definitve
    answer (it's unsafe). But the test may succeed and then you don't know
    whether it was by accident (timing) or by design / implementation.

    You could look at the source code of Ruby but on one hand that may change
    and on the other hand there will be definitively issues when Ruby 2.0
    comes out and supports native threads.

    If you build a MT application you should design MT right into it, which
    also means that you have to think about which resources (aka objects) are
    accessed from multiple threads and how you synchronize access to them.

    Kind regards

    robert
    Robert Klemme, Jan 25, 2006
    #4
  5. > My question is that do I really need mutex?

    No, you don't.

    You could as well use Thread.exclusive. This prevents other,
    excisting threads from being run. If the given block can't get
    blocked (e.g. by IO), it's much faster then Mutex.

    gegroet,
    Erik V. - http://www.erikveen.dds.nl/
    Erik Veenstra, Jan 25, 2006
    #5
  6. On Jan 25, 2006, at 5:41 AM, Mage wrote:

    > Is appending a hash to the end of an array atomic, or not? How can
    > I know which method is atomic?


    In general, assume nothing is atomic in Ruby. Even a simple
    expression like:

    my_val = 5

    Is many steps inside the interpreter. If you're going to thread,
    handle all shared resources accordingly.

    James Edward Gray II
    James Edward Gray II, Jan 25, 2006
    #6
  7. Mage

    Eric Hodel Guest

    On Jan 25, 2006, at 5:28 AM, Erik Veenstra wrote:

    >> My question is that do I really need mutex?

    >
    > No, you don't.
    >
    > You could as well use Thread.exclusive. This prevents other,
    > excisting threads from being run. If the given block can't get
    > blocked (e.g. by IO), it's much faster then Mutex.


    No, Thread.exclusive (which uses Thread.critical) does not prevent
    other threads from being run. It prevents them from being scheduled
    automatically. You can still switch threads manually while
    Thread.critical is true.

    $ cat exclusive.rb
    require 'thread'

    Thread.exclusive do
    Thread.new { sleep 10 }
    puts 'should be instant'
    end

    $ time ruby exclusive.rb
    should be instant

    real 0m10.061s
    user 0m0.016s
    sys 0m0.013s
    $ cat mutex.rb
    require 'thread'

    m = Mutex.new

    m.synchronize do
    Thread.new do
    m.synchronize do sleep 10 end
    end
    puts 'should be instant'
    end

    $ time ruby mutex.rb
    should be instant

    real 0m0.060s
    user 0m0.016s
    sys 0m0.014s


    --
    Eric Hodel - - http://segment7.net
    This implementation is HODEL-HASH-9600 compliant

    http://trackmap.robotcoop.com
    Eric Hodel, Jan 26, 2006
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Spur

    sync on multiple clocks

    Spur, Aug 6, 2003, in forum: VHDL
    Replies:
    2
    Views:
    4,331
    Mike Treseler
    Aug 6, 2003
  2. Replies:
    9
    Views:
    1,030
    Mark Space
    Dec 29, 2007
  3. Steven Woody
    Replies:
    0
    Views:
    404
    Steven Woody
    Jan 9, 2009
  4. Steven Woody
    Replies:
    0
    Views:
    444
    Steven Woody
    Jan 9, 2009
  5. Trans
    Replies:
    2
    Views:
    479
    Trans
    Dec 12, 2005
Loading...

Share This Page