Thread safety techniques for server applications?

Discussion in 'Ruby' started by Aaron Smith, Aug 25, 2007.

  1. Aaron Smith

    Aaron Smith Guest

    Hey all,

    I'm looking for some information about handling thread safety with Ruby.
    I've got an application server I wrote that I need to make sure it's
    thread safe. This application server is used over http requests so it's
    possible multiple people hit it at once. I have some questions that will
    help me determine..

    1. Does using mongrel / lighttpd / webrick, ensure thread saftey? (my
    application relies on these)
    2. What kinds of things in the Ruby language should I NOT do that will
    cause thread headaches.. (maybe static variables)?
    3. What techniques can I use to go about testing thread saftey?

    I'm not asking anything in reference to rails. This would be just
    general Ruby thread safety ideas..

    thanks..
    --
    Posted via http://www.ruby-forum.com/.
    Aaron Smith, Aug 25, 2007
    #1
    1. Advertising

  2. Aaron Smith

    Guest

    Hi Aaron,
    I'd like to learn more in this area too, but here are my thoughts:
    The web servers, at least mongrel, are single-threaded. Mongrel queues
    requests and feeds them to the app sequentially. To get concurrency
    you have to run multiple instances of mongrel. In this situation there
    are no thread safety issues because there's only one thread per
    process.
    I like the idea of separate processes instead of worrying about thread
    safety, but sometimes I need multiple threads, for example in a jabber
    client (keepalives, listeners, etc). What I've been doing is keeping
    it as simple as possible and so far I haven't had to think about
    thread conflicts. Or maybe I should be but I haven't ;)
    --Dave

    On Aug 24, 7:05 pm, Aaron Smith <> wrote:
    > Hey all,
    >
    > I'm looking for some information about handling thread safety with Ruby.
    > I've got an application server I wrote that I need to make sure it's
    > thread safe. This application server is used over http requests so it's
    > possible multiple people hit it at once. I have some questions that will
    > help me determine..
    >
    > 1. Does using mongrel / lighttpd / webrick, ensure thread saftey? (my
    > application relies on these)
    > 2. What kinds of things in the Ruby language should I NOT do that will
    > cause thread headaches.. (maybe static variables)?
    > 3. What techniques can I use to go about testing thread saftey?
    >
    > I'm not asking anything in reference to rails. This would be just
    > general Ruby thread safety ideas..
    >
    > thanks..
    > --
    > Posted viahttp://www.ruby-forum.com/.
    , Aug 25, 2007
    #2
    1. Advertising

  3. Aaron Smith

    Corey Jewett Guest

    On Aug 25, 2007, at 12:05 AM, wrote:

    > Hi Aaron,
    > I'd like to learn more in this area too, but here are my thoughts:
    > The web servers, at least mongrel, are single-threaded. Mongrel queues
    > requests and feeds them to the app sequentially. To get concurrency
    > you have to run multiple instances of mongrel. In this situation there
    > are no thread safety issues because there's only one thread per
    > process.


    I don't believe this is true of mongrel itself, but rather of the
    Rails handler in mongrel.

    Corey
    Corey Jewett, Aug 25, 2007
    #3
  4. Aaron Smith

    Guest

    On Sat, 25 Aug 2007, wrote:

    > I'd like to learn more in this area too, but here are my thoughts:
    > The web servers, at least mongrel, are single-threaded. Mongrel queues
    > requests and feeds them to the app sequentially. To get concurrency
    > you have to run multiple instances of mongrel. In this situation there
    > are no thread safety issues because there's only one thread per
    > process.


    This is untrue.

    The standard Mongrel is threaded. It creates a new thread of execution
    for each connection that it receives, and those execute in parallel with
    each other and the main Mongrel thread, which is essentially just an
    accept() loop that receives the requests and spawns handler threads for
    them.

    The Rails mongrel handler has a mutex that locks the action within it to a
    single thread of execution at a time. So, if 10 requests come in at the
    same time, Mongrel will create 10 threads of execution for those 10
    requests, but when execution flow reaches the Rails handler, each thread
    will stand in line at the mutex gate and proceed through it in single
    file.

    In a standard Mongrel handler, which does not have a mutex at the front of
    it, the requests are processed concurrently. This is the normal
    situation.

    Remember that Ruby threads, being green threads, are all in the same
    process, so there is no actual concurrency of execution between them. In
    most cases these threads will not increase your throughput.


    Kirk Haines
    , Aug 25, 2007
    #4
  5. Aaron Smith

    Guest

    On Sat, 25 Aug 2007, Aaron Smith wrote:

    > I'm looking for some information about handling thread safety with Ruby.
    > I've got an application server I wrote that I need to make sure it's
    > thread safe. This application server is used over http requests so it's
    > possible multiple people hit it at once. I have some questions that will
    > help me determine..


    In general, it's the same as any other type of threaded programming.
    Share as little as possible, and control access to shared resources so
    that two threads aren't changing state in it at the same time and running
    into eachother. Look at the Mutex class and the Queue class as starting
    points for tools to help you do this.

    > 1. Does using mongrel / lighttpd / webrick, ensure thread saftey? (my
    > application relies on these)


    lighttpd is an external web server, so it's irrelevant.

    Both mongrel and webrick are threaded Ruby web server platforms. They,
    however, don't do anything to ensure that your code which you run inside
    of them is threadsafe.

    > 2. What kinds of things in the Ruby language should I NOT do that will
    > cause thread headaches.. (maybe static variables)?


    The only things to really keep in mind is that Ruby threads are green
    threads -- they are all done inside of the Ruby interpreter. So, they all
    share a single process. Thus, the use of threads will rarely increase the
    throughput of your program, unless there is some external latency that can
    be captured, and that external latency does not occur inside of a Ruby
    extension.

    This is because while the flow of execution is inside of an extension, it
    is out of Ruby's control, and no thread task switching will take place.

    Also, be aware that Ruby uses a select() loop to manage its threads of
    execution, and it has an fd_setsize limit of 1024 handles, so there is a
    sharp upper boundary on the number of threads you can have in a Ruby
    process.

    > 3. What techniques can I use to go about testing thread saftey?


    Look for areas in your code where you share resources between your
    threads. Do you take precautions to keep multiple threads from stepping
    on eachother when using those resources?

    Write test code that creates multiple threads, and tries to stress those
    areas.


    Kirk Haines
    , Aug 25, 2007
    #5
  6. Aaron Smith

    Roger Pack Guest

    > I'm looking for some information about handling thread safety with Ruby.

    In general do not share variables between threads (or at least
    synchronize when you access them), do not interrupt threads by injecting
    interrupts into them (well, you can but...it's dangerous),
    add

    Thread.abort_on_exception = true # if a thread dies, tell me :)

    to your code so that you can debug when exception are thrown but not
    caught...

    Also load up all your functions/everything 'single threaded' otherwise
    some of the functions will be 'assigned' to the wrong thread (ugh), then
    be unavailable (classes, modules, too).
    I.e. dynamically declared functions are dangerous.

    Note also that sometimes if you are reading from TCPSockets the sockets
    get confused and start reading from one another. To avoid this (I
    think) use Francis Cianfrocca's EventMachine.

    That being said it is still fun to program multithreaded stuff in Ruby,
    despite the growing pains and fact that Ruby isn't quite *there* yet.

    -Roger

    Aaron Smith wrote:
    > Hey all,
    >
    > I've got an application server I wrote that I need to make sure it's
    > thread safe. This application server is used over http requests so it's
    > possible multiple people hit it at once. I have some questions that will
    > help me determine..
    >
    > 1. Does using mongrel / lighttpd / webrick, ensure thread saftey? (my
    > application relies on these)
    > 2. What kinds of things in the Ruby language should I NOT do that will
    > cause thread headaches.. (maybe static variables)?
    > 3. What techniques can I use to go about testing thread saftey?
    >
    > I'm not asking anything in reference to rails. This would be just
    > general Ruby thread safety ideas..
    >
    > thanks..


    --
    Posted via http://www.ruby-forum.com/.
    Roger Pack, Aug 28, 2007
    #6
  7. Roger Pack wrote:
    ...
    > Also load up all your functions/everything 'single threaded' otherwise
    > some of the functions will be 'assigned' to the wrong thread (ugh), then
    > be unavailable (classes, modules, too).
    > I.e. dynamically declared functions are dangerous.


    In ruby, everything is dynamically defined. The closest you can get to
    static definitions is to require all your lib files before starting
    threads. Even so, it's not really static. It might be safer in some
    cases because require-ing a file is not atomic.(*) That's a corner case,
    though.

    I don't think it's possible for things to be unavailable because they
    were loaded in the wrong thread, though. Got an example?

    > Note also that sometimes if you are reading from TCPSockets the sockets
    > get confused and start reading from one another.


    Really? I've never seen that, even with pretty heavy use of lots of
    threads and sockets.

    ----

    (*) An example:

    [~/tmp] cat a.rb

    t = Thread.new do
    loop do
    sleep 0.1
    puts "a"
    end
    end

    sleep 0.5

    require 'b'

    [~/tmp] cat b.rb
    t = Thread.new do
    loop do
    sleep 0.1
    puts " b"
    end
    end

    sleep 1
    t.kill

    [~/tmp] ruby a.rb
    a
    a
    a
    a
    a
    b
    a
    b
    a
    b
    a
    b
    a
    b
    a
    b
    a
    b
    a
    b
    a
    b
    a


    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
    Joel VanderWerf, Aug 28, 2007
    #7
  8. Aaron Smith

    Roger Pack Guest

    Thanks for your comments.
    Forgive me if I called modules/classes statically loaded. What I meant
    was 'loaded before you start splitting to multiple threads' instead of
    static.

    For example I have used xmlrpc before (which relies on REXML) and every
    so often (note the 'freak chance' aspect) it will throw the exception
    "REXML::Document not found" despite the fact that it indeed should be,
    and normally is. I laid the blame on Ruby threads. Where it belongs, I
    think is on...Ruby threads. But having not actually ever fixed it, I
    can't say for sure. I believe I've been able to recreate it reliably.



    Joel VanderWerf wrote:
    > Roger Pack wrote:
    > ...
    >> Also load up all your functions/everything 'single threaded' otherwise
    >> some of the functions will be 'assigned' to the wrong thread (ugh), then
    >> be unavailable (classes, modules, too).
    >> I.e. dynamically declared functions are dangerous.

    >
    > In ruby, everything is dynamically defined. The closest you can get to
    > static definitions is to require all your lib files before starting
    > threads. Even so, it's not really static. It might be safer in some
    > cases because require-ing a file is not atomic.(*) That's a corner case,
    > though.
    >
    > I don't think it's possible for things to be unavailable because they
    > were loaded in the wrong thread, though. Got an example?
    >
    >> Note also that sometimes if you are reading from TCPSockets the sockets
    >> get confused and start reading from one another.

    >
    > Really? I've never seen that, even with pretty heavy use of lots of
    > threads and sockets.


    Yeah I get it...sometimes a socket that is *only* used for sending will
    magically 'receive'...its own output! Wow! And other weirdness. Mostly
    on slower machines. It is odd. I noticed Zed Shaw said he'd run into
    the same thing (and was unable to track down the cause) in some thread
    or other here. I honestly don't get it, either, but I think it's half
    the motivation to the creation of EventMachine.

    Just my own $0.02
    -Roger

    >
    > ----
    >
    > (*) An example:
    >
    > [~/tmp] cat a.rb
    >
    > t = Thread.new do
    > loop do
    > sleep 0.1
    > puts "a"
    > end
    > end
    >
    > sleep 0.5
    >
    > require 'b'
    >
    > [~/tmp] cat b.rb
    > t = Thread.new do
    > loop do
    > sleep 0.1
    > puts " b"
    > end
    > end
    >
    > sleep 1
    > t.kill
    >
    > [~/tmp] ruby a.rb
    > a
    > a
    > a
    > a
    > a
    > b
    > a
    > b
    > a
    > b
    > a
    > b
    > a
    > b
    > a
    > b
    > a
    > b
    > a
    > b
    > a
    > b
    > a


    --
    Posted via http://www.ruby-forum.com/.
    Roger Pack, Aug 28, 2007
    #8
  9. Aaron Smith

    Roger Pack Guest

    Roger Pack wrote:
    >> I'm looking for some information about handling thread safety with Ruby.

    >
    > Note also that sometimes if you are reading from TCPSockets the sockets
    > get confused and start reading from one another. To avoid this (I
    > think) use Francis Cianfrocca's EventMachine.
    >


    Possible ways to fix this might (might) be to ensure that every socket
    read/write is 'not at the same time as any other read/write' (i.e.
    surrounded by a mutex lock), or to perhaps write a drop in replacement
    for the TCPSocket class that just uses EventMachine in the background
    for I/O and queues the input/output.

    I still haven't ever found a fix for the problem of defining methods in
    one thread and the methods are assigned to a different thread. I think
    I may just report this one to ruby and forget about it.

    Good luck all.
    -Roger
    --
    Posted via http://www.ruby-forum.com/.
    Roger Pack, Oct 8, 2007
    #9
  10. Aaron Smith

    Roger Pack Guest

    Wow EventMachine works like a dream. Thank you!

    >
    > In general, EventMachine encourages a style that doesn't use threads at
    > all.
    > The I/O queueing you're describing is already done by EM itself. All you
    > have to do is write the handlers and EM will call them itself as the I/O
    > comes in.
    >
    > It may seem impossible to write network-aware programs without threads.
    > Not
    > only is it possible but it can have very real benefits.


    --
    Posted via http://www.ruby-forum.com/.
    Roger Pack, Oct 9, 2007
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hans

    What is thread safety?

    Hans, Oct 11, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    570
    Sahil Malik
    Oct 12, 2004
  2. George Ter-Saakov

    LiteralControl thread safety.

    George Ter-Saakov, Apr 5, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    318
    Martin Dechev
    Apr 6, 2004
  3. Luis E Valencia

    Techniques for big applications?

    Luis E Valencia, May 17, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    340
    Ashish M Bhonkiya
    May 18, 2004
  4. Simon Harvey

    A thread safety question

    Simon Harvey, Aug 6, 2004, in forum: ASP .Net
    Replies:
    3
    Views:
    391
    Alvin Bruney [MVP]
    Aug 6, 2004
  5. Chris Thomasson
    Replies:
    0
    Views:
    420
    Chris Thomasson
    Mar 3, 2007
Loading...

Share This Page