Fault Tolerant DRb?

Discussion in 'Ruby' started by Kirk Haines, Jul 1, 2005.

  1. Kirk Haines

    Kirk Haines Guest

    Just pondering different things this morning, and my mind came back to
    something I've thought about now and again.

    Assume you are using a DRb service for....something. It doesn't matter what.
    The case is the same whether one is accessing an array via DRb or a Rinda
    Ring. Is there some reasonably easy way of making a service work in a fault
    tolerant way? That is, one could have two processes on two different
    machines both offering the same service. If one process dies, the data is
    still present on the other, and the clients of that service can continue
    operating without data loss?


    Kirk Haines
     
    Kirk Haines, Jul 1, 2005
    #1
    1. Advertising

  2. Kirk Haines

    Ara.T.Howard Guest

    On Sat, 2 Jul 2005, Kirk Haines wrote:

    > Just pondering different things this morning, and my mind came back to
    > something I've thought about now and again.
    >
    > Assume you are using a DRb service for....something. It doesn't matter what.
    > The case is the same whether one is accessing an array via DRb or a Rinda
    > Ring. Is there some reasonably easy way of making a service work in a fault
    > tolerant way? That is, one could have two processes on two different
    > machines both offering the same service. If one process dies, the data is
    > still present on the other, and the clients of that service can continue
    > operating without data loss?
    >
    >
    > Kirk Haines


    i've done tons of ha (high availability) setups before for stateful and
    stateless machines. suffice it to say it is almost un-imaginably complex.
    consider:

    * how to you tell if one machine is down vs. the network just being slow?
    for instance on our machines monthly backups might make any machine seem
    dead (can't ping) for 20 minutes or more. typically this is solved via a
    serial cable between nodes to ping on using real-time priorities.

    * if you have the data on both machines and it can EVER be written to
    (modified) how to you bring the data back in sync when a machine has died
    but is now back up?

    these problems are solved - but it's still amazingly hard to get right. check
    out the linux-ha project (google it).

    depending on you needs you may be able to code something simple that 'good
    enough' but you'll need some sort of distributed transaction capability and
    the easist way to get that is via a real rdbms like postgresql. however, once
    you have that setup it's stilly to use drb unless your data is terrible to
    model within the relational model.

    feel free to contact me offline if you want to setup an ha box(es).

    hth.

    -a
    --
    ===============================================================================
    | email :: ara [dot] t [dot] howard [at] noaa [dot] gov
    | phone :: 303.497.6469
    | My religion is very simple. My religion is kindness.
    | --Tenzin Gyatso
    ===============================================================================
     
    Ara.T.Howard, Jul 1, 2005
    #2
    1. Advertising

  3. Hi Kirk,

    I have written something like this a long time back to build fault tolera=
    nt database clusters.
    It became pretty messy pretty quick (of course I was not as proficient in=
    Ruby back then ;-)).=20
    So I have some questions:

    --- Kirk Haines <> wrote:
    > Assume you are using a DRb service for....something. It doesn't matter=

    what. =20
    > The case is the same whether one is accessing an array via DRb or a Rin=

    da=20
    > Ring. Is there some reasonably easy way of making a service work in a =

    fault=20
    > tolerant way? That is, one could have two processes on two different=20
    > machines both offering the same service. If one process dies, the data=

    is=20
    > still present on the other,=20

    ^^^^^^^^^^^^^^^^^^^^^^^^^^
    How do you propose to ensure that? Is it on a shared file system (like NF=
    S).
    If true, then take a look at Ara's rq package:

    http://www.codeforpeople.com/lib/ruby/rq/rq-2.3.0/TUTORIAL

    If false, then think of some "easy" way of replication.

    > and the clients of that service can continue=20
    > operating without data loss?


    I had to worry about how the clients who were in the middle of a request =
    would know that the
    service is no longer available.=20

    >=20
    > Kirk Haines
    >=20


    -- shanko
    >=20




    =09
    ____________________________________________________=20
    Yahoo! Sports=20
    Rekindle the Rivalries. Sign up for Fantasy Football=20
    http://football.fantasysports.yahoo.com
     
    Shashank Date, Jul 1, 2005
    #3
  4. Kirk Haines

    Kirk Haines Guest

    On Friday 01 July 2005 10:40 am, Ara.T.Howard wrote:

    > depending on you needs you may be able to code something simple that 'good
    > enough' but you'll need some sort of distributed transaction capability and
    > the easist way to get that is via a real rdbms like postgresql. however,
    > once you have that setup it's stilly to use drb unless your data is
    > terrible to model within the relational model.


    LOL. All valid points. You never know, though. Sometimes when one asks for
    something magical and unlikely, someone else pipes up and delivers. It was
    worth a shot. Thanks Ara (and Shashank) for the comments.


    Kirk Haines
     
    Kirk Haines, Jul 1, 2005
    #4
  5. Kirk Haines

    Guest

    On Jul 1, 2005, at 12:25 PM, Kirk Haines wrote:
    > Assume you are using a DRb service for....something. It doesn't
    > matter what.
    > The case is the same whether one is accessing an array via DRb or a
    > Rinda
    > Ring. Is there some reasonably easy way of making a service work
    > in a fault
    > tolerant way?


    You might want to take a look at some of the software and ideas at
    http://www.cse.cuhk.edu.hk/~xychen/GroupCS/gcs.htm

    This page has a great summary of toolkits that implement
    "process group communication" or "virtual synchrony". A variety of
    toolkits
    have evolved and been released in various forms. While I don't know
    of any
    ruby implementation or wrapper for these ideas/software it would be a
    great
    project.

    The goal of process group communication is to send a series of
    messages to a
    named group of recipients and ensure that every member of the group
    receives
    the messages in a globally consistent order in the presence of
    communication
    and/or hardware failures. From this foundation you can build a
    variety of
    fault tolerant systems.


    Gary Wright
     
    , Jul 1, 2005
    #5
  6. Kirk Haines

    Ara.T.Howard Guest

    On Sat, 2 Jul 2005 wrote:

    >
    > On Jul 1, 2005, at 12:25 PM, Kirk Haines wrote:
    >> Assume you are using a DRb service for....something. It doesn't matter
    >> what.
    >> The case is the same whether one is accessing an array via DRb or a Rinda
    >> Ring. Is there some reasonably easy way of making a service work in a
    >> fault
    >> tolerant way?

    >
    > You might want to take a look at some of the software and ideas at
    > http://www.cse.cuhk.edu.hk/~xychen/GroupCS/gcs.htm
    >
    > This page has a great summary of toolkits that implement
    > "process group communication" or "virtual synchrony". A variety of toolkits
    > have evolved and been released in various forms. While I don't know of any
    > ruby implementation or wrapper for these ideas/software it would be a great
    > project.
    >
    > The goal of process group communication is to send a series of messages to a
    > named group of recipients and ensure that every member of the group receives
    > the messages in a globally consistent order in the presence of communication
    > and/or hardware failures. From this foundation you can build a variety of
    > fault tolerant systems.
    >
    > Gary Wright


    http://raa.ruby-lang.org/project/rb_spread/

    cheers.

    -a
    --
    ===============================================================================
    | email :: ara [dot] t [dot] howard [at] noaa [dot] gov
    | phone :: 303.497.6469
    | My religion is very simple. My religion is kindness.
    | --Tenzin Gyatso
    ===============================================================================
     
    Ara.T.Howard, Jul 1, 2005
    #6
  7. Kirk Haines

    Ara.T.Howard Guest

    On Fri, 1 Jul 2005, Gary Wright wrote:

    >
    > On Jul 1, 2005, at 3:46 PM, Ara.T.Howard wrote:
    >>>

    >>
    >> http://raa.ruby-lang.org/project/rb_spread/

    >
    > Cool! After I posted my link I found the main Spread
    > site and have been reading about it for the last hour or so.
    >
    > Now I have something to play with!


    i think i may have a patched version of this around... seems like there was a
    little buggette or two in it... let me know if you can't get it working and
    i'll look for it.

    cheers.

    -a
    --
    ===============================================================================
    | email :: ara [dot] t [dot] howard [at] noaa [dot] gov
    | phone :: 303.497.6469
    | My religion is very simple. My religion is kindness.
    | --Tenzin Gyatso
    ===============================================================================
     
    Ara.T.Howard, Jul 1, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tolerant comparator

    , Aug 31, 2005, in forum: VHDL
    Replies:
    4
    Views:
    477
    Andy Peters
    Sep 1, 2005
  2. Skybuck Flying

    Using free space for more fault tolerant file systems.

    Skybuck Flying, Sep 27, 2011, in forum: C Programming
    Replies:
    8
    Views:
    371
    Skybuck Flying
    Sep 28, 2011
  3. Miles Keaton
    Replies:
    3
    Views:
    181
    Miles Keaton
    Mar 30, 2005
  4. Don Stocks

    Fault Tolerant TupleSpace?

    Don Stocks, Jul 10, 2006, in forum: Ruby
    Replies:
    0
    Views:
    142
    Don Stocks
    Jul 10, 2006
  5. Edgardo Hames
    Replies:
    1
    Views:
    127
    ВаÑиÌлий ГригоÌрьевич ЗаÌйце
    Jan 7, 2010
Loading...

Share This Page