Testing - Network Layer Fault Injection in Ruby

Discussion in 'Ruby' started by James Tucker, Oct 14, 2007.

  1. James Tucker

    James Tucker Guest

    Hi all,

    I've yet to find something that fills this hole in my test cases.

    Does anyone do this, and if so, what tools do they use?

    Thanks,

    James.
     
    James Tucker, Oct 14, 2007
    #1
    1. Advertising

  2. James Tucker

    Eric Hodel Guest

    On Oct 14, 2007, at 13:23 , James Tucker wrote:
    > I've yet to find something that fills this hole in my test cases.
    > Does anyone do this, and if so, what tools do they use?


    I do some of this with custom stubs in memcache-client, see below.
    I've also simulated call failures (Errno::whatever) with slightly
    more powerful stubs that can take a block to raise the exception.

    Here's a fake socket:

    class FakeSocket

    attr_reader :written, :data

    def initialize
    @written = StringIO.new
    @data = StringIO.new
    end

    def write(data)
    @written.write data
    end

    def gets
    @data.gets
    end

    def read(arg)
    @data.read arg
    end

    end

    And here's a test using it:

    def test_flush_all_failure
    socket = FakeSocket.new
    socket.data.write "ERROR\r\n"
    socket.data.rewind
    server = FakeServer.new socket
    def server.host() "localhost"; end
    def server.port() 11211; end

    @cache.servers = []
    @cache.servers << server

    assert_raise MemCache::MemCacheError do
    @cache.flush_all
    end

    assert_equal "flush_all\r\n", socket.written.string
    end


    --
    Poor workers blame their tools. Good workers build better tools. The
    best workers get their tools to do the work for them. -- Syndicate Wars
     
    Eric Hodel, Oct 14, 2007
    #2
    1. Advertising

  3. James Tucker

    James Tucker Guest

    Eric,

    Thanks, that's somewhat the route I had started down for some of the units. I probably shouldn't have omitted from my orignal question, the fact that I'm also using OpenSSL.

    Some of the specific issues I have been dealing with are caused by a harsh environment (trans-atlantic links and poor ADSL connections), and in some cases this causes irrecoverable blocking. I think I've managed to cover all of the blocking cases, but local tests are hard to implement, as you have to fake things like a full TCP connection timeout. (Which requires not sending and timing out packet data which normally the OS doesn't want you to do).

    It's also difficult to find out where the blocks are, as the state renders the app essentially useless, and happens in a different environment than that of the testing / development stage.

    As some of this connection state stuff is actually outside of pure ruby, I was somewhat hoping for an external solution, such as a fault injection proxy socket or similar. Do you know of any?

    Regards,

    James.



    Eric Hodel wrote:
    > On Oct 14, 2007, at 13:23 , James Tucker wrote:
    >> I've yet to find something that fills this hole in my test cases.
    >> Does anyone do this, and if so, what tools do they use?

    >
    > I do some of this with custom stubs in memcache-client, see below. I've
    > also simulated call failures (Errno::whatever) with slightly more
    > powerful stubs that can take a block to raise the exception.
    >
    > Here's a fake socket:
    >
    > class FakeSocket
    >
    > attr_reader :written, :data
    >
    > def initialize
    > @written = StringIO.new
    > @data = StringIO.new
    > end
    >
    > def write(data)
    > @written.write data
    > end
    >
    > def gets
    > @data.gets
    > end
    >
    > def read(arg)
    > @data.read arg
    > end
    >
    > end
    >
    > And here's a test using it:
    >
    > def test_flush_all_failure
    > socket = FakeSocket.new
    > socket.data.write "ERROR\r\n"
    > socket.data.rewind
    > server = FakeServer.new socket
    > def server.host() "localhost"; end
    > def server.port() 11211; end
    >
    > @cache.servers = []
    > @cache.servers << server
    >
    > assert_raise MemCache::MemCacheError do
    > @cache.flush_all
    > end
    >
    > assert_equal "flush_all\r\n", socket.written.string
    > end
    >
    >
    > --
    > Poor workers blame their tools. Good workers build better tools. The
    > best workers get their tools to do the work for them. -- Syndicate Wars
    >
    >
    >
    >
     
    James Tucker, Oct 14, 2007
    #3
  4. James Tucker

    Eric Hodel Guest

    On Oct 14, 2007, at 14:52 , James Tucker wrote:
    > Thanks, that's somewhat the route I had started down for some of
    > the units. I probably shouldn't have omitted from my orignal
    > question, the fact that I'm also using OpenSSL.
    >
    > Some of the specific issues I have been dealing with are caused by
    > a harsh environment (trans-atlantic links and poor ADSL
    > connections), and in some cases this causes irrecoverable blocking.
    > I think I've managed to cover all of the blocking cases, but local
    > tests are hard to implement, as you have to fake things like a full
    > TCP connection timeout. (Which requires not sending and timing out
    > packet data which normally the OS doesn't want you to do).
    >
    > It's also difficult to find out where the blocks are, as the state
    > renders the app essentially useless, and happens in a different
    > environment than that of the testing / development stage.
    >
    > As some of this connection state stuff is actually outside of pure
    > ruby, I was somewhat hoping for an external solution, such as a
    > fault injection proxy socket or similar. Do you know of any?


    I don't I've not needed that level of fault tolerance.

    --
    Poor workers blame their tools. Good workers build better tools. The
    best workers get their tools to do the work for them. -- Syndicate Wars
     
    Eric Hodel, Oct 15, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?YzY3NjIyOA==?=

    do it in database layer or application layer

    =?Utf-8?B?YzY3NjIyOA==?=, Jan 26, 2006, in forum: ASP .Net
    Replies:
    3
    Views:
    458
    Karl Seguin [MVP]
    Jan 27, 2006
  2. et
    Replies:
    2
    Views:
    1,957
  3. dan
    Replies:
    25
    Views:
    1,379
    Uncle Bob (Robert C. Martin)
    Oct 28, 2003
  4. Universe
    Replies:
    1
    Views:
    494
    Universe
    Oct 24, 2003
  5. Dhananjay
    Replies:
    1
    Views:
    1,206
    sloan
    Dec 18, 2006
Loading...

Share This Page