Camping or Merb

Discussion in 'Ruby' started by Eduardo Tongson, Sep 11, 2007.

  1. SGkgZm9sa3MsCgpXZSB3aWxsIGJlIGNyZWF0aW5nIGEgaGVhdnkgdHJhZmZpYyBzaXRlIHRoYXQg
    dXNlcyBBamF4LCBGbGFzaCB2aWRlbwphbmQgaGFzIGhlYXZ5IHBlcnNpc3RlbnQgZGF0YWJhc2Ug
    Y29ubmVjdGl2aXR5LiBEYXRhYmFzZSB3aWxsIGJlIG1vc3QKcHJvYmFibHkgUG9zdGdyZVNRTC4K
    ClVzYWdlIHJlcXVpcmVzIHNwZWVkIGNsb3NlIHRvIHJlYWwgdGltZSwgYmFuZHdpZHRoIG9yIGxh
    dGVuY3kgd2lsbCBub3QKYmUgYW4gaXNzdWUuIEknbSBwcmV0dHkgc3VyZSB3ZSBjYW4gYWNjb21w
    bGlzaCBhbGwgb2YgdGhpcyBpZiB3ZSBzdGljawp0byBwdXJlIFJ1YnkuIEJ1dCB3ZSB3b3VsZCBs
    aWtlIHRvIGNyZWF0ZSBpdCB1c2luZyBhIHdlYiBmcmFtZXdvcmsgYW5kCnRha2UgYWR2YW50YWdl
    IG9mIFN3aWZ0aXBseSwgRXJ1YmlzLCBNb25ncmVsIGV0Yy4gLgoKTXkgcmVzZWFyY2ggc2hvd3Mg
    dGhhdCB3ZSB3aWxsIGJlIG5lZWRpbmcgU3dpZnRpcGxpZWQgTW9uZ3JlbCBhbmQKc2hvdWxkIGF2
    b2lkIEFjdGl2ZXJlY29yZCAodXNlIE9nPykuIFRoZSBjbG9zZXN0IGZyYW1ld29ya3MgaW4gbWlu
    ZAphcmUgQ2FtcGluZyBhbmQgTWVyYi4gV2hpY2ggb2YgdGhlc2UgdHdvIHdvdWxkIGJlIHRoZSBi
    ZXN0IG9wdGlvbiBmb3IKdXM/CgogICAgICAgIHDHnQo=
     
    Eduardo Tongson, Sep 11, 2007
    #1
    1. Advertisements

  2. My research shows that we will be needing Swiftiplied Mongrel and
    Have a look at Sequel, a lightweight ORM library for Ruby:

    http://code.google.com/p/ruby-sequel/

    Sharon
     
    Sharon Rosner, Sep 11, 2007
    #2
    1. Advertisements

  3. Camping is very hard to modify, so merb is probably better choice,
    swiftiply is nice but it's new solution and hard to tell how well it
    works yet.

    anyway i would recommend nginx as load balancer and for serving static
    files and flv's, evented mongrel for backend, merb for backend, erubis
    for templates, and sequel fro postgresql database - supports connection
    pooling and persistent connections.

    i tested my own micro-framework based on evented mongrel handlers and it
    was working 20 (yes 20) times faster then standard rails setup.

    if you are interested in high availbility and perfomance tunning please
    contact me.
     
    Marcin Raczkowski, Sep 11, 2007
    #3
  4. Eduardo Tongson

    khaines Guest

    I'd suggest looking at Merb, or maybe Ramaze, or my framework, IOWA.
    IOWA is definitely the fastest of the bunch. Merb is #2 in speed, is
    flexible, and has an active community. Ramaze is a newcomer. It's kind
    of to Nitro what Merb is to Rails. It's slower than Merb, but is a very
    nice framework with some good ideas.

    For ORM layers, you should do some benchmarks. AR is not particularly
    fast. Og...might be faster. Maybe not. I'm not really sure. Also check
    Sequel and you might try my ORM, Kansas (http://withruby.com/kansas for a
    more recent snapshot). It's fairly lightweight and fast, and quite
    capable, and has been in active use on production sites for years.

    I originally wrote Swiftiply to use with IOWA, so it does work very well
    with it. Regardless of what you choose to do, though, feel free to
    contact me if you have any questions or need any help.


    Kirk Haines
     
    khaines, Sep 11, 2007
    #4
  5. Eduardo Tongson

    Roger Pack Guest

    so you prefer nginx over swiftiply? Have you tried your mockup with
    swiftiply? Just wondering.
    -Roger
     
    Roger Pack, Sep 14, 2007
    #5
  6. Eduardo Tongson

    khaines Guest

    Me?

    I wrote Swiftiply, and yes, I use it in lieu of nginx or Apache on several
    sites, now.


    Kirk Haines
     
    khaines, Sep 15, 2007
    #6
  7. Eduardo Tongson

    khaines Guest

    Swiftiply has SSL on the horizon. EventMachine will handle SSL
    connections, so that is not a problem. There are just a few config
    management issues in the way of full fledged support. I have it on my
    roadmap for the 0.7.0 release.


    Kirk Haines
     
    khaines, Sep 15, 2007
    #7
  8. Eduardo Tongson

    khaines Guest

    Apologies for the back to back posts. On static file handling, you are
    right. nginx is about as fast as it gets. But that's fine. Swiftiply's
    static file handling doesn't have to be that fast to be fast enough. My
    target range is 8000 to 9000 small static files per second, and at least
    100 megabytes per second of data transfer for large static files, on my
    test machine. That is the area the threshold that I am working to
    continue to meet or beat as I work on Swiftiply. It's not nginx speeds,
    but it's better than Apache speeds (out of a single process, too), and
    it's fast enough to saturate a gigabit port if one is doing big files, and
    a 100mbs port if one is doing small files, so it's fast enough for the
    vast majority of deployments.


    Kirk Haines
     
    khaines, Sep 15, 2007
    #8
  9. How do you use Swiftcore IOWA? Is it better than using IOWA with HA
    Proxy with evented mongrel?

    BTW http://iowa.swiftcore.org is not responding as of Sat Sep 15
    02:16:36 UTC 2007.
     
    Eduardo Tongson, Sep 15, 2007
    #9
  10. I'll try to reply to few previous post here.

    I didn't test swiftyply very much, and i didn't do benchmarks comparing
    to nginx so i have no idea how speed comparsion looks.

    But i DID test nginx vs. apache vs. apache2 vs. light and nginx always
    wins :)

    now mongrel vs. evented mongrel - evented mongrel wins BIG

    clasic mongrel - forking mongrel (my own modification that makes mongrel
    act like apache - fork on request) - a little bit slower then classic,
    but for low traffic sites - much better choice then cluster - you only
    need one mongrel instance in your memory

    mongrel vs. nginx on static files - nginx wins - but it's power is only
    needed when working with lots of small files. i also have moded mongrel
    that is designed to handle static pages - it allows caching most used
    files in memory - like css styles and main page, i'm planing to improve
    it with sendfile syscall for speed on files that are not in cache and
    mmap on things that are in cache.
    BUT all this improving is only for my fun and pleasure becouse you are
    more likely to run out of bandwidth before speed will be an issue

    now frameworks - i didn't test all and mostly I'm basing this list on my
    old benchmarks, what khaines said, and other things found on network
    Order from slowest to fastest

    - rails
    - nitro
    - ramazee
    - merb - almost 15 times faster then rails in my tests
    - iowa

    somewhere between merb and iowa probably would be my micro-framework,
    still unreleased - but i'm geting into it and it'll be availble this
    week i think for alfa

    now ORM
    - AR
    - Og
    - Sequel / Kansas - no idea which one is faster i didn't test them

    sequel is 2-3 times faster then AR on small and frequent queries, but
    have nice features like connection pooling for postgres which gives it
    even more advantege.

    Okey that would be all in terms of speed, but remember that usually
    lower level you work on - more time you spent and more bugs you make, we
    were considering using somethning other then rails in our projects and
    we found out that if project doesn't REALLY need speed, costs of
    adopting new technology, training, testing etc. exceed what we would
    save on hardware.

    cheers
     
    Marcin Raczkowski, Sep 15, 2007
    #10
  11. Eduardo Tongson

    Roger Pack Guest


    I guess the concern would be that using swiftiply to handle static
    content might be a little slower than nginx. Any thoughts?

    If this were a speed problem, a possible fix might be to add to
    EventMachine a call along the lines of 'here's a file's path, I want you
    to pipe that file to this socket' or something. Then maybe swiftiply
    could get about as fast as C based. If it were slower. Just some
    random thoughts. Thanks Kirk.

    -Roger
     
    Roger Pack, Sep 17, 2007
    #11
  12. Eduardo Tongson

    khaines Guest

    A little slower than nginx can still be very fast.

    My design goals are to be in the 8k-9k files per second range on my
    development hardware, for small files, and preferably > 120 megabytes per
    second for large files.

    It is already a somewhat optimized path that lets it be that fast. Part
    of what I have been working on over the last month (ugh), along with HTTP
    caching support, are some changes to further streamline and optimize that
    path. However, after a point, additional speed is mostly irrelevant
    because one has already exceeded the bandwidth of one's port. If you have
    a 100mbs port, you're going to stream large files at a small percentage of
    120 megabytes per second, anyway, and even the typical small image files
    that are common on web sites are going to saturate your bandwidth at a
    speed less than 8k/second, so the fact that nginx can do 16k/second isn't
    so meaningful, most of the time.

    That said....I _think_ small files handling may bump up into the 11k to
    12k/second range on my hardware. Large file streaming will remain about
    the same. I am tinkering with it as I type this, in fact.

    I'm also tinkering with a microframework, tentatively labeled Wisteria
    (might be something up at wisteria.swiftcore.org later this week), based
    off a lot of this work. It's aimed at things which need to be done very
    fast, or which require a bunch of long running connections (file uploads
    and downloads).

    At its simplest, an earlier version of it that I was playing with last
    week would do 16k "Hello World" type requests per second. I think that if
    this pans out (it's rather directly derived from other Swiftiply work),
    it'll probably end up doing around 11 thousand to 12 thousand simple
    requests per second, and should be able to saturate a gigabit port, for
    upload or download handling, on one or maybe two processes.


    Kirk Haines
     
    khaines, Sep 17, 2007
    #12
  13. Eduardo Tongson

    Roger Pack Guest

    A little slower than nginx can still be very fast.
    Wow that would be fast: 120MB/s~1Gb/s.
    Let us know if it works out and maybe swiftiply really will become the
    default for Rails serving :)
    Cheers.
    -Roger
     
    Roger Pack, Sep 17, 2007
    #13
  14. I strongly recommend *not* using swiftiply. It's basically unmaintained.
    I posted a reproducible bug a long time ago, and it was ignored.

    There's little point going with Merb, because that's basically
    unmaintained now as well - it's been merged into Rails 3.

    Camping is an example of a "micro framework". It uses too much
    metaprogramming magic for my taste. If you want a micro framework, I'd
    suggest Sinatra instead.

    Rails is obviously an example of a big framework ("macro framework"?).
    Whether you use that will depend on how much you want to make use of the
    structuring and helper methods it provides. It's a love-or-hate thing,
    so if you love it, use it. I would not worry about performance when
    using either of these: they both sit on top of Rack, and if you have
    specific methods which require the lowest possible latency, you can
    write them as Rack methods (under Rails they call this "Rails Metal")

    Mongrel works fine but there are newer derivatives you should look at -
    thin, unicorn, rainbows!

    With any of those you'll probably want an apache or nginx proxy sitting
    in front. Alternatively look at Phusion Passenger which runs directly
    within apache or nginx.

    HTH,

    Brian.
     
    Brian Candler, Oct 9, 2010
    #14
  15. Eduardo Tongson

    Mark T Guest

    Hi Brian,

    btw.. What was the Swiftiply bug?

    MarkT
     
    Mark T, Oct 14, 2010
    #15
  16. The most severe problem I had with swiftiply was when I mounted an app
    on a subdirectory with :prefix. If a request came outside that prefix
    then it crashed immediately. Details at
    http://rubyforge.org/tracker/index.php?func=detail&aid=14949&group_id=3637&atid=14008

    I also submitted two minor patches:
    http://rubyforge.org/tracker/index.php?group_id=3637&atid=14010
    The pidfile fix certainly hasn't been applied in the current git code;
    the other patch looks like it's no longer relevant.

    I actually quite like the concept of swiftiply, where you can have a
    dynamic pool of workers which can "pull" HTTP requests from a
    multiplexor, but it's a non-standard protocol and IMO ties you too
    heavily to this one implementation. If you use regular HTTP proxying
    then at least it's HTTP, and you have a whole range of both proxies and
    HTTP servers to choose from. If you want a dynamic worker pool you have
    at least Passenger and FastCGI to choose from.
     
    Brian Candler, Oct 14, 2010
    #16
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.