Two Advanced Ruby Performance Questions

[Giles Bowkett]

Hey Ezra, this is an awesome e-mail, are you going to blog it? Can I?
I just forwarded it to a client, and I would have sounded much more
intelligent in a phone interview the other day if I'd had this
knowledge at my disposal then. This is the most useful thing I've read
on Rails scaling in a while.

 

[Ezra Zygmuntowicz]

Hey Ezra, this is an awesome e-mail, are you going to blog it? Can I?
I just forwarded it to a client, and I would have sounded much more
intelligent in a phone interview the other day if I'd had this
knowledge at my disposal then. This is the most useful thing I've read
on Rails scaling in a while.

Hey Giles-

Feel free to do what you like with the post. I am writing a lot
about this topic for my rails deployment book for the pragprogs. I
hope to get it out in beta form soon.

Cheers-
-- Ezra Zygmuntowicz
-- Lead Rails Evangelist
-- (e-mail address removed)
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

 

[Giles Bowkett]

Sweet! I think that book will be EXTREMELY popular by the way. It's
the question you hear asked over and over again and you seem to have
some very coherent and battle-tested answers.

I'm working on a site that involves a lot of file uploads, it sounds
like I should look into using Merb rather than Rails for that part of
the system?

 

[Sunny Hirai]

I know you don't intend to use Rails, but there's a
performance-specific blog dealing with Rails that might be of interest
to you:

http://railsexpress.de/blog/

Maybe you can get in touch with those guys and get them to share some
fo their experiences.

Thanks for the recommendation. The website looks to be some more of what
I'm looking for.

Hi Ezra,

Could you comment on Xen instances yielding better performance then
instances of Ruby on a single server. Is this a mild improvement, a
significant one or for redundancy/easy of deployment?

By the way, the ideas in engineyard are fascinating. Offering a hosting
platform with scalability built in. Very Nice.

I'd be willing to pay for an early beta of your book.

Sunny Hirai
CEO, MeZine Inc.

 

[khaines]

Our application servers on our current application run in highly
optimized Coldfusion (sub 100ms page response times) has about a dozen

It depends a LOT on what your are doing in that 100ms, but in general,
once can do a lot in 100ms. Below is an extraction from the logging for a
current IOWA based site. Timings are in the middle column; the first
number is the amount of time taken to receive the request (which, for the
mode of operation of this app, should always be 0.0), and the second is is
the amount of time taken for the application to receive the request,
generate the requested content, and pass it back up to the web server for
delivery to the client. Bytes transmitted in the response is at the end
of the line.

2006-11-29 11:18:22 :: (0.0/0.002295) :: /index.html "200 OK" 14276B
2006-11-29 11:18:22 :: (0.0/0.001249) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:18:25 :: (0.0/0.003757) :: /index.html "200 OK" 14282B
2006-11-29 11:18:25 :: (0.0/0.001513) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:18:27 :: (0.0/0.002254) :: /index.html "200 OK" 14282B
2006-11-29 11:18:28 :: (0.0/0.001244) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:18:32 :: (0.0/0.008826) :: /about.html "200 OK" 13434B
2006-11-29 11:18:32 :: (0.0/0.001643) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:33 :: (0.0/0.016653) :: /funds/xxx_plus_fund.html "200 OK" 18798B
2006-11-29 11:18:34 :: (0.0/0.018764) :: /about/management.html "200 OK" 18405B
2006-11-29 11:18:34 :: (0.0/0.001325) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:37 :: (0.0/0.018813) :: /about/investment.html "200 OK" 14089B
2006-11-29 11:18:37 :: (0.0/0.001263) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:40 :: (0.0/0.003241) :: /about.html "200 OK" 13434B
2006-11-29 11:18:40 :: (0.0/0.001491) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:40 :: (0.0/0.003661) :: / "200 OK" 14276B
2006-11-29 11:18:41 :: (0.0/0.002577) :: /about/management.html "200 OK" 18405B
2006-11-29 11:18:41 :: (0.0/0.001258) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:43 :: (0.0/0.001244) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:44 :: (0.0/0.018290) :: /about/sales.html "200 OK" 16254B
2006-11-29 11:18:45 :: (0.0/0.001241) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:18:47 :: (0.0/0.002186) :: /index.html "200 OK" 14276B
2006-11-29 11:18:47 :: (0.0/0.001641) :: /homestyles.css "304 Not Modified" 0B
2006-11-29 11:19:20 :: (0.0/0.010662) :: /products.html "200 OK" 13570B
2006-11-29 11:19:20 :: (0.0/0.001643) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:19:25 :: (0.0/0.004319) :: /products.html "200 OK" 22589B
2006-11-29 11:19:25 :: (0.0/0.001242) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:19:31 :: (0.0/0.008743) :: /products.html "200 OK" 19099B
2006-11-29 11:19:32 :: (0.0/0.001484) :: /styles.css "304 Not Modified" 0B
2006-11-29 11:19:37 :: (0.0/0.003661) :: /products.html "200 OK" 25243B
2006-11-29 11:19:37 :: (0.0/0.001220) :: /styles.css "304 Not Modified" 0B

Every one of these requests is dynamic. The pages all involve content
pulled from a database, with navigation that is dynamically generated
based on db contents, and the products pages all have tables of fund
statistics queried from the database.

Now, admittedly, the app is caching information where it can, but these
logs demonstrate that it's very reasonable to expect good performance from
Ruby.

These are the logs from a single process, and when hammered, will handle
as many as 200 requests per second, even with very high concurrency. The
web server configuration fails due to high concurrency long before the
application has a problem. The server itself is a very mundane, middle of
the road 32 bit dual processor Athlon box, and is not dedicated to this
one site. It has 60 IOWA based sites/applications on it, currently, in
addition to other duties.

Overall, I'm not really sure what kind of persistence and
non-persistence there is between page requests when Ruby is attached a
web server.

This depends on the architecture of the framework. Generally, for
frameworks that utilize a persistent process type backend, everything that
is time consuming to deal with can persist from request to request.

application. We have a large amount of good code in ColdFusion but as an
agile company, I can see the benefits of Ruby down the line, especially
after a couple of years. Mostly, I love the clean syntax and the overall
design of the language.

Just an aside, but I spent some time converting ColdFusion pages for
displaying mutual fund information to a Ruby application (a MUCH earlier
version of IOWA) a few years ago, and the end result was both faster and
cleaner.



Kirk Haines

 

[Ezra Zygmuntowicz]

Thanks for the recommendation. The website looks to be some more of
what
I'm looking for.

Hi Ezra,

Could you comment on Xen instances yielding better performance then
instances of Ruby on a single server. Is this a mild improvement, a
significant one or for redundancy/easy of deployment?

By the way, the ideas in engineyard are fascinating. Offering a
hosting
platform with scalability built in. Very Nice.

I'd be willing to pay for an early beta of your book.

Sunny Hirai

Hey Sunny-

The whole thing about Xen is involved in the idea of easy
scalability. The behavior of production apps run on a cluster of
mongrels that I have seen has a sweet spot. About 3 or 4 mongrels in
a cluster behind a front webserver like nginx all running in one Xen
instance can serve quite a bit of traffic. Any more then 5 mongrels
in a small cluster like this and you start to see diminishing
returns. And when you add a shared filesystem to the mix like gfs
then you can add and remove nodes from an app cluster at will. Just
by cloning the xen instance and having it join the cluster and mount
the rails applications on a gfs mount. Doing it this way you can have
up to 16 xen instances all sharing a gfs filesystem all serving the
same rails application. So when it is time to deploy new code you
only have to deploy the code to one of these xen nodes and then just
restart the mongrels on all nodes that share the same gfs mount. Then
all nodes are serving the new code. All of these nodes run behind
hardware load balancers so you can bring them down and up in a piggy
back fashion and never have down time to the website users.

We have found that Xen only imposes a performance overhead of 5-9%
compared to non virtualized linux. So going with xen makes it so much
easier to scale across boxes that the tradeoff is well worth it. The
only boxes you may not want Xen on is the database box or boxes. If
you have super heavy database traffic then you may want to go non
virtualized on that box. But saying that, we still run our mysql
clusters on xen instances and have not had performance problems with
it. We use coraid AoE SAN which is block level ATA disk access over
ethernet but without the overhead of tcp/ip. So none of our servers
have hard disk drives. They only have 128Mb flash ata chips for the
main Xen dom0's to boot off of. Then they load all the domU's off of
the SAN.

Rails tends to use a lot of memory but the cpu usage is not so bad.
Our cpu's are sleeping and its always ram that needs to be increased.
We are going to start buying big 4 processor boxes with 32 or 64gigs
of ram and virtualizing on top of those. Currently we use boxes with
two dual-core amd opterons with 8gigs of ram.

Using many smaller rails app server nodes behind load balancing with
all of them sharing state with the database and the shared filesystem
makes applications feel responsive and distribute the load in a nice
way.

Cheers-
-- Ezra Zygmuntowicz
-- Lead Rails Evangelist
-- (e-mail address removed)
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)

 

[Sunny Hirai]

Hi Ezra,

Thanks for the detailed reply.

Sunny Hirai
CEO, MeZine Inc.