Joel Spolsky on languages for web programming

[khaines]

5. Database "bottlenecks": Most of the "industrial strength" databases
-- Oracle, PostgreSQL, MySQL, SQL Server 2005, DB2, etc. -- are
co-optimized with the operating systems and the platform hardware.
They've done their homework; they've done their performance engineering,
profiling and micro-benchmarking.

As good as they are, a poorly engineered database schema can make the
RDBMS work much harder than it needs to, requiring more hardware than is
necessary.

It is very easy, though, for even simple queries on a simple, optimized
DB schema on a well tuned engine to be the single largest bottleneck for a
dynamic web site or web based application.

Take, for example, a site that I am working on today.

It is dynamically generated using data from a simple db, but it is ok for
there to be a short latency between changes to db data and changes
appearing on the site pages.

Querying everything from the db for every request was netting around 35
pages per second -- a little under 3 hundredths of a second per page
generation. Changing the code so that it queries no more than twice a
minute, operating off of cached data between queries, dropped the page
generation to around 5 or 6 thousandths of a second -- about 170ish a
second. For comparison, on my development server I get about 620 page
loads a second from a static version of the content.

The database presents a significant bottleneck that I can fortunately work
around through a little bit of caching. If, for whatever reason, I could
not employ this caching, that db bottleneck could be a much more
significant issue than any Ruby speed issues.


Kirk Haines


BTW, just for comparison, a version of this content rendered and delivered
through CakePHP on PHP4, with no db data caching, renders about 17 pages
per second.

 

[Squeamizh]

I too found that beyond ironic.

James Edward Gray II

Do you know what ironic means?

 

[James Edward Gray II]

Do you know what ironic means?

ironic |=C4=AB=CB=88r=C3=A4nik| |a=C9=AA=CB=8Cr=C9=91n=C9=AAk| =
|=CA=8C=C9=AA=CB=8Cr=C9=92n=C9=AAk|
adjective
using or characterized by irony : his mouth curved into an ironic smile.
=E2=80=A2 happening in the opposite way to what is expected, and =
typically =20
causing wry amusement because of this : [with clause ] it was ironic =20
that now that everybody had plenty of money for food, they couldn't =20
obtain it because everything was rationed.

-=3D-=3D-=3D-=3D-=3D-

My usages of the term seems OK to me.

Joel bashed Ruby for being unproven, then went on to say that they =20
were using a custom language they invented. By his own definition, =20
that is unproven. That's "happening in the opposite way to what is =20
expected" because of his earlier statements and it also causes me =20
"wry amusement."

Did this attack on my grammar serve some purpose?

James Edward Gray II

 

[Joel VanderWerf]

Do you know what ironic means?

After gem install MerriamWebster, ri irony outputs:

3 a (1) : incongruity between the actual result of a sequence of events
and the normal or expected result (2) : an event or result marked by
such incongruity b : incongruity between a situation developed in a
drama and the accompanying words or actions that is understood by the
audience but not by the characters in the play -- called also dramatic
irony, tragic irony

In this case, "tragic irony" seems apt.

...Just another Joel

 

[Bil Kleb]

After gem install MerriamWebster, ri irony outputs:

Dang cool. I missed that one. However, I only get:

Attempting remote installation of 'MerriamWebster'
ERROR: While executing gem ... (NoMethodError)
undefined method `name' for -517611318:Fixnum

on ruby 1.8.2 (2004-12-25) [powerpc-darwin8.2.0]

Later,

 

[Chad Perrin]

After gem install MerriamWebster, ri irony outputs:

[ Considering the responses by both you and James, I guess the real
question here is "Does Squeamizh know what 'ironic' means?"

 

[Eero Saynatkari]

After gem install MerriamWebster, ri irony outputs:

[ Considering the responses by both you and James, I guess the real
question here is "Does Squeamizh know what 'ironic' means?"

Someone needs more George Carlin. In any case, I
would not necessarily call this 'ironic', rather
'hypocritical' or 'oxymoronic'.

It would approach irony if Joel's big software
project would fail because Wasabi was unstable
and broke the whole thing

 

[Chad Perrin]

Do you know what ironic means?


After gem install MerriamWebster, ri irony outputs:

[ Considering the responses by both you and James, I guess the real
question here is "Does Squeamizh know what 'ironic' means?"

Someone needs more George Carlin. In any case, I
would not necessarily call this 'ironic', rather
'hypocritical' or 'oxymoronic'.

Joel was not being ironic -- rather, the circumstance of Joel's
commentary was itself ironic.

It would approach irony if Joel's big software
project would fail because Wasabi was unstable
and broke the whole thing

That, too, would be ironic -- even more so.

 

[James Edward Gray II]

After gem install MerriamWebster, ri irony outputs:

Dang cool. I missed that one. However, I only get:

Attempting remote installation of 'MerriamWebster'
ERROR: While executing gem ... (NoMethodError)
undefined method `name' for -517611318:Fixnum

on ruby 1.8.2 (2004-12-25) [powerpc-darwin8.2.0]

This is going to sound funny, but try it again. A couple of times if
you have to.

I've seen that error before and it seems to come and go. Certain
things seem to stress it more. I often saw it when building the
RDocs for gems, for example.

Interestingly, I have not seem it in some time now. I'm guessing my
switch to an Intel Mac or upgrade to 1.8.4 resolved the issue.

Hope that helps.

James Edward Gray II

 

[vidar]

In message "Re: Joel Spolsky on languages for web programming"

(2) Although Ruby runs slower than other languages in those
micro-benchmarks, real bottleneck in the most application lies in
either network connection or databases. So we don't have to buy 5
times as many boxes.

Agreed. I manage engineering at Edgeio (http://www.edgeio.com/). We
don't use Rails, but we do use Ruby extensively on our _backend_. All
of our backend was initially written in C++, but I decided to start
migrating it to Ruby because:

1) It massively cut complexity. As an example I replaced a very
primitive, simple messaging middleware system with 700 lines of Ruby
that was significantly more advanced. Adding the same features to the
C++ version would easily have brought it to 3000-4000 lines of code.
More importantly, it took less time to write the Ruby version than to
write the much more primitive C++ code. And that system was my first
ever Ruby app, while I've been doing C/C++ for more than 10 years.

2) We hardly ever max out CPU's. We _can't_ buy boxes with slow enough
CPU's to make it become an issue for most of our servers. The messaging
middleware app handles millions of messages per day and rarely takes up
10% of a single CPU on the servers it runs on.

3) Most of the CPU time used by our Ruby apps is spend waiting for IO.
For the messaging middleware server, a conservative estimate is that
about 80% of the time is spent _in the kernel_ in read() or write()
syscalls. So even assuming that we could get a 10 times speedup of the
code we have written from moving the code back to C/C++, that would
only translate to a reduction in CPU use from 10% to 8.2% (reducing
time spent in userspace from 2% to 0.2%). Heck, even a 100 times
speedup of the userspace code would still only reduce the CPU use from
10% to 8.02%...

4) Servers are cheap. Engineers are not. A relatively senior engineer
easily costs USD 8k-12k a month (salary+social costs). A single
man-month of wasted effort could buy me a lot of extra processing
power.

5) When we scale, the limitations we run into are from most to least
important: disk io capacity, memory, network latency, cpu. Optimizing
algorithms to minimize disk writes, minimize memory usage (granted,
this is an area were we have to be careful with Ruby) and minimize
latency are all more important than reducing cpu usage. As a result,
because we'll generally have to buy/lease more servers based on io
capacity and memory, we have a lot of wasted CPU capacity. For _most_
(not all) of our backend code, spending time speeding up the code would
not save us anything.

6) IF/when we have to rewrite things in C/C++ to speed it up at any
time, it'll still be far more effective to translate small, isolated
parts of working, well tested Ruby code directly and create extensions
of them, than to write the whole apps as C/C++.

Vidar

 

[Alder Green]

http://www.joelonsoftware.com/items/2006/09/01.html

Actually, despite the fact that I love Ruby a lot, I'm inclined to
partially agree with him on this. Presently, our company does have
some Rails-based web applications deployed but they're predominantly
applications geared for use by only a few people (internal client use
only); we've not yet tried to deploy a real public-facing web
application based on Rails. For that, it works really well.

If anything, your hitherto positive expeirence should cast an
optimistic outlook for Rails' performance in a wider-audience
application.

high load applications; my own experiences attempting to optimize
plain Ruby code for performance have been simultaneously frustrating
and rewarding. I doubt I could do the same with a Rails app.

Rails performacne optimization is very different from Ruby
optimization. There's only so much you can to optimize Ruby for simple
operations. Web applications, however, are usually complex. Solving or
even merely identifying bottlenecks involves making and testing broad
changes rapidly. And that's where Ruby/Rails really shines.

There are many significant optimizations I was able to discover and
implement in Rails, that I simply wouldn't be able to do in PHP
because of its less modular and malleable nature. In fact, I think
VHLLs like Ruby (in conjuction with appropriate frameworks like Rails)
have a solid advantage there over lower level languages, and this edge
will only increase in the future.

So for
now we're gonna stick with PHP for our public facing web applications,
even if it is even worse for i18n/l10n/m17n applications than Ruby
is...

The first steps I took in Rails were rewriting existing PHP
applications with it. So I have a certain estimate about how the Rails
compares to PHP for very similar applications. Keep in mind those
initial translations were completely naive; I didn't know Rails at the
time, and I didn't do anything like the optimizations mentioned above.

Generally, those naive applications weren't significantly slower than
their PHP equivalents (which themselves were old, tested, optimized
code). At most, the PHP was twice as fast (would handle twice the
requests over the same time period). And that's a maximum estimate -
generally, any difference was hardly noticeable.

In my experience a lot of production PHP applications can be optimized
to at least 2-3 times of their existing performance (and that's
assuming there wasn't really sloppy programming going on, in which
case the gain might easily be many times more). So I don't see why
Rails is any less viable than PHP in that regard. In fact, quite the
contrary: Rails applications I worked on frequently ended up getting
some refactoring after going production, and becoming 2-3 times more
efficient. Refactoring a working PHP application, however, is commonly
such a PITA, I was happy to close the hood after the engine was
running and not bother opening it again unless in the unfortunate
event of a bug.

Regards,
-Alder

 

[Alvin Ryder]

http://www.joelonsoftware.com/items/2006/09/01.html

Actually, despite the fact that I love Ruby a lot, I'm inclined to
partially agree with him on this. Presently, our company does have
some Rails-based web applications deployed but they're predominantly
applications geared for use by only a few people (internal client use
only); we've not yet tried to deploy a real public-facing web
application based on Rails. For that, it works really well. We're
taking a wait and see attitude before we attempt to use Rails for any
high load applications; my own experiences attempting to optimize
plain Ruby code for performance have been simultaneously frustrating
and rewarding. I doubt I could do the same with a Rails app. So for
now we're gonna stick with PHP for our public facing web applications,
even if it is even worse for i18n/l10n/m17n applications than Ruby
is...

Interesting post Dido ;-)

I have never seen one web project fail due to slow language execution,
yet I have seen many fail because developers boggled down with
complexity, bloat and sluggish time to market!

After a couple of week's worth of work in Perl, people would say "wow
that's totally awesome" but with Java they say "hmmm, is that all"?

Java and C# are no guarantee for success. Sure the production
environments are solid but you cannot ignore the development economics,
you seem to need 10 instead of 3 people and 5 times as long. These are
serious issues!

Ok, sure Java's OO may be nicer than Perl 5's but once you brew
HTML/Javascript/JSP/JSTL/EL/tags/JSF or Struts together the result
isn't exactly what I'd call pretty. Java is in no way a safe bet.

How about C#, well it runs in Windows and without serious and expensive
firewalls you just can't go anywhere near the Internet. Yes .net is
prettier than the java's web stack but there is still way too much to
learn and when the framework doesn't do what you need you're left in
the cold big time. Again .NET is no guarantee to success either.

Ruby and Rails just get straight to the point. They make common things
easy and elegant. If execution speed really is a problem then I reckon
it'll get fixed.

As for developing major sites with Rails, most managers don't have the
balls. They'd rather pay millions to get a java solution, it isn't
their money on the budget so they gutlessly pour it down the java hole
and hope for the best. If the project fails they blame the team or
throw more money and bodies at the problem, of course it's not java's
fault or theirs.

Anyway I don't hold prejedice again java or c# but they are in no way a
safe bet.

Cheers.

 

[Chad Perrin]

As for developing major sites with Rails, most managers don't have the
balls. They'd rather pay millions to get a java solution, it isn't
their money on the budget so they gutlessly pour it down the java hole
and hope for the best. If the project fails they blame the team or
throw more money and bodies at the problem, of course it's not java's
fault or theirs.

Anyway I don't hold prejedice again java or c# but they are in no way a
safe bet.

Sure it is. You'll (almost) never have to fear for your job based on a
decision to go with Java or a Microsoft "solution", even if it is
entirely the WRONG decision. You could cost the company millions, end
up getting dozens of people laid off, and tank the entire project, but
if the language by which you did so is Java or C# you may still have job
security (as long as you haven't made other high-profile bad decisions).
The problem with job security in that circumstance only really arises if
there was a bitter power struggle over whether to go with Java or .NET,
and your side "won", then the project tanked at a cost of millions. The
opposing "side" might just blame the language/framework decision.

On the flipside, even where from a technical standpoint it's almost
impossible to avoid thinking something like Ruby on Rails, or Perl's
Catalyst, or Python's Django, is the best option, you may well find
yourself losing a job even if you made the right decision and the
project was well on its way to being a howling success. All it takes is
a poorly-timed change in management structure, and they may junk all the
work that has already been done at a cost of millions to rewrite
everything in Java or C# (or, God forbid, VB.NET), and fire you and all
your buddies for doing great work very quickly in the "wrong" language.

Corporate politics. Whee.

 

[William Grosso]

At the risk of talking to a wall ... give me a break.

Ruby's a very nice language. Rail's an amazing framework.

But. They are neither dominant nor even close to universally
appropriate. They have things they do well, and they have things
they don't do well, and they have things they don't do at all.
And they have design foci which make them appropriate for some
tasks, and not appropriate for others.

All of which should be completely obvious.

Here's the sermon: Pretending that decisions you don't understand
were made entirely for political reasons, or because the people making
the decision are stupid, is a sure-fire way to prevent yourself from
ever learning anything. Instead of indulging in free-form bile, why
not ask "What would have to be true for that to be the right decision?"

You'd be surprised how much insight such a simple question can
generate.


William Grosso

 

[Chad Perrin]

At the risk of talking to a wall ... give me a break.

Ruby's a very nice language. Rail's an amazing framework.

But. They are neither dominant nor even close to universally
appropriate. They have things they do well, and they have things
they don't do well, and they have things they don't do at all.
And they have design foci which make them appropriate for some
tasks, and not appropriate for others.

Who said otherwise? And why are you top-posting?

All of which should be completely obvious.

Here's the sermon: Pretending that decisions you don't understand
were made entirely for political reasons, or because the people making
the decision are stupid, is a sure-fire way to prevent yourself from
ever learning anything. Instead of indulging in free-form bile, why
not ask "What would have to be true for that to be the right decision?"

Perhaps you should read what I said a second, and maybe even third,
time. In paraphrase, it was (summarized):

Regardless of how good or bad a decision a given language is for a
given task, Ruby is more likely to get you fired that Java.

You'd be surprised how much insight such a simple question can
generate.

You might be surprised by how much actually reading and trying to
understand makes, as opposed to jumping to conclusions about someone's
malicious intent regarding a discussion of the corporate politics of
language and tool choice.

Despite the fact it got this far via top-posting, a no-no here at
ruby-talk/comp.lang.ruby, I'll leave the text you quoted at the bottom
so you can more easily peruse it again at your leisure.

 

[David Vallner]

I didn't. Depending on what the language needs to do it, would be easy to
write a lanuage. From what he wrote, it sounds like wasabi was written to
his specific domain, and so of course he would trust it more then some
other
general purpose language write by some one else. Now, I'm not saying that
other should trust wasabi. I know I would not, but that's just me trusting
what I know, more then just blanking trusting Jole.

And another point is that quite a few Ruby frameworks do come to
defining a domain-specific language in Ruby - cf. Og data definition,
Puppet, rake. There's a (maybe not quite fine) line between a very
specific framework and a DSL that just gets crossed, and I don't believe
rubyists are the innocents to throw the first stone.

Then again, if Ruby is to be accused of being too much of an unsafe toy,
might as well go the full way

David Vallner

 

[David Vallner]

Perhaps you should read what I said a second, and maybe even third,
time. In paraphrase, it was (summarized):

Regardless of how good or bad a decision a given language is for a
given task, Ruby is more likely to get you fired that Java.

To be fair, it's not just corporate politics. Statistically, it's more
likely a development house will have a strong base of Java developers or
C# developers (C#, while being very young and so far an abomination unto
Nuggan, is reasonably Java compatible), and that starting a Rails
project means you'll probably have to get people with no Ruby experience
on the team, or create a burden on the company in case the original team
falls apart and quits to other companies regarding maintenance, or whatever.

While the programming language decision might or might not have anything
to do with whether the project succeeds, choosing a *locally* unproven
language DOES make the project inherently higher-risk, and makes the
managers overall nervous - whence the likelihood of getting fired being
higher. It's not punishment for your failure, it's more for all the
other mess you could've caused even if the project succeeded, even if
the management might not be consciously aware of that.

How good a language or the frameworks for it are to initially develop
something in is not (maybe not by far) the most important factor when
making a decision.

David Vallner

 

[David Vallner]

Utter pants. I mean, you used the word "bloat", which should make people
lose any debate by default.

Java and C# are no guarantee for success.

Neither is Ruby / Rails. *No technology* is a guarantee for success, no
technology ever was, and I'll bet a gold bar against a plastic spoon no
technology ever will. Technology used is a very important decision to
make, but it never single-handedly moves you from doable to undoable or
vice versa.

you seem to need 10 instead of 3 people and 5 times as long.

Pure, unadulterated shite. Give me numbers. Credible statistics and real
research, not random anectodal success stories that are too pathethic to
sell Herbalife diet pills.

Also, initial development cost isn't a very important factor. Recalls
your uni software lifecycle charts about how much of a project's life is
maintenance. For a successful project, the numbers are very much true.
With a successful product comes the responsibility of supporting it and
keeping it successful, and in some cases this responsibility creates
ongoing costs that dwarf the initial development horribly.

Ok, sure Java's OO may be nicer than Perl 5's but once you brew
HTML/Javascript/JSP/JSTL/EL/tags/JSF or Struts together the result
isn't exactly what I'd call pretty. Java is in no way a safe bet.

Noone cares about pretty. It's also a completely irrelevant issue when
deciding on implementation language if you're at least remotely responsible.

How about C#, well it runs in Windows and without serious and expensive
firewalls you just can't go anywhere near the Internet.

You need to tighten off Unix-based servers too. Heck, there are even
serious and expensive firewalls for Linux around too, because not
everyone has an in-house iptables guru.

Ruby and Rails just get straight to the point. They make common things
easy and elegant.

Sometimes things aren't so common. Ruby and Rails DO have faults. Just
google around, I'm not going to go namecall out of respect and out of a
sense of realism - every technology has flaws and any mudslinging would
only lead to a pointless flamewar. Sometimes they are uneducated rants
and / or whining, but some of them are valid.

And if you do NOT go out and learn about these flaws, and what impact
they could have, and be fully aware of them when making the
implementation technology decision on a project to consider the severity
of their impact under the circumstances of your project, then your
decision may cause a lot of trouble.

If execution speed really is a problem then I reckon
it'll get fixed.

Speaking purely theorethically, Ruby can not be made as performant as
Java or C# could be made if they had ideally performing implementations.
Latent typing makes it almost impossible to do certain optimizations as
static typing does. That's pure fact. Of course, it's not saying Ruby
can't be fast enough - but there have been people with more experience
at the performance side of software development that talked much better
about that

As for developing major sites with Rails, most managers don't have the
balls.

I advise you go on throught freshman year on a management school. It's
the managers' job to "not have balls" and risk when there's apparently
nothing to be had from taking it. If you want to be a Ruby advocate, you
need to be able to persuade them, not yourself, of the advantages or
using it.

They'd rather pay millions to get a java solution, it isn't
their money on the budget so they gutlessly pour it down the java hole
and hope for the best. If the project fails they blame the team or
throw more money and bodies at the problem, of course it's not java's
fault or theirs.

That's because it's not. Since projects never fail purely on a
technology solution - they fail on results of bad project planning more
often than not (assigning novice programmers to large projects that will
probably go over their heads), mistaken business objectives as whoever
contracted the software finds he isn't really interested in what the
tech demos had to show at all.

And the stereotype of lazy management that never gets punished is good
to make Dilbert strips from - in real life, it probably doesn't hold
true outside of a select few huge moloch companies, or on the opposite
side of the spectrum small short-lived hick-led shops where the bosses
kids and nephews gets all sort of crap assigned to get better allowance.
In a well-led company with working internal management processes, when
the shit hits the fan, everyone gets the stink.

David Vallner

 

[Alex Young]

And another point is that quite a few Ruby frameworks do come to
defining a domain-specific language in Ruby - cf. Og data definition,
Puppet, rake. There's a (maybe not quite fine) line between a very
specific framework and a DSL that just gets crossed, and I don't believe
rubyists are the innocents to throw the first stone.

That's not quite the same - those DSL's build upon a known and well
understood foundation, because they use Ruby's syntax to their own ends.
I'm inferring from the very little information that's out there that
Wasabi has its own parser, and that makes it a very, very different
beast to a DSL in the sense that I've come across the term in Ruby.

 

[James Britt]

Alex Young wrote:

I'm inferring from the very little information that's out there that
Wasabi has its own parser, and that makes it a very, very different
beast to a DSL in the sense that I've come across the term in Ruby.

Some Wasabi info:

http://www.joelonsoftware.com/items/2006/09/01b.html
http://programming.reddit.com/info/g0fa/comments



--
James Britt

"Simplicity of the language is not what matters, but
simplicity of use."
- Richard A. O'Keefe in squeak-dev mailing list