Ruby vs. PHP

[M. Edward (Ed) Borasky]

--------------090002030006040401090908
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Nice. I wasn't expecting anything as effective for a quick, by-the-pants
analysis as a box plot. What did you use to develop the graphic? I'm
curious. . . .

R ... and no, I haven't ported the benchmarks to R ... and yes, I should.

I also suppose I should post the R code to RubyForge.

Correct me if I'm wrong, but . . . won't doing this as a gcc-comparative
ratio bias slower languages toward an exaggerated upper bound on the box?

I admit I haven't done this sort of thing in a while, so my sorta
heuristic analysis of what I see may be prone to minor glitches.

Well, actually, yes, the gcc "box" is a meaningless point. Also, the
*log* of the ratio is better behaved than the ratio itself. It's more
like a Gaussian, which is why the whole geometric mean thing works.

Both yarv and ruby appear to be prone to low-end statistical outliers, or
perhaps to a "long tail" on the slow side of things.

Yeah ... again, it looks better with the log of the ratios.

Actually, the ruby and php medians (using lower-case to denote
implementations) are almost identical, and perl's isn't far off,
according to this. It appears that only in instances approaching
worst-case does ruby really begin to suffer. Also, if I'm not entirely
missing the implications of a gcc-ratio comparison, I'm not entirely sure
that we can trust the huge visual differences in the height of the boxes
to indicate significant performance problem areas.

OK ... you've convinced me ... I actually computed the logs, so I might
as well make the boxplots of them too.

I'm also not sure the jruby implementation's numbers here are
representative of JRuby's strengths. Does JRuby benefit from the
benefits of Java's optimizing VM for long-running processes? I know it's
probably suffering, in micro-benchmarks, from increased start-up times as
it loads the VM.

I think it's more a case of this being an old jRuby -- Charlie tunes it
more or less daily and I've seen some benchmarks where it beats MRI.

I'm surprised to see python's median so low. Are these benchmarks heavy
on bytecode-optimized Python, or do they tend to use standard interpreted
Python?

I'll have to check that ... I assumed it was bytecode.

Here's the boxplot of the log (base 10) ratios. *Very* interesting --
Perl has a heavier tail on the *fast* side than it does on the slow side.

--------------090002030006040401090908
Content-Type: application/pdf;
name="ln-shootout.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename="ln-shootout.pdf"

JVBERi0xLjEKJYHigeOBz4HTXHIKMSAwIG9iago8PAovQ3JlYXRpb25EYXRlIChEOjIwMDcx
MDA2MjEyMDQ0KQovTW9kRGF0ZSAoRDoyMDA3MTAwNjIxMjA0NCkKL1RpdGxlIChSIEdyYXBo
aWNzIE91dHB1dCkKL1Byb2R1Y2VyIChSIDIuNi4wKQovQ3JlYXRvciAoUikKPj4KZW5kb2Jq
CjIgMCBvYmoKPDwKL1R5cGUgL0NhdGFsb2cKL1BhZ2VzIDMgMCBSCj4+CmVuZG9iago1IDAg
b2JqCjw8Ci9UeXBlIC9Gb250Ci9TdWJ0eXBlIC9UeXBlMQovTmFtZSAvRjEKL0Jhc2VGb250
IC9aYXBmRGluZ2JhdHMKPj4KZW5kb2JqCjYgMCBvYmoKPDwKL1R5cGUgL1BhZ2UKL1BhcmVu
dCAzIDAgUgovQ29udGVudHMgNyAwIFIKL1Jlc291cmNlcyA0IDAgUgo+PgplbmRvYmoKNyAw
IG9iago8PAovTGVuZ3RoIDggMCBSCj4+CnN0cmVhbQ0KcQpRIHEgNTkuMDQgNzMuNDQgNjMw
LjcyIDQwNy41MiByZSBXIG4KMC4wMDAgMC4wMDAgMC4wMDAgUkcKMi4yNSB3CltdIDAgZAox
IEoKMSBqCjEwLjAwIE0KOTAuNzQgOTIuMzcgbSAxNTcuNDkgOTIuMzcgbCBTCjAuNzUgdwpb
IDMuMDAgNS4wMF0gMCBkCjEyNC4xMSA5Mi4zNyBtIDEyNC4xMSA5Mi4zNyBsIFMKMTI0LjEx
IDkyLjM3IG0gMTI0LjExIDkyLjM3IGwgUwowLjc1IHcKW10gMCBkCjEwNy40MyA5Mi4zNyBt
IDE0MC44MCA5Mi4zNyBsIFMKMTA3LjQzIDkyLjM3IG0gMTQwLjgwIDkyLjM3IGwgUwoxMDcu
NDMgOTIuMzcgbSAxNDAuODAgOTIuMzcgbCBTCjEwNy40MyA5Mi4zNyBtIDE0MC44MCA5Mi4z
NyBsIFMKMTA3LjQzIDkyLjM3IG0gMTQwLjgwIDkyLjM3IGwgUwoxMDcuNDMgOTIuMzcgbSAx
NDAuODAgOTIuMzcgbCBTCjEwNy40MyA5Mi4zNyBtIDE0MC44MCA5Mi4zNyBsIFMKMTA3LjQz
IDkyLjM3IG0gMTQwLjgwIDkyLjM3IGwgUwoxMDcuNDMgOTIuMzcgbSAxNDAuODAgOTIuMzcg
bCBTCjEwNy40MyA5Mi4zNyBtIDE0MC44MCA5Mi4zNyBsIFMKMTA3LjQzIDkyLjM3IG0gMTQw
LjgwIDkyLjM3IGwgUwoxMDcuNDMgOTIuMzcgbSAxNDAuODAgOTIuMzcgbCBTCjEwNy40MyA5
Mi4zNyBtIDE0MC44MCA5Mi4zNyBsIFMKMTA3LjQzIDkyLjM3IG0gMTQwLjgwIDkyLjM3IGwg
Uwo5MC43NCA5Mi4zNyBtCjE1Ny40OSA5Mi4zNyBsCjE1Ny40OSA5Mi4zNyBsCjkwLjc0IDky
LjM3IGwKOTAuNzQgOTIuMzcgbApTCjIuMjUgdwpbXSAwIGQKMTc0LjE3IDIwMi43NyBtIDI0
MC45MSAyMDIuNzcgbCBTCjAuNzUgdwpbIDMuMDAgNS4wMF0gMCBkCjIwNy41NCAxMDMuNzYg
bSAyMDcuNTQgMTcyLjE5IGwgUwoyMDcuNTQgMzY2LjU5IG0gMjA3LjU0IDMwNy43MiBsIFMK
MC43NSB3CltdIDAgZAoxOTAuODYgMTAzLjc2IG0gMjI0LjIzIDEwMy43NiBsIFMKMTkwLjg2
IDM2Ni41OSBtIDIyNC4yMyAzNjYuNTkgbCBTCjE5MC44NiAxMDMuNzYgbSAyMjQuMjMgMTAz
Ljc2IGwgUwoxOTAuODYgMzY2LjU5IG0gMjI0LjIzIDM2Ni41OSBsIFMKMTkwLjg2IDEwMy43
NiBtIDIyNC4yMyAxMDMuNzYgbCBTCjE5MC44NiAzNjYuNTkgbSAyMjQuMjMgMzY2LjU5IGwg
UwoxOTAuODYgMTAzLjc2IG0gMjI0LjIzIDEwMy43NiBsIFMKMTkwLjg2IDM2Ni41OSBtIDIy
NC4yMyAzNjYuNTkgbCBTCjE5MC44NiAxMDMuNzYgbSAyMjQuMjMgMTAzLjc2IGwgUwoxOTAu
ODYgMzY2LjU5IG0gMjI0LjIzIDM2Ni41OSBsIFMKMTkwLjg2IDEwMy43NiBtIDIyNC4yMyAx
MDMuNzYgbCBTCjE5MC44NiAzNjYuNTkgbSAyMjQuMjMgMzY2LjU5IGwgUwoxOTAuODYgMTAz
Ljc2IG0gMjI0LjIzIDEwMy43NiBsIFMKMTkwLjg2IDM2Ni41OSBtIDIyNC4yMyAzNjYuNTkg
bCBTCjE3NC4xNyAxNzIuMTkgbQoyNDAuOTEgMTcyLjE5IGwKMjQwLjkxIDMwNy43MiBsCjE3
NC4xNyAzMDcuNzIgbAoxNzQuMTcgMTcyLjE5IGwKUwoyLjI1IHcKW10gMCBkCjI1Ny42MCAy
MjYuNTcgbSAzMjQuMzQgMjI2LjU3IGwgUwowLjc1IHcKWyAzLjAwIDUuMDBdIDAgZAoyOTAu
OTcgMTEwLjI4IG0gMjkwLjk3IDE2OS44NCBsIFMKMjkwLjk3IDM0MS4yMyBtIDI5MC45NyAy
ODUuOTAgbCBTCjAuNzUgdwpbXSAwIGQKMjc0LjI5IDExMC4yOCBtIDMwNy42NiAxMTAuMjgg
bCBTCjI3NC4yOSAzNDEuMjMgbSAzMDcuNjYgMzQxLjIzIGwgUwoyNzQuMjkgMTEwLjI4IG0g
MzA3LjY2IDExMC4yOCBsIFMKMjc0LjI5IDM0MS4yMyBtIDMwNy42NiAzNDEuMjMgbCBTCjI3
NC4yOSAxMTAuMjggbSAzMDcuNjYgMTEwLjI4IGwgUwoyNzQuMjkgMzQxLjIzIG0gMzA3LjY2
IDM0MS4yMyBsIFMKMjc0LjI5IDExMC4yOCBtIDMwNy42NiAxMTAuMjggbCBTCjI3NC4yOSAz
NDEuMjMgbSAzMDcuNjYgMzQxLjIzIGwgUwoyNzQuMjkgMTEwLjI4IG0gMzA3LjY2IDExMC4y
OCBsIFMKMjc0LjI5IDM0MS4yMyBtIDMwNy42NiAzNDEuMjMgbCBTCjI3NC4yOSAxMTAuMjgg
bSAzMDcuNjYgMTEwLjI4IGwgUwoyNzQuMjkgMzQxLjIzIG0gMzA3LjY2IDM0MS4yMyBsIFMK
Mjc0LjI5IDExMC4yOCBtIDMwNy42NiAxMTAuMjggbCBTCjI3NC4yOSAzNDEuMjMgbSAzMDcu
NjYgMzQxLjIzIGwgUwoyNTcuNjAgMTY5Ljg0IG0KMzI0LjM0IDE2OS44NCBsCjMyNC4zNCAy
ODUuOTAgbAoyNTcuNjAgMjg1LjkwIGwKMjU3LjYwIDE2OS44NCBsClMKMi4yNSB3CltdIDAg
ZAozNDEuMDMgMjU4Ljg5IG0gNDA3Ljc3IDI1OC44OSBsIFMKMC43NSB3ClsgMy4wMCA1LjAw
XSAwIGQKMzc0LjQwIDg4LjUzIG0gMzc0LjQwIDE0Ni4xMSBsIFMKMzc0LjQwIDM2My4yNCBt
IDM3NC40MCAyOTYuODkgbCBTCjAuNzUgdwpbXSAwIGQKMzU3LjcxIDg4LjUzIG0gMzkxLjA5
IDg4LjUzIGwgUwozNTcuNzEgMzYzLjI0IG0gMzkxLjA5IDM2My4yNCBsIFMKMzU3LjcxIDg4
LjUzIG0gMzkxLjA5IDg4LjUzIGwgUwozNTcuNzEgMzYzLjI0IG0gMzkxLjA5IDM2My4yNCBs
IFMKMzU3LjcxIDg4LjUzIG0gMzkxLjA5IDg4LjUzIGwgUwozNTcuNzEgMzYzLjI0IG0gMzkx
LjA5IDM2My4yNCBsIFMKMzU3LjcxIDg4LjUzIG0gMzkxLjA5IDg4LjUzIGwgUwozNTcuNzEg
MzYzLjI0IG0gMzkxLjA5IDM2My4yNCBsIFMKMzU3LjcxIDg4LjUzIG0gMzkxLjA5IDg4LjUz
IGwgUwozNTcuNzEgMzYzLjI0IG0gMzkxLjA5IDM2My4yNCBsIFMKMzU3LjcxIDg4LjUzIG0g
MzkxLjA5IDg4LjUzIGwgUwozNTcuNzEgMzYzLjI0IG0gMzkxLjA5IDM2My4yNCBsIFMKMzU3
LjcxIDg4LjUzIG0gMzkxLjA5IDg4LjUzIGwgUwozNTcuNzEgMzYzLjI0IG0gMzkxLjA5IDM2
My4yNCBsIFMKMzQxLjAzIDE0Ni4xMSBtCjQwNy43NyAxNDYuMTEgbAo0MDcuNzcgMjk2Ljg5
IGwKMzQxLjAzIDI5Ni44OSBsCjM0MS4wMyAxNDYuMTEgbApTCjIuMjUgdwpbXSAwIGQKNDI0
LjQ2IDI2Ny4xMiBtIDQ5MS4yMCAyNjcuMTIgbCBTCjAuNzUgdwpbIDMuMDAgNS4wMF0gMCBk
CjQ1Ny44MyAxMTAuMzggbSA0NTcuODMgMTQ5LjEwIGwgUwo0NTcuODMgMzQ5LjY3IG0gNDU3
LjgzIDI5Ni40MCBsIFMKMC43NSB3CltdIDAgZAo0NDEuMTQgMTEwLjM4IG0gNDc0LjUxIDEx
MC4zOCBsIFMKNDQxLjE0IDM0OS42NyBtIDQ3NC41MSAzNDkuNjcgbCBTCjQ0MS4xNCAxMTAu
MzggbSA0NzQuNTEgMTEwLjM4IGwgUwo0NDEuMTQgMzQ5LjY3IG0gNDc0LjUxIDM0OS42NyBs
IFMKNDQxLjE0IDExMC4zOCBtIDQ3NC41MSAxMTAuMzggbCBTCjQ0MS4xNCAzNDkuNjcgbSA0
NzQuNTEgMzQ5LjY3IGwgUwo0NDEuMTQgMTEwLjM4IG0gNDc0LjUxIDExMC4zOCBsIFMKNDQx
LjE0IDM0OS42NyBtIDQ3NC41MSAzNDkuNjcgbCBTCjQ0MS4xNCAxMTAuMzggbSA0NzQuNTEg
MTEwLjM4IGwgUwo0NDEuMTQgMzQ5LjY3IG0gNDc0LjUxIDM0OS42NyBsIFMKNDQxLjE0IDEx
MC4zOCBtIDQ3NC41MSAxMTAuMzggbCBTCjQ0MS4xNCAzNDkuNjcgbSA0NzQuNTEgMzQ5LjY3
IGwgUwo0NDEuMTQgMTEwLjM4IG0gNDc0LjUxIDExMC4zOCBsIFMKNDQxLjE0IDM0OS42NyBt
IDQ3NC41MSAzNDkuNjcgbCBTCjQyNC40NiAxNDkuMTAgbQo0OTEuMjAgMTQ5LjEwIGwKNDkx
LjIwIDI5Ni40MCBsCjQyNC40NiAyOTYuNDAgbAo0MjQuNDYgMTQ5LjEwIGwKUwoyLjI1IHcK
W10gMCBkCjUwNy44OSAyNzEuNjMgbSA1NzQuNjMgMjcxLjYzIGwgUwowLjc1IHcKWyAzLjAw
IDUuMDBdIDAgZAo1NDEuMjYgMTExLjMzIG0gNTQxLjI2IDE4Ni4zNSBsIFMKNTQxLjI2IDQx
Mi44NCBtIDU0MS4yNiAzNTMuNTYgbCBTCjAuNzUgdwpbXSAwIGQKNTI0LjU3IDExMS4zMyBt
IDU1Ny45NCAxMTEuMzMgbCBTCjUyNC41NyA0MTIuODQgbSA1NTcuOTQgNDEyLjg0IGwgUwo1
MjQuNTcgMTExLjMzIG0gNTU3Ljk0IDExMS4zMyBsIFMKNTI0LjU3IDQxMi44NCBtIDU1Ny45
NCA0MTIuODQgbCBTCjUyNC41NyAxMTEuMzMgbSA1NTcuOTQgMTExLjMzIGwgUwo1MjQuNTcg
NDEyLjg0IG0gNTU3Ljk0IDQxMi44NCBsIFMKNTI0LjU3IDExMS4zMyBtIDU1Ny45NCAxMTEu
MzMgbCBTCjUyNC41NyA0MTIuODQgbSA1NTcuOTQgNDEyLjg0IGwgUwo1MjQuNTcgMTExLjMz
IG0gNTU3Ljk0IDExMS4zMyBsIFMKNTI0LjU3IDQxMi44NCBtIDU1Ny45NCA0MTIuODQgbCBT
CjUyNC41NyAxMTEuMzMgbSA1NTcuOTQgMTExLjMzIGwgUwo1MjQuNTcgNDEyLjg0IG0gNTU3
Ljk0IDQxMi44NCBsIFMKNTI0LjU3IDExMS4zMyBtIDU1Ny45NCAxMTEuMzMgbCBTCjUyNC41
NyA0MTIuODQgbSA1NTcuOTQgNDEyLjg0IGwgUwo1MDcuODkgMTg2LjM1IG0KNTc0LjYzIDE4
Ni4zNSBsCjU3NC42MyAzNTMuNTYgbAo1MDcuODkgMzUzLjU2IGwKNTA3Ljg5IDE4Ni4zNSBs
ClMKMi4yNSB3CltdIDAgZAo1OTEuMzEgMjkxLjIwIG0gNjU4LjA2IDI5MS4yMCBsIFMKMC43
NSB3ClsgMy4wMCA1LjAwXSAwIGQKNjI0LjY5IDE1NC4zOSBtIDYyNC42OSAyMjAuNDAgbCBT
CjYyNC42OSA0NjUuODcgbSA2MjQuNjkgMzg5LjA3IGwgUwowLjc1IHcKW10gMCBkCjYwOC4w
MCAxNTQuMzkgbSA2NDEuMzcgMTU0LjM5IGwgUwo2MDguMDAgNDY1Ljg3IG0gNjQxLjM3IDQ2
NS44NyBsIFMKNjA4LjAwIDE1NC4zOSBtIDY0MS4zNyAxNTQuMzkgbCBTCjYwOC4wMCA0NjUu
ODcgbSA2NDEuMzcgNDY1Ljg3IGwgUwo2MDguMDAgMTU0LjM5IG0gNjQxLjM3IDE1NC4zOSBs
IFMKNjA4LjAwIDQ2NS44NyBtIDY0MS4zNyA0NjUuODcgbCBTCjYwOC4wMCAxNTQuMzkgbSA2
NDEuMzcgMTU0LjM5IGwgUwo2MDguMDAgNDY1Ljg3IG0gNjQxLjM3IDQ2NS44NyBsIFMKNjA4
LjAwIDE1NC4zOSBtIDY0MS4zNyAxNTQuMzkgbCBTCjYwOC4wMCA0NjUuODcgbSA2NDEuMzcg
NDY1Ljg3IGwgUwo2MDguMDAgMTU0LjM5IG0gNjQxLjM3IDE1NC4zOSBsIFMKNjA4LjAwIDQ2
NS44NyBtIDY0MS4zNyA0NjUuODcgbCBTCjYwOC4wMCAxNTQuMzkgbSA2NDEuMzcgMTU0LjM5
IGwgUwo2MDguMDAgNDY1Ljg3IG0gNjQxLjM3IDQ2NS44NyBsIFMKNTkxLjMxIDIyMC40MCBt
CjY1OC4wNiAyMjAuNDAgbAo2NTguMDYgMzg5LjA3IGwKNTkxLjMxIDM4OS4wNyBsCjU5MS4z
MSAyMjAuNDAgbApTClEgcQowLjAwMCAwLjAwMCAwLjAwMCBSRwowLjc1IHcKW10gMCBkCjEg
SgoxIGoKMTAuMDAgTQoxMjQuMTEgNzMuNDQgbSA2MjQuNjkgNzMuNDQgbCBTCjEyNC4xMSA3
My40NCBtIDEyNC4xMSA2Ni4yNCBsIFMKMjA3LjU0IDczLjQ0IG0gMjA3LjU0IDY2LjI0IGwg
UwoyOTAuOTcgNzMuNDQgbSAyOTAuOTcgNjYuMjQgbCBTCjM3NC40MCA3My40NCBtIDM3NC40
MCA2Ni4yNCBsIFMKNDU3LjgzIDczLjQ0IG0gNDU3LjgzIDY2LjI0IGwgUwo1NDEuMjYgNzMu
NDQgbSA1NDEuMjYgNjYuMjQgbCBTCjYyNC42OSA3My40NCBtIDYyNC42OSA2Ni4yNCBsIFMK
QlQKMC4wMDAgMC4wMDAgMC4wMDAgcmcKL0YyIDEgVGYgMTIuMDAgMC4wMCAtMC4wMCAxMi4w
MCAxMTQuNzggNDcuNTIgVG0gKGdjYykgVGoKL0YyIDEgVGYgMTIuMDAgMC4wMCAtMC4wMCAx
Mi4wMCAxOTYuMTUgNDcuNTIgVG0gKHlhcnYpIFRqCi9GMiAxIFRmIDEyLjAwIDAuMDAgLTAu
MDAgMTIuMDAgMjczLjE0IDQ3LjUyIFRtIChweXRob24pIFRqCi9GMiAxIFRmIDEyLjAwIDAu
MDAgLTAuMDAgMTIuMDAgMzY0LjMxIDQ3LjUyIFRtIChwZXJsKSBUagovRjIgMSBUZiAxMi4w
MCAwLjAwIC0wLjAwIDEyLjAwIDQ0Ny44MiA0Ny41MiBUbSAocGhwKSBUagovRjIgMSBUZiAx
Mi4wMCAwLjAwIC0wLjAwIDEyLjAwIDUyOS42MiA0Ny41MiBUbSAocnVieSkgVGoKL0YyIDEg
VGYgMTIuMDAgMC4wMCAtMC4wMCAxMi4wMCA2MTEuNzEgNDcuNTIgVG0gKGpydWJ5KSBUagpF
VAo1OS4wNCA5Mi4zNyBtIDU5LjA0IDQ0NC43MyBsIFMKNTkuMDQgOTIuMzcgbSA1MS44NCA5
Mi4zNyBsIFMKNTkuMDQgMTUxLjEwIG0gNTEuODQgMTUxLjEwIGwgUwo1OS4wNCAyMDkuODMg
bSA1MS44NCAyMDkuODMgbCBTCjU5LjA0IDI2OC41NSBtIDUxLjg0IDI2OC41NSBsIFMKNTku
MDQgMzI3LjI4IG0gNTEuODQgMzI3LjI4IGwgUwo1OS4wNCAzODYuMDAgbSA1MS44NCAzODYu
MDAgbCBTCjU5LjA0IDQ0NC43MyBtIDUxLjg0IDQ0NC43MyBsIFMKQlQKL0YyIDEgVGYgMC4w
MCAxMi4wMCAtMTIuMDAgMC4wMCA0MS43NiA4NC4wMyBUbSAoMC4wKSBUagovRjIgMSBUZiAw
LjAwIDEyLjAwIC0xMi4wMCAwLjAwIDQxLjc2IDE0Mi43NiBUbSAoMC41KSBUagovRjIgMSBU
ZiAwLjAwIDEyLjAwIC0xMi4wMCAwLjAwIDQxLjc2IDIwMS40OSBUbSAoMS4wKSBUagovRjIg
MSBUZiAwLjAwIDEyLjAwIC0xMi4wMCAwLjAwIDQxLjc2IDI2MC4yMSBUbSAoMS41KSBUagov
RjIgMSBUZiAwLjAwIDEyLjAwIC0xMi4wMCAwLjAwIDQxLjc2IDMxOC45NCBUbSAoMi4wKSBU
agovRjIgMSBUZiAwLjAwIDEyLjAwIC0xMi4wMCAwLjAwIDQxLjc2IDM3Ny42NiBUbSAoMi41
KSBUagovRjIgMSBUZiAwLjAwIDEyLjAwIC0xMi4wMCAwLjAwIDQxLjc2IDQzNi4zOSBUbSAo
My4wKSBUagpFVApRIHEKQlQKMC4wMDAgMC4wMDAgMC4wMDAgcmcKL0YyIDEgVGYgMC4wMCAx
Mi4wMCAtMTIuMDAgMC4wMCAxMi45NiAyNDAuODUgVG0gKExvZ1woMTBcKSBSYXRpbykgVGoK
RVQKUSBxCjAuMDAwIDAuMDAwIDAuMDAwIFJHCjAuNzUgdwpbXSAwIGQKMSBKCjEgagoxMC4w
MCBNCjU5LjA0IDczLjQ0IG0KNjg5Ljc2IDczLjQ0IGwKNjg5Ljc2IDQ4MC45NiBsCjU5LjA0
IDQ4MC45NiBsCjU5LjA0IDczLjQ0IGwKUwpRCmVuZHN0cmVhbQplbmRvYmoKOCAwIG9iago2
NjM3CmVuZG9iagozIDAgb2JqCjw8Ci9UeXBlIC9QYWdlcwovS2lkcyBbCjYgMCBSCl0KL0Nv
dW50IDEKL01lZGlhQm94IFswIDAgNzIwIDU0MF0KPj4KZW5kb2JqCjQgMCBvYmoKPDwKL1By
b2NTZXQgWy9QREYgL1RleHRdCi9Gb250IDw8IC9GMSA1IDAgUiAvRjIgMTAgMCBSID4+Ci9F
eHRHU3RhdGUgPDwgPj4KPj4KZW5kb2JqCjkgMCBvYmoKPDwKL1R5cGUgL0VuY29kaW5nCi9C
YXNlRW5jb2RpbmcgL1dpbkFuc2lFbmNvZGluZwovRGlmZmVyZW5jZXMgWyA0NS9taW51cyA5
Ni9xdW90ZWxlZnQKMTQ0L2RvdGxlc3NpIC9ncmF2ZSAvYWN1dGUgL2NpcmN1bWZsZXggL3Rp
bGRlIC9tYWNyb24gL2JyZXZlIC9kb3RhY2NlbnQKL2RpZXJlc2lzIC8ubm90ZGVmIC9yaW5n
IC9jZWRpbGxhIC8ubm90ZGVmIC9odW5nYXJ1bWxhdXQgL29nb25layAvY2Fyb24gL3NwYWNl
XQo+PgplbmRvYmoKMTAgMCBvYmogPDwKL1R5cGUgL0ZvbnQKL1N1YnR5cGUgL1R5cGUxCi9O
YW1lIC9GMgovQmFzZUZvbnQgL0hlbHZldGljYQovRW5jb2RpbmcgOSAwIFIKPj4gZW5kb2Jq
CnhyZWYKMCAxMQowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMjEgMDAwMDAgbiAKMDAw
MDAwMDE2MyAwMDAwMCBuIAowMDAwMDA3MDg1IDAwMDAwIG4gCjAwMDAwMDcxNjggMDAwMDAg
biAKMDAwMDAwMDIxMiAwMDAwMCBuIAowMDAwMDAwMjk1IDAwMDAwIG4gCjAwMDAwMDAzNzUg
MDAwMDAgbiAKMDAwMDAwNzA2NSAwMDAwMCBuIAowMDAwMDA3MjYxIDAwMDAwIG4gCjAwMDAw
MDc1MTggMDAwMDAgbiAKdHJhaWxlcgo8PAovU2l6ZSAxMQovSW5mbyAxIDAgUgovUm9vdCAy
IDAgUgo+PgpzdGFydHhyZWYKNzYxNQolJUVPRgo=
--------------090002030006040401090908--

 

[Chad Perrin]

R ... and no, I haven't ported the benchmarks to R ... and yes, I should.

I also suppose I should post the R code to RubyForge.

Then maybe we can turn a translation of that R code into Ruby into a Ruby
Quiz with some kind of additional requirements . . .

. . or not.

Well, actually, yes, the gcc "box" is a meaningless point. Also, the
*log* of the ratio is better behaved than the ratio itself. It's more
like a Gaussian, which is why the whole geometric mean thing works.

Good to know I wasn't getting high on the smell of dishwasher detergent
when I questioned the choice of a ratio to gcc performance.

Yeah ... again, it looks better with the log of the ratios.


OK ... you've convinced me ... I actually computed the logs, so I might
as well make the boxplots of them too.

Thanks!

These results look a lot more like what I expected, especially with the
high-performance tail weighting for main Perl and PHP implementations.
I'm still pretty impressed with Python's median performance here, as
contrasted with what I expected -- unless those are bytecode performance
benchmarks, in which case it's not quite as surprising. Also, since the
kind of work I tend to do doesn't really lend itself to persistent
bytecode compilation, I'd be less inclined to consider those benchmarks
(assuming I thought performance benchmarking in this class of languages
was likely to benefit me much in language choice at all).

Also . . . the log of ratio box plots don't make JRuby look like such a
dog in comparison with the rest, which is nice to see, even though it
still lags overall here.

By the way, reading this sentence of yours was the point at which I
thought "Y'know, I like disagreeing with this guy more than I like
agreeing with most other people I encounter in the Internet." A
disagreement with you seems to be going quite well -- polite,
informative, and educational. I'm learning while I'm stubbornly
disputing you, in other words. Thanks.

I think it's more a case of this being an old jRuby -- Charlie tunes it
more or less daily and I've seen some benchmarks where it beats MRI.

I'm curious about both newer numbers and the effects of VM startup and VM
optimization on performance here -- though not quite curious enough to
bother installing it myself.

I'll have to check that ... I assumed it was bytecode.

Here's the boxplot of the log (base 10) ratios. *Very* interesting --
Perl has a heavier tail on the *fast* side than it does on the slow side.

I've known for a long time that Perl does some impressive things with
execution performance. The JIT parse tree compilation it does is a big
piece of that -- as is the fine-tuning the JIT compilation process has
gotten over the years.

 

[khaines]

gcc 1.0
yarv 13.2
perl 14.0
python 14.2
php 15.5
ruby 29.9
jruby 55.0