Does anyone have benchmark programs for YARV?

[SASADA Koichi]

Hi,

I'm developing YARV - Yet Another RubyVM. I'm looking for
benchmark programs to evaluate YARV's performance. So, Does
anyone have suitable benchmarks for this purpose?

Current YARV has some limitations (especially, can't require
any files). But I can fix YARV or your program to work and
measure execution time.

If you have a ruby program which is slow for you, feel free to
send me that Perhaps, YARV will be optimized for your
application.

Thanks,

 

[Florian Gross]

I'm developing YARV - Yet Another RubyVM. I'm looking for
benchmark programs to evaluate YARV's performance. So, Does
anyone have suitable benchmarks for this purpose?

Maybe you can have a look at the Ruby code for the Debian language
shootout (available at http://shootout.alioth.debian.org/) -- that way
you could also easily compare performance to that of other programming
languages which might be a good motivation.

 

[SASADA Koichi]

Hi,

I already use that
http://www.atdot.net/svn/yarv/trunk/benchmark/

Thanks anyway.

 

[Jamis Buck]

Hi,

I'm developing YARV - Yet Another RubyVM. I'm looking for
benchmark programs to evaluate YARV's performance. So, Does
anyone have suitable benchmarks for this purpose?

Current YARV has some limitations (especially, can't require
any files). But I can fix YARV or your program to work and
measure execution time.

If you have a ruby program which is slow for you, feel free to
send me that Perhaps, YARV will be optimized for your
application.

Thanks,

I don't know if this is what you are looking for or not, but Needle
(http://rubyforge.org/projects/needle) has a 'benchmarks' subdirectory
with a handful of different benchmarks.

If you manage to get them to run under YARV, I'd love to see the results.

- Jamis

 

[Florian Gross]

I already use that
http://www.atdot.net/svn/yarv/trunk/benchmark/

Interesting, is the speedup in percent available somewhere?

 

[gabriele renzi]

SASADA Koichi ha scritto:

Hi,

I'm developing YARV - Yet Another RubyVM. I'm looking for
benchmark programs to evaluate YARV's performance. So, Does
anyone have suitable benchmarks for this purpose?

I don't have a benchmark.. but I think the OO Richards Benchmark as
found here:
http://pws.prserv.net/dlissett//ben/bench1-ruby-28.htm
may be interesting.

Current YARV has some limitations (especially, can't require
any files). But I can fix YARV or your program to work and
measure execution time.

If you have a ruby program which is slow for you, feel free to
send me that Perhaps, YARV will be optimized for your
application.

Not that it is a valuable application... but I have this dumb example of
using backtracking to solve the map coloring problem as found here:
http://www.ccs.neu.edu/home/dorai/t-y-scheme/t-y-scheme-Z-H-16.html#node_sec_14.4.2

It stresses callcc a lot and takes ~15 seconds to terminate on my old
athlon 2000 box.
If you want take a look at it at:
http://rafb.net/paste/results/x4gckY22.html

(sorry for italian naming I guess it is still understandable

 

[gabriele renzi]

gabriele renzi ha scritto:

SASADA Koichi ha scritto:



I don't have a benchmark.. but I think the OO Richards Benchmark as
found here:
http://pws.prserv.net/dlissett//ben/bench1-ruby-28.htm
may be interesting.




Not that it is a valuable application... but I have this dumb example of
using backtracking to solve the map coloring problem as found here:
http://www.ccs.neu.edu/home/dorai/t-y-scheme/t-y-scheme-Z-H-16.html#node_sec_14.4.2

It stresses callcc a lot and takes ~15 seconds to terminate on my old
athlon 2000 box.
If you want take a look at it at:
http://rafb.net/paste/results/x4gckY22.html

btw, today I should study butwhen one is forced to do stuff, he revertes
to something else. So I get yarv and tried it on my winxp setup

[gcc version 3.2.3 (mingw special 20030504-1), athlon 2000+ today
snapshot from 1.9 and yarv at revision 77].

Did make test-all and got it run fine (even if there were some warnings
in the compilation)
Then I stuffed my code as bm_4col.rb in the benchmark/ dir and launched
make bench. I got a crash on my code[1], but, well, I didn't expect it
to work .
Results for other benchmarks are really impressive tough.
(i.e. the ackermann function timing is :
ruby 2.493000 0.010000 2.503000 ( 2.624000)
yarv 0.250000 0.000000 0.250000 ( 0.261000)
)

If someone's curious I've put them at:
ada2.unipv.it/~riffraff/bench.log.gz





[1]
../benchmark/bm_4col.rb:93: [BUG] BUG: unknown node: NODE_MASGN
ruby 1.9.0 (2004-12-03) [i386-mingw32]

 

[gabriele renzi]

gabriele renzi ha scritto:

further note:
make test-all from time to time gives:


c:/ruby/lib/ruby/1.9/test/unit/util/backtracefilter.rb:16: [BUG]
Segmentation fault
ruby 1.9.0 (2004-12-03) [i386-mingw32]


This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
make: *** [test-all] Error 3

 

[SASADA Koichi]

Hi,

Interesting, is the speedup in percent available somewhere?

today's result:
http://www.atdot.net/yarv/bench_20041206.txt

Regards,
SASADA Koichi

 

[SASADA Koichi]

Hi,

I don't know if this is what you are looking for or not, but Needle
(http://rubyforge.org/projects/needle) has a 'benchmarks' subdirectory
with a handful of different benchmarks.

If you manage to get them to run under YARV, I'd love to see the results.

- Jamis

I'll try it if 'require' is implemented.

Thanks,
SASADA Koichi

 

[SASADA Koichi]

Hi,

btw, today I should study butwhen one is forced to do stuff, he revertes
to something else. So I get yarv and tried it on my winxp setup

[gcc version 3.2.3 (mingw special 20030504-1), athlon 2000+ today
snapshot from 1.9 and yarv at revision 77].

Did make test-all and got it run fine (even if there were some warnings
in the compilation)
Then I stuffed my code as bm_4col.rb in the benchmark/ dir and launched
make bench. I got a crash on my code[1], but, well, I didn't expect it
to work .
Results for other benchmarks are really impressive tough.
(i.e. the ackermann function timing is :
ruby 2.493000 0.010000 2.503000 ( 2.624000)
yarv 0.250000 0.000000 0.250000 ( 0.261000)
)

If someone's curious I've put them at:
ada2.unipv.it/~riffraff/bench.log.gz

I'll try to run your program.

Thanks,
SASADA Koichi

 

[SASADA Koichi]

Hi,

gabriele renzi ha scritto:

further note:
make test-all from time to time gives:


c:/ruby/lib/ruby/1.9/test/unit/util/backtracefilter.rb:16: [BUG]
Segmentation fault
ruby 1.9.0 (2004-12-03) [i386-mingw32]


This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
make: *** [test-all] Error 3

Could you run following command on today's revision?

$ make test-all TESTOPT=-v


Thanks,
SASADA Koichi

 

[Michael Neumann]

Hi,



today's result:
http://www.atdot.net/yarv/bench_20041206.txt

wow! very impressive results... looking forward to be able to use yarv...

Regards,

Michael

 

[gabriele renzi]

SASADA Koichi ha scritto:

Hi,

gabriele renzi ha scritto:

further note:
make test-all from time to time gives:


c:/ruby/lib/ruby/1.9/test/unit/util/backtracefilter.rb:16: [BUG]
Segmentation fault
ruby 1.9.0 (2004-12-03) [i386-mingw32]


This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
make: *** [test-all] Error 3

Could you run following command on today's revision?

I tried it yesterday and it worked (I tried many times and always worked)

So revision 81 seem to work fine, revisions 82/83 cause the compiler to
fail:

C:\yarv\trunk>make
gcc -g -O2 -I. -Ic:/ruby/lib/ruby/1.9/i386-mingw32
-Ic:/ruby/lib/ruby/1.9/i386
-mingw32 -I. -DDISPATCH_THREADED_CODE -DDISPATCH_DIRECT_THREADED_CODE
-DOPT_BASI
C_OPERATIONS -DOPT_REGEXP_MATCH -c compile.c
gcc -g -O2 -fno-crossjumping -I. -Ic:/ruby/lib/ruby/1.9/i386-mingw32
-Ic:/ruby
/lib/ruby/1.9/i386-mingw32 -I. -DDISPATCH_THREADED_CODE
-DDISPATCH_DIRECT_THREAD
ED_CODE -DOPT_BASIC_OPERATIONS -DOPT_REGEXP_MATCH -c vm.c
cc1.exe: unrecognized option `-fno-crossjumping'
make: *** [vm.o] Error 1

 

[Simon Strandgaard]

I'm developing YARV - Yet Another RubyVM. I'm looking for
benchmark programs to evaluate YARV's performance. So, Does
anyone have suitable benchmarks for this purpose?

I have made a ruby compatible regexp engine.. written entirely in Ruby.
It builds some rather big AST's and does some heavy computations.

On my 700 MHz box it takes 37.8 seconds to run the default tests.
And it takes 19.6 seconds to run all the ~1650 rubicon regexps.
There is 580 tests and 677 assertions (and 0 failures, 0 errors).

http://raa.ruby-lang.org/project/regexp/

 

[gabriele renzi]

gabriele renzi ha scritto:

gcc -g -O2 -fno-crossjumping -I. -Ic:/ruby/lib/ruby/1.9/i386-mingw32
-Ic:/ruby
/lib/ruby/1.9/i386-mingw32 -I. -DDISPATCH_THREADED_CODE
-DDISPATCH_DIRECT_THREAD
ED_CODE -DOPT_BASIC_OPERATIONS -DOPT_REGEXP_MATCH -c vm.c
cc1.exe: unrecognized option `-fno-crossjumping'
make: *** [vm.o] Error 1

fwiw, this was, again a winxp box on a athlon 2000+,
gcc version 3.2.3 (mingw special 20030504-1)


Using gcc version 3.4.2 (mingw-special) it compiles and runs fine on
that windows box.
BTW, in case I find some problems should I report them via the tracker
interface at rubyforge, or via yarv-devel, or just ignore them because
it's a 0.0.0x release?

 

[SASADA Koichi]

Hi,

fwiw, this was, again a winxp box on a athlon 2000+,
gcc version 3.2.3 (mingw special 20030504-1)


Using gcc version 3.4.2 (mingw-special) it compiles and runs fine on
that windows box.

I added check that gcc is support this option or not.

BTW, in case I find some problems should I report them via the tracker
interface at rubyforge, or via yarv-devel, or just ignore them because
it's a 0.0.0x release?

Plsase post yarv-devel


Regards,
ko1

 

[John Tromp]

I have a nice integer benchmark solving Connect-4,
currently available in Java and C (under 500 lines each):

http://www.cwi.nl/~tromp/c4/fhour.html

I'm currently busy porting it to Haskell,
but a Ruby version would also be a welcome addition.

Does anybody here have the time and energy to port it to Ruby?

regards,
-John