[ANN] Using RubyInline for Optimization

D

Dominik Bathon

I wrote an article on using RubyInline for optimization where I take =
png.rb, sprinkle in a little profiling and a little C and make it go =
over 100 times faster.

Nice article, but in this case it is possible to get almost the same =

speedup in pure Ruby:

Base version:

def to_blob
blob =3D []
blob << [137, 80, 78, 71, 13, 10, 26, 10].pack("C*") # PNG signatur=
e
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack("N2C5=
"))
# 0 =3D=3D filter type code "none"
data =3D @data.map { |row| [0] + row.map { |p| p.values } }.flatten=

blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.pack("C*"), 9)=
)
blob << PNG.chunk('IEND', '')
blob.join
end

$ time ruby -Ilib profile.rb

real 0m15.504s
user 0m15.119s
sys 0m0.309s


Avoiding flatten (and using a literal for the signature):

def to_blob
blob =3D []
blob << "\211PNG\r\n\032\n" # PNG signature
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack("N2C5=
"))
# 0 =3D=3D filter type code "none"
data =3D @data.map { |row| "\0" < row.map { |p| p.values.pack("C*")=
=

}.join }
blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.join, 9))
blob << PNG.chunk('IEND', '')
blob.join
end

$ time ruby -Ilib profile.rb

real 0m10.190s
user 0m10.081s
sys 0m0.043s


Using String#% instead of Array#pack:

format_str =3D "%c%c%c%c"
data =3D @data.map { |row| "\0" < row.map { |p| format_str % p.valu=
es =

}.join }

instead of

data =3D @data.map { |row| "\0" < row.map { |p| p.values.pack("C*")=
=

}.join }

$ time ruby -Ilib profile.rb

real 0m4.974s
user 0m4.911s
sys 0m0.031s


Caching the string representation of the values in PNG::Color (because =

each pixel is the same instance of color in this case):

Add to PNG::Color

def values_str
@values_str ||=3D "%c%c%c%c" % @values
end

Use

data =3D @data.map { |row| "\0" < row.map { |p| p.values_str }.join=
}

instead of

format_str =3D "%c%c%c%c"
data =3D @data.map { |row| "\0" < row.map { |p| format_str % p.valu=
es =

}.join }

$ time ruby -Ilib profile.rb

real 0m2.489s
user 0m2.463s
sys 0m0.013s


Improving PNG::Canvas#initialize:

Use

@data =3D Array.new(@width) { |x| Array.new(@height, background) }

instead of

@data =3D Array.new(@width) { |x| Array.new(@height) { background }=
}

$ time ruby -Ilib profile.rb

real 0m1.941s
user 0m1.914s
sys 0m0.014s


Representing the values in PNG::Color as String (instead of as Array) (s=
ee =

complete patch below):

$ time ruby -Ilib profile.rb

real 0m1.492s
user 0m1.445s
sys 0m0.015s


So, it is ten times faster in pure Ruby.

Dominik


--- png-1.0.0/lib/png.rb 2006-08-31 22:57:13.000000000 +0200
+++ png-1.0.0_opt/lib/png.rb 2006-09-02 19:26:46.000000000 +0200
@@ -71,15 +71,21 @@
##
# Writes the PNG to +path+.

- def save(path)
- File.open(path, "w") do |f|
- f.write [137, 80, 78, 71, 13, 10, 26, 10].pack("C*") # PNG signat=
ure
- f.write PNG.chunk('IHDR',
+ def to_blob
+ blob =3D []
+ blob << "\211PNG\r\n\032\n" # PNG signature
+ blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 =

].pack("N2C5"))
# 0 =3D=3D filter type code "none"
- data =3D @data.map { |row| [0] + row.map { |p| p.values } }.flatt=
en
- f.write PNG.chunk('IDAT', Zlib::Deflate.deflate(data.pack("C*"), =
9))
- f.write PNG.chunk('IEND', '')
+ data =3D @data.map { |row| "\0" < row.map { |p| p.values }.join }
+ blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.join, 9))
+ blob << PNG.chunk('IEND', '')
+ blob.join
+ end
+
+ def save(path)
+ File.open(path, "w") do |f|
+ f.write to_blob
end
end

@@ -94,7 +100,7 @@
# Creates a new color with values +red+, +green+, +blue+, and +alp=
ha+.

def initialize(red, green, blue, alpha)
- @values =3D [red, green, blue, alpha]
+ @values =3D "%c%c%c%c" % [red, green, blue, alpha]
end

##
@@ -151,7 +157,7 @@
end

def inspect # :nodoc:
- "#<%s %02x %02x %02x %02x>" % [self.class, *@values]
+ "#<%s %02x %02x %02x %02x>" % [self.class, r, b, g, a]
end

end
@@ -179,7 +185,7 @@
def initialize(height, width, background =3D Color::White)
@height =3D height
@width =3D width
- @data =3D Array.new(@width) { |x| Array.new(@height) { background=
} }
+ @data =3D Array.new(@width) { |x| Array.new(@height, background) =
}
end

##
 
E

Eric Hodel

I wrote an article on using RubyInline for optimization where I
take png.rb, sprinkle in a little profiling and a little C and
make it go over 100 times faster.

Nice article, but in this case it is possible to get almost the
same speedup in pure Ruby:

$ time ruby -Ilib profile.rb

real 0m15.504s
user 0m15.119s
sys 0m0.309s


Avoiding flatten (and using a literal for the signature):

def to_blob
blob = []
blob << "\211PNG\r\n\032\n" # PNG signature
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack
("N2C5"))
# 0 == filter type code "none"
data = @data.map { |row| "\0" < row.map { |p| p.values.pack
("C*") }.join }
blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.join, 9))
blob << PNG.chunk('IEND', '')
blob.join
end

This change gives a broken PNG. You meant to call #<<, not #<.

I see no significant speedup when using #<<.
[values array for string changes]

These are good.
Improving PNG::Canvas#initialize:

Use

@data = Array.new(@width) { |x| Array.new(@height, background) }

instead of

@data = Array.new(@width) { |x| Array.new(@height)
{ background } }

$ time ruby -Ilib profile.rb

real 0m1.941s
user 0m1.914s
sys 0m0.014s

Not really worth optimizing, since the improvement is so small. In
the optimized RubyInline version only 20% of the time is spent here
with no image generation. Adding image generation makes this
optimization insignificant.

Using your changes from the pure-ruby version I went from about 30
seconds to about 4 seconds, or 7.5 times faster.
 
D

Dominik Bathon

=
=
=
Nice article, but in this case it is possible to get almost the same =
=
speedup in pure Ruby:

$ time ruby -Ilib profile.rb

real 0m15.504s
user 0m15.119s
sys 0m0.309s


Avoiding flatten (and using a literal for the signature):

def to_blob
blob =3D []
blob << "\211PNG\r\n\032\n" # PNG signature
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack =
("N2C5"))
# 0 =3D=3D filter type code "none"
data =3D @data.map { |row| "\0" < row.map { |p| p.values.pack("C*=
") =
This change gives a broken PNG. You meant to call #<<, not #<.

Oops, of course I meant #<<. I actually had tested that my changes still=
=

produce correct results when I did the optimizations originally. Then I =
=

recreated them to get the times for each step and made that typo, oh wel=
l.

Just for reference, the time for the final version with #< changed to #<=
< =

on my machine:

$ time ruby -Ilib profile.rb

real 0m1.901s
user 0m1.841s
sys 0m0.032s
I see no significant speedup when using #<<.
[values array for string changes]

These are good.
Improving PNG::Canvas#initialize:

Use

@data =3D Array.new(@width) { |x| Array.new(@height, background) = }

instead of

@data =3D Array.new(@width) { |x| Array.new(@height) { background= } }

$ time ruby -Ilib profile.rb

real 0m1.941s
user 0m1.914s
sys 0m0.014s

Not really worth optimizing, since the improvement is so small. In th=
e =
optimized RubyInline version only 20% of the time is spent here with n=
o =
image generation. Adding image generation makes this optimization =
insignificant.

Yes, it's not much for the total runtime, but it makes this one line abo=
ut =

14 times faster:

$ time ruby -e "40.times { Array.new(400) { Array.new(400) { 0 } } }"

real 0m2.430s
user 0m2.402s
sys 0m0.017s

$ time ruby -e "40.times { Array.new(400) { Array.new(400, 0) } }"

real 0m0.165s
user 0m0.147s
sys 0m0.016s
 
M

Mike Berrow

Eric said:
I wrote an article on using RubyInline for optimization where I take
png.rb, sprinkle in a little profiling and a little C and make it go
over 100 times faster.

http://segment7.net/projects/ruby/inline_optimization.html

Hi, I installed the gems but I immediately run into this:

C:\Inline\png-1.0.0>ruby -Ilib example/profile.rb
example/profile.rb:8:in `draw': undefined method `to_blob' for
#<PNG:0x28c7b28>
(NoMethodError)
from example/profile.rb:15
from example/profile.rb:15

... on Windows.

I get the same thing in cygwin:

$ time ruby -Ilib example/profile.rb
example/profile.rb:8:in `draw': undefined method `to_blob' for
#<PNG:0x28c7b28>
(NoMethodError)
from example/profile.rb:15
from example/profile.rb:15
real 0m2.361s
user 0m0.015s
sys 0m0.046s

The rdoc does not show a a 'to_blob' method for PNG

Any idea?

-- Mike Berrow
 
E

Eric Hodel

Hi, I installed the gems but I immediately run into this:

C:\Inline\png-1.0.0>ruby -Ilib example/profile.rb
example/profile.rb:8:in `draw': undefined method `to_blob' for
#<PNG:0x28c7b28>
(NoMethodError)

Any idea?

From the article:
== Play along at home!

If you want to play along at home, download and unpack png-1.0.0.tgz
and save the profile benchmark code into example/.

I also added PNG#to_blob to eliminate the need to write the file to
disk, it looks almost exactly like PNG#save:

def to_blob
blob = []
blob << [137, 80, 78, 71, 13, 10, 26, 10].pack("C*") # PNG signature
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack
("N2C5"))
# 0 == filter type code "none"
data = @data.map { |row| [0] + row.map { |p| p.values } }.flatten
blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.pack("C*"), 9))
blob << PNG.chunk('IEND', '')
blob.join
end
I added my RubyInline extensions in png.rb and modified png.rb as
appropriate, so while you play along you should do that too.
 
M

Mike Berrow

Eric said:
== Play along at home!
....

Thanks,
but it seems I am missing some setup needed for RubyInline itself

I tried running the hello.rb in
C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo

and got this ...

C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo>ruby hello.rb
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:64:in
`test': can't
convert nil into String (TypeError)
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:64:in `
rootdir'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:73:in `
directory'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:253:in
`so_name'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:287:in
`load_cache'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:601:in
`inline'
from hello.rb:7

In Cygwin I get ...

$ ruby hello.rb
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in ``':
No such
file or directory - cl -nologo -LD -MD -Zi -O2b2xg- -G6 -I
c:/ruby/lib/ruby/1.8
/i386-mswin32 -o C:\cygwin\home\User1/.ruby_inline/Inline_Hello_5d41.so
C:
\cygwin\home\User1/.ruby_inline/Inline_Hello_5d41.c -link
/INCREMENTAL:no
/EXPORT:Init_Inline_Hello_5d41 (Errno::ENOENT)
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in
`build'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:602:in
`inline'
from hello.rb:7

I know I have MingW32 on here since I installed Dev-C++ and have that
running.
At C:\Dev-Cpp\bin there is a gcc.exe

Is it it just a matter of hooking it up this right way in RubyInline ?

-- Mike Berrow
 
E

Eric Hodel

Thanks,
but it seems I am missing some setup needed for RubyInline itself

I tried running the hello.rb in
C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo

and got this ...

C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo>ruby hello.rb
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:64:in
`test': can't
convert nil into String (TypeError)
from

You need to set INLINEDIR or HOME in your environment. This will be
fixed in the next release.
In Cygwin I get ...

$ ruby hello.rb
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in
``':
No such
file or directory - cl -nologo -LD -MD -Zi -O2b2xg- -G6 -I
c:/ruby/lib/ruby/1.8
/i386-mswin32 -o C:\cygwin\home\User1/.ruby_inline/
Inline_Hello_5d41.so
C:
\cygwin\home\User1/.ruby_inline/Inline_Hello_5d41.c -link
/INCREMENTAL:no
/EXPORT:Init_Inline_Hello_5d41 (Errno::ENOENT)
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in
`build'

You need the MS C compiler.
I know I have MingW32 on here since I installed Dev-C++ and have that
running.
At C:\Dev-Cpp\bin there is a gcc.exe

Your ruby wasn't compiled with GCC. If you want inline to use GCC,
you need to compile ruby with it.
Is it it just a matter of hooking it up this right way in RubyInline ?

RubyInline uses whatever C compiler Ruby was built with. If you
don't have the same compiler installed inline won't be able to do its
thing. For windows using the one-click installer you need to either
install the same compiler or build ruby from scratch with a GCC.
 
A

ara.t.howard

Your ruby wasn't compiled with GCC. If you want inline to use GCC, you need
to compile ruby with it.


RubyInline uses whatever C compiler Ruby was built with. If you don't have
the same compiler installed inline won't be able to do its thing. For
windows using the one-click installer you need to either install the same
compiler or build ruby from scratch with a GCC.

or campaign for an msys based ruby dist which would make this all go away ;-)

-a
 
R

Ryan Davis

or campaign for an msys based ruby dist which would make this all
go away ;-)

That would be ideal, but seems unlikely. Is there any progress on
this front?
 
J

Joel VanderWerf

Ryan said:
On Sep 11, 2006, at 10:22 AM, (e-mail address removed) wrote: ...

That would be ideal, but seems unlikely. Is there any progress on this
front?

I wonder how hard it would be to adapt the OCI build process (rakefiles
and other scripts) to generate both mingw and mswin based installers?
I'm sure it's much more than s/nmake/make/ and so on, but maybe it's
feasible.

Not to suggest that Curt should take this on, but there seems to be a
critical mass of interested folks here who would really like to have an
windows OCI that contains a gnu toolchain and can load gcc-built extensions.

(I'm doing a svn co to see what goes into the OCI, and to find out how
naive I am....)
 
A

ara.t.howard

That would be ideal, but seems unlikely. Is there any progress on this front?

maybe we should just make one? if it were me, i'd probably just compile
everything, zip it, and let people unpack it. a bat script to setup the
environment, associactions, etc. might be nice, but i'm a fan of keeping
stuff out of system space and letting people just point their env at it...

what do you think? basically i'm thinking

msys-ruby-1.0.0.tgz

(unpack)

cd c:\msys-ruby-1.0.0\

setup.bat # sets up env vars, file assoc, etc.


setup.bat, for that matter, need only configure %PATH and then spawn setup.rb.

thoughts?

-a
 
T

Tim Pease

maybe we should just make one? if it were me, i'd probably just compile
everything, zip it, and let people unpack it. a bat script to setup the
environment, associactions, etc. might be nice, but i'm a fan of keeping
stuff out of system space and letting people just point their env at it...

what do you think? basically i'm thinking

msys-ruby-1.0.0.tgz

(unpack)

cd c:\msys-ruby-1.0.0\

setup.bat # sets up env vars, file assoc, etc.


setup.bat, for that matter, need only configure %PATH and then spawn setup.rb.

thoughts?

The command line is fine for a Ruby developer. For an "average" end
user of some Ruby program, a one-click installer is essential on the
Windows platform.

Along those same lines, you'll also want to automagically install gem,
win32 stuff, etc. -- all the good things on the windows platform that
*NIX does not need (except for gem -- that's always good stuff).

TwP
 
A

Austin Ziegler

or campaign for an msys based ruby dist which would make this all go away ;-)
...and introduce three dozen other problems that aren't worth using MSYS for.

-austin
 
A

Austin Ziegler

That would be ideal, but seems unlikely. Is there any progress on
this front?

No. I have, unfortunately, been busy with wedding planning and have
not yet had time to pull together a promised email. Soon, I hope. The
invitations are sent.

-austin
 
A

ara.t.howard

...and introduce three dozen other problems that aren't worth using MSYS for.

i'll confess that i have no idea if your are right or not and that chances are
very good that you are - still, it seems like it may be a worthwhile
experiment.

cheers.

-a
 
C

Curt Hibbs

i'll confess that i have no idea if your are right or not and that chances are
very good that you are - still, it seems like it may be a worthwhile
experiment.

cheers.

-a

No matter what path is taken, someone will still be unhappy. My goal
is to make things as simple as possible for 80% of the Windows users.

Curt
 
T

Tom Allison

Tim said:
The command line is fine for a Ruby developer. For an "average" end
user of some Ruby program, a one-click installer is essential on the
Windows platform.

Along those same lines, you'll also want to automagically install gem,
win32 stuff, etc. -- all the good things on the windows platform that
*NIX does not need (except for gem -- that's always good stuff).

TwP

The "average" user might not care about InLine Ruby as much as you do...
I have a lot more to learn before I need InLine and by then it might be worth
DIY to get there.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,009
Latest member
GidgetGamb

Latest Threads

Top