I am new to Ruby and I could use some expert advice as to how I can make this code run faster.

[Ruby Maniac]

I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.

def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end

About 2.8 times as fast.

def scramble( fname )
_fname = fname + ".scrambled"
mask = 0x8080808080808080
data = File.open( fname, "rb" ){|f| f.read }
pad = (8 - (data.size % 8)) % 8
data << "p" * pad
File.open( _fname, "wb"){|ff|
array = data.unpack('Q*')
array.map!{|x| x | mask }
ff.write( array.pack('Q*')[0 ... -pad] )
}
end- Hide quoted text -

- Show quoted text -

Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).

Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.

 

[Bill Kelly]

Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).

Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.

I'm only just now reading about Psyco. But the speed gains
described on the introduction page[1] seem about what one would
expect if similar techniques were used with Ruby:

"For common code, expect at least a 2x speed-up, more typically 4x.
But where Psyco shines is when running algorithmical code ..."

"The difference with the traditional approach to JIT compilers is
that Psyco writes several version of the same blocks (a block is a
bit of a function), which are optimized by being specialized to
some kinds of variables (a "kind" can mean a type, but it is more
general)."

So it could compile specialized versions of (portions of) a method
that would be called if the arguments are Floats or Fixnums.

I would imagine such a technique could work with Ruby, provided
one agrees to disallow various Numeric methods from being
redefined.

For example, Ruby allows things like this:

class Fixnum
alias_method ld_times, :*
def *(other)
warn("i'm thinking: #{self} * #{other}")
old_times(other)
end
end
i'm thinking: 6 * 7
=> 42


I'd suppose if one is willing to give up the ability to modify
a certain set of Fixnum, Float, and perhaps String methods, one
could make the equivalent of Psyco for Ruby.

Indeed, there are a couple of already existing projects in
this general category:

http://rubyforge.org/projects/ruby2c/
http://rubyforge.org/projects/ruby2cext/



[1] http://psyco.sourceforge.net/introduction.html


Regards,

Bill

 

[Mohit Sindhwani]

I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.

def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end

About 2.8 times as fast.

def scramble( fname )
_fname = fname + ".scrambled"
mask = 0x8080808080808080
data = File.open( fname, "rb" ){|f| f.read }
pad = (8 - (data.size % 8)) % 8
data << "p" * pad
File.open( _fname, "wb"){|ff|
array = data.unpack('Q*')
array.map!{|x| x | mask }
ff.write( array.pack('Q*')[0 ... -pad] )
}
end- Hide quoted text -

- Show quoted text -

Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).

Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.

I'm still confused about why someone wouldn't just have a C extension
for a Ruby module instead of going through all this claimed hard work.

We know the facts: a VM solution is better than an interpreted language;
a compiled solution is better than a VM.

Yes, we get it - Python, with the effort you put in, is faster. But,
it's simply a case of matching problems to languages. I don't think
anyone on this group is saying that Ruby is the answer for everything.
There are known issues and limitations to Ruby - some of these will be
resolved and some will not.

As I mentioned before (I think in this thread), Ruby serves its purpose
in many situations. You would not write the firmware of a mobile phone
in Ruby - nor would you in Python - nor would you in Java. You would
most likely write it in C/C++ and where needed, a bit of assembly.

We can discuss this endlessly. Why not write our web applications in
assembly? It is shown that Java takes more resources than C++ and I saw
a paper that showed that on average, C++ /may/ be quite a bit slower
than C and people claim assembly is about 10x faster than C in extreme
cases. So, let's just write everything in assembly language and get a
speedup of 2000 times or some absurd number like that.

Match languages to the problems and the resources. You can, with a C
extension, easily get the performance you talk about. The point is that
for anything complex, it is very likely that Ruby will let you get to
the point where you know where the bottlenecks are much quicker.

Cheers,
Mohit.
9/24/2007 | 11:08 AM.

 

[Ruby Maniac]

From: "Ruby Maniac" <[email protected]>

Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).

Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.

I'm only just now reading about Psyco. But the speed gains
described on the introduction page[1] seem about what one would
expect if similar techniques were used with Ruby:

"For common code, expect at least a 2x speed-up, more typically 4x.
But where Psyco shines is when running algorithmical code ..."

"The difference with the traditional approach to JIT compilers is
that Psyco writes several version of the same blocks (a block is a
bit of a function), which are optimized by being specialized to
some kinds of variables (a "kind" can mean a type, but it is more
general)."

So it could compile specialized versions of (portions of) a method
that would be called if the arguments are Floats or Fixnums.

I would imagine such a technique could work with Ruby, provided
one agrees to disallow various Numeric methods from being
redefined.

For example, Ruby allows things like this:

class Fixnum
alias_method ld_times, :*
def *(other)
warn("i'm thinking: #{self} * #{other}")
old_times(other)
end
end

i'm thinking: 6 * 7
=> 42

I'd suppose if one is willing to give up the ability to modify
a certain set of Fixnum, Float, and perhaps String methods, one
could make the equivalent of Psyco for Ruby.

Indeed, there are a couple of already existing projects in
this general category:

http://rubyforge.org/projects/ruby2c/
http://rubyforge.org/projects/ruby2cext/

[1]http://psyco.sourceforge.net/introduction.html

Regards,

Bill

ruby2c and ruby2cext are hacks that convert Ruby code into C code.

Pysco is a JIT that works transparently at runtime.

If you want to code C/C++ then code C/C++ and be done with it.

If you want to code Ruby then live with the benefits or detriments of
using Ruby including the less than stellar performance.

I have coded both Ruby and Python and for me some aspects of Ruby code
is bass-ackwards and less than intuitive even though Ruby is supposed
to be an easy to understand langauge that is intuitive, blah-blah-
blah. For me Python seems far more intuitive than Ruby could ever
be.

Maybe I am just not cool enough to think Ruby is god's gift to
programming but when I write code I want nothing less than performance
relative to the amount of time I wish to spend writing code.

If I wanted blazing performance I would write Aseembly Language (been
there and done that back in the day when 8 bits was all there was) but
one must commit a lot of time to hand-crafting Assembly code and most
of the time I just don't have that much time to spend writing code.

Once upon a time I used to code everything in C but that was well
before C++ existed as anything other than an idea. And 'yes' I was
coding some rather intense C code, back in the day, using C++ like
constructs.

Python uses some of the same syntax as Ruby. Python is dynamic like
Ruby. Python source can be compiled into byte-codes and rolled-up
into an .EXE for easy delivery to end users. I can code Windows
services usign Python if I wanted to but I cannot code Windows
services using Ruby whether I wanted to or not.

When I write code that I wish to distribute I would rather not have to
ship source code. I realize shipping source code is supposed to be
"cool" but I think it is more "cool" to withhold my source code so my
customers might be more tempted to buy my object-code than hack-up my
source code. Also I think it is more "cool" to cause those who wish
to reverse engineer my code have to spend more time doing so rather
than simply reading the source code such as is the case with Ruby.

Also if someone could code a JIT for Ruby like Psyco then someone
would have done this already and we wouldn't have to talk about ruby2c
or ruby2ext.

The bottom line is this: "Ruby coders will always have to spend more
time making their Ruby code run faster than Python coders will have to
spend making their Python code run faster and Python code will always
run faster than Ruby code even after YARV has been released to save
the day until such time as someone codes a JIT for Ruby." <-- Wake me
up when this statement proves to be false because I will be happy when
the day comes when Ruby code runs faster than Python code and I don't
have to ship Ruby source code.

 

[Konrad Meyer]

--nextPart2117252.tOTnPM945E
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Quoth Ruby Maniac:

From: "Ruby Maniac" <[email protected]>

Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).

Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.

I'm only just now reading about Psyco. But the speed gains
described on the introduction page[1] seem about what one would
expect if similar techniques were used with Ruby:

"For common code, expect at least a 2x speed-up, more typically 4x.
But where Psyco shines is when running algorithmical code ..."

"The difference with the traditional approach to JIT compilers is
that Psyco writes several version of the same blocks (a block is a
bit of a function), which are optimized by being specialized to
some kinds of variables (a "kind" can mean a type, but it is more
general)."

So it could compile specialized versions of (portions of) a method
that would be called if the arguments are Floats or Fixnums.

I would imagine such a technique could work with Ruby, provided
one agrees to disallow various Numeric methods from being
redefined.

For example, Ruby allows things like this:

class Fixnum
alias_method ld_times, :*
def *(other)
warn("i'm thinking: #{self} * #{other}")
old_times(other)
end
end

x =3D 6; y =3D 7
x * y

i'm thinking: 6 * 7
=3D> 42

I'd suppose if one is willing to give up the ability to modify
a certain set of Fixnum, Float, and perhaps String methods, one
could make the equivalent of Psyco for Ruby. =20

Indeed, there are a couple of already existing projects in
this general category:

http://rubyforge.org/projects/ruby2c/
http://rubyforge.org/projects/ruby2cext/

[1]http://psyco.sourceforge.net/introduction.html

Regards,

Bill

=20
ruby2c and ruby2cext are hacks that convert Ruby code into C code.
=20
Pysco is a JIT that works transparently at runtime.
=20
If you want to code C/C++ then code C/C++ and be done with it.
=20
If you want to code Ruby then live with the benefits or detriments of
using Ruby including the less than stellar performance.
=20
I have coded both Ruby and Python and for me some aspects of Ruby code
is bass-ackwards and less than intuitive even though Ruby is supposed
to be an easy to understand langauge that is intuitive, blah-blah-
blah. For me Python seems far more intuitive than Ruby could ever
be.
=20
Maybe I am just not cool enough to think Ruby is god's gift to
programming but when I write code I want nothing less than performance
relative to the amount of time I wish to spend writing code.
=20
If I wanted blazing performance I would write Aseembly Language (been
there and done that back in the day when 8 bits was all there was) but
one must commit a lot of time to hand-crafting Assembly code and most
of the time I just don't have that much time to spend writing code.
=20
Once upon a time I used to code everything in C but that was well
before C++ existed as anything other than an idea. And 'yes' I was
coding some rather intense C code, back in the day, using C++ like
constructs.
=20
Python uses some of the same syntax as Ruby. Python is dynamic like
Ruby. Python source can be compiled into byte-codes and rolled-up
into an .EXE for easy delivery to end users. I can code Windows
services usign Python if I wanted to but I cannot code Windows
services using Ruby whether I wanted to or not.
=20
When I write code that I wish to distribute I would rather not have to
ship source code. I realize shipping source code is supposed to be
"cool" but I think it is more "cool" to withhold my source code so my
customers might be more tempted to buy my object-code than hack-up my
source code. Also I think it is more "cool" to cause those who wish
to reverse engineer my code have to spend more time doing so rather
than simply reading the source code such as is the case with Ruby.
=20
Also if someone could code a JIT for Ruby like Psyco then someone
would have done this already and we wouldn't have to talk about ruby2c
or ruby2ext.
=20
The bottom line is this: "Ruby coders will always have to spend more
time making their Ruby code run faster than Python coders will have to
spend making their Python code run faster and Python code will always
run faster than Ruby code even after YARV has been released to save
the day until such time as someone codes a JIT for Ruby." <-- Wake me
up when this statement proves to be false because I will be happy when
the day comes when Ruby code runs faster than Python code and I don't
have to ship Ruby source code.

Yup, a troll.

Someone go get the troll-repellent charms.
=2D-=20
Konrad Meyer <[email protected]> http://konrad.sobertillnoon.com/

--nextPart2117252.tOTnPM945E
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQBG9zjyCHB0oCiR2cwRAnGOAJ4pF467vbmRbahxAGpCK8qupAmXJgCffyCu
YEhR3mcj6leoLFes6dqINBU=
=deee
-----END PGP SIGNATURE-----

--nextPart2117252.tOTnPM945E--

 

[Ruby Maniac]

If you actually want speed, use a compiled language.
This FreeBasic code is about 160 times as fast and
demonstrates that using Python for any reason
whatsoever is unjustified.

Use Ruby when ease of programming is paramount;
use something like FreeBasic or LuaJIT when speed
is paramount. Never submit to Python perversion.

sub scramble( filename as string )
dim outname as string
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byte

outname = filename & ".scrambled"
in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open outname for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close
end sub- Hide quoted text -

- Show quoted text -

Okay I downloaded Free BASIC and installed it but your code does not
work !

On the one hand, FreeBASIC is certainly worth looking into but on the
other hand - there ain't no debugger than I know of unless you know of
a debugger for FreeBASIC.

 

[Bill Kelly]

I have coded both Ruby and Python and for me some aspects of Ruby code
is bass-ackwards and less than intuitive even though Ruby is supposed
to be an easy to understand langauge that is intuitive, blah-blah-
blah. For me Python seems far more intuitive than Ruby could ever
be.

I must be missing the point of this discussion then?

If you like Python, find a job programming Python.

Problem solved?


Regards,

Bill

 

[William James]

Okay I downloaded Free BASIC and installed it but your code does not
work !

On the one hand, FreeBASIC is certainly worth looking into but on the
other hand - there ain't no debugger than I know of unless you know of
a debugger for FreeBASIC.

Did it compile without errors? Did you make sure that no lines of the
source were broken by your newsgroup program? (If using google,
click on "More options" and "Show original".) I didn't include code
to invoke the subroutine; I left that up to you.

Debuggers are crutches; I never use them.

Here's a complete program. Change the filename that is read by
the program.


function scramble( filename as string ) as double
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byte

in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open filename & ".scrambled" for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close '-- Caution. Closes all open files.
return 0.0
end function

print "! ";
print -timer + scramble("testjunk.csv") + timer

 

[Ilmari Heikkinen]

Hi,

At Fri, 21 Sep 2007 07:05:10 +0900,
Ruby Maniac wrote in [ruby-talk:270110]:

def scramble(fname)
_fname = fname + ".scrambled"

File.new(fname, "rb") do |f|
File.new(_fname, "wb+") do |ff|
ff.write(f.read.tr("\0-\177", "\200-\377"))
end
end

end

And the benchmark results for the above code on a 6.7M file:

$ time ruby -rscramble -e 'scramble_original("trace")'

real 0m11.626s
user 0m9.781s
sys 0m1.844s

$ time ruby -rscramble -e 'scramble_nobu("trace")'

real 0m0.099s
user 0m0.044s
sys 0m0.032s

116x speedup and 7 less loc

$ cat scramble.rb
def scramble_original(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end

def scramble_nobu(fname)
File.open(fname, 'rb'){|in_f|
File.open(fname + '.scrambled', 'wb'){|out_f|
out_f.write(in_f.read.tr("\0-\177", "\200-\377"))
}
}
end

 

[Ruby Maniac]

If you actually want speed, use a compiled language.
This FreeBasic code is about 160 times as fast and
demonstrates that using Python for any reason
whatsoever is unjustified.

Use Ruby when ease of programming is paramount;
use something like FreeBasic or LuaJIT when speed
is paramount. Never submit to Python perversion.

sub scramble( filename as string )
dim outname as string
dim as integer in_handle, out_handle, i, size
dim bytes(1 to 4096) as byte

outname = filename & ".scrambled"
in_handle = freefile
open filename for binary access read as in_handle
out_handle = freefile
open outname for binary access write as out_handle
while not eof( in_handle )
size = seek( in_handle )
get #in_handle, , bytes()
size = seek( in_handle ) - size
for i = 1 to size
bytes(i) = bytes(i) or &h80
next
put #out_handle, , bytes(1), size
wend
close
end sub- Hide quoted text -

- Show quoted text -

Finally got this FreeBASIC code to work but it reports a runtime of 3
seconds on a machine that reports 0.811 seconds for Python using Psyco
so this means FreeBASIC is not anywhere near as fast as Python
considering the Python code reads the whole file and then performs a
single String Translate and then write the whole file in 3 operations.

 

[Ruby Maniac]

Quoth Ruby Maniac:

From: "Ruby Maniac" <[email protected]>
Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).
Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.
I'm only just now reading about Psyco. But the speed gains
described on the introduction page[1] seem about what one would
expect if similar techniques were used with Ruby:
"For common code, expect at least a 2x speed-up, more typically 4x.
But where Psyco shines is when running algorithmical code ..."
"The difference with the traditional approach to JIT compilers is
that Psyco writes several version of the same blocks (a block is a
bit of a function), which are optimized by being specialized to
some kinds of variables (a "kind" can mean a type, but it is more
general)."
So it could compile specialized versions of (portions of) a method
that would be called if the arguments are Floats or Fixnums.
I would imagine such a technique could work with Ruby, provided
one agrees to disallow various Numeric methods from being
redefined.
For example, Ruby allows things like this:
class Fixnum
alias_method ld_times, :*
def *(other)
warn("i'm thinking: #{self} * #{other}")
old_times(other)
end
end
x = 6; y = 7
x * y
i'm thinking: 6 * 7
=> 42
I'd suppose if one is willing to give up the ability to modify
a certain set of Fixnum, Float, and perhaps String methods, one
could make the equivalent of Psyco for Ruby.
Indeed, there are a couple of already existing projects in
this general category:
http://rubyforge.org/projects/ruby2c/
http://rubyforge.org/projects/ruby2cext/
[1]http://psyco.sourceforge.net/introduction.html
Regards,
Bill

ruby2c and ruby2cext are hacks that convert Ruby code into C code.

Pysco is a JIT that works transparently at runtime.

If you want to code C/C++ then code C/C++ and be done with it.

If you want to code Ruby then live with the benefits or detriments of
using Ruby including the less than stellar performance.

I have coded both Ruby and Python and for me some aspects of Ruby code
is bass-ackwards and less than intuitive even though Ruby is supposed
to be an easy to understand langauge that is intuitive, blah-blah-
blah. For me Python seems far more intuitive than Ruby could ever
be.

Maybe I am just not cool enough to think Ruby is god's gift to
programming but when I write code I want nothing less than performance
relative to the amount of time I wish to spend writing code.

If I wanted blazing performance I would write Aseembly Language (been
there and done that back in the day when 8 bits was all there was) but
one must commit a lot of time to hand-crafting Assembly code and most
of the time I just don't have that much time to spend writing code.

Once upon a time I used to code everything in C but that was well
before C++ existed as anything other than an idea. And 'yes' I was
coding some rather intense C code, back in the day, using C++ like
constructs.

Python uses some of the same syntax as Ruby. Python is dynamic like
Ruby. Python source can be compiled into byte-codes and rolled-up
into an .EXE for easy delivery to end users. I can code Windows
services usign Python if I wanted to but I cannot code Windows
services using Ruby whether I wanted to or not.

When I write code that I wish to distribute I would rather not have to
ship source code. I realize shipping source code is supposed to be
"cool" but I think it is more "cool" to withhold my source code so my
customers might be more tempted to buy my object-code than hack-up my
source code. Also I think it is more "cool" to cause those who wish
to reverse engineer my code have to spend more time doing so rather
than simply reading the source code such as is the case with Ruby.

Also if someone could code a JIT for Ruby like Psyco then someone
would have done this already and we wouldn't have to talk about ruby2c
or ruby2ext.

The bottom line is this: "Ruby coders will always have to spend more
time making their Ruby code run faster than Python coders will have to
spend making their Python code run faster and Python code will always
run faster than Ruby code even after YARV has been released to save
the day until such time as someone codes a JIT for Ruby." <-- Wake me
up when this statement proves to be false because I will be happy when
the day comes when Ruby code runs faster than Python code and I don't
have to ship Ruby source code.

Yup, a troll.

Someone go get the troll-repellent charms.
--
Konrad Meyer <[email protected]>http://konrad.sobertillnoon.com/

signature.asc
1KDownload- Hide quoted text -

- Show quoted text -

It is truly sad that people, like me, are labeled as being "trolls"
whenever we try to interject some reality into a world where people
clearly do not want anything that resembles reality.

Why not just admit the fact that Ruby is slow in spite of the fact
that Ruby has a compelling syntax with compelling features but it is
the slowest langauge aside from JavaScript and once upon a time I
loved JavaScript so much I coded everything using it but JavaScript is
slow and I never made any attempt to hide that fact.

 

[Paul Brannan]

Since you didn't provide python code for us to compare against, I can only
assume that the code to which you refer is from this thread:

http://groups.google.com/group/comp...=st&q=python+scramble&rnum=4#7efd9f93b7834a23

I don't know which version of the code you used, but here's a version
using String#tr which is equivalent to Michael Spencer's version using
translate:

def scramble_tr(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.tr!("\x0-\x7f", "\x80-\xff")
ff.write(l)
end
f.close()
ff.close()
end

Since I'm on a much slower machine, I used a 1MB file:

cout@bean:~/tmp$ ruby test5.rb testfile
user system total real
scramble 10.780000 0.120000 10.900000 ( 11.514982)
_scramble 11.630000 0.050000 11.680000 ( 11.918237)
scramble_tr 0.250000 0.030000 0.280000 ( 0.416945)

If you're going to do a comparison, please make sure you aren't
comparing apples to oranges. The language shootout isn't perfect, but
it does a pretty good job:

http://shootout.alioth.debian.org/

Hope this helps,

Paul

 

[William James]

Finally got this FreeBASIC code to work but it reports a runtime of 3
seconds on a machine that reports 0.811 seconds for Python using Psyco
so this means FreeBASIC is not anywhere near as fast as Python
considering the Python code reads the whole file and then performs a
single String Translate and then write the whole file in 3 operations.

The speed of the disk drive plays a large role here. Make sure that
both programs are reading the file from the same drive and writing to
the
same drive. Also, the disk cache may have an effect. Run the
FreeBasic
program several times in rapid succession; you may find the first run
is a lot
slower than the subsequent runs.

I just tried the program on a file of size 10,332,028 bytes on a
3.19GHz machine. Time: 0.1257647355875013 sec.

FreeBasic isn't as fast as a top-notch C compiler, but it produces
decent machine language.

 

[Ilmari Heikkinen]

Hi,

At Fri, 21 Sep 2007 07:05:10 +0900,
Ruby Maniac wrote in [ruby-talk:270110]:

def scramble(fname)
_fname = fname + ".scrambled"

File.new(fname, "rb") do |f|
File.new(_fname, "wb+") do |ff|
ff.write(f.read.tr("\0-\177", "\200-\377"))
end
end

end

And the benchmark results for the above code on a 6.7M file:

$ time ruby -rscramble -e 'scramble_original("trace")'

real 0m11.626s
user 0m9.781s
sys 0m1.844s

$ time ruby -rscramble -e 'scramble_nobu("trace")'

real 0m0.099s
user 0m0.044s
sys 0m0.032s

And here's a C version:

$ time ./scramble trace trace.scrambled

real 0m0.063s
user 0m0.044s
sys 0m0.020s

$ cat scramble.c # I'm no C wiz so pardon the sucky error handling.
#include <stdlib.h>
#include <stdio.h>

#define BUF_SIZE 4096

int main(int argc, char** argv) {
FILE *in_f, *out_f;
char *buf;
int i, rsz, wsz;

if (argc != 3) {
printf("USAGE: %s INFILE OUTFILE\n", argv[0]);
return 1;
}

buf = (char *)malloc(BUF_SIZE);
if (NULL == buf) {
printf("Failed to allocate buffer\n");
return 8;
}

in_f = fopen(argv[1], "r");
if (NULL == in_f) {
printf("Failed to open input file %s\n", argv[0]);
return 2;
}

out_f = fopen(argv[2], "w");
if (NULL == out_f) {
printf("Failed to open output file %s\n", argv[1]);
fclose(in_f);
return 4;
}

while (0 != (rsz = fread(buf, 1, BUF_SIZE, in_f))) {
for(i=0; i<rsz; i++)
buf |= 0x80;
wsz = fwrite(buf, 1, rsz, out_f);
if (wsz != rsz) {
printf("Failed to write to output file\n");
break;
}
}

fclose(in_f);
fclose(out_f);

}

 

[Joel VanderWerf]

And here's a C version:

To give C its fair due, you might try with

unsigned long *buf;

and

buf |= 0x80808080;

(and the other necessary changes, of course).

 

[Marc Heiler]

"Benchmark it so you can find the bottlenecks."

I believe for _newcomers_, it is much easier and
ultimately better to point them at "better" habits
(insofar as "what is faster"), than tell them to
use benchmarks on their own.

There seems to exist a slight need for these kind of
questions, as they do reappear again and again, probably to
squeeze the max out of ruby's performance.

 

[Ilmari Heikkinen]

And here's a C version:

To give C its fair due, you might try with

unsigned long *buf;

and

buf |= 0x80808080;

(and the other necessary changes, of course).

I tried that a while ago, it was roughly twice as fast as the per-byte version.
But it had a bug with the last % 4 bytes and I wasn't hardcore enough to
bother fixing it.

And a version with 64-bit ints was even faster. But it had a bug too.

Anyone wish to contribute working versions ?

 

[William James]

Finally got this FreeBASIC code to work

Will you explain how you got it to work? The version I posted
works for me. (I don't know how "- Hide quoted text -" got into
the source.) What was wrong with my code?

 

[Ruby Maniac]

Ruby Maniac wrote:
I am new to Ruby and I could use some expert advice as to how I can
make this code run faster.
def scramble(fname)
f = File.new(fname, "rb")
_fname = fname + ".scrambled"
begin
File.exist?(_fname) if File.delete(_fname)
rescue
end
ff = File.new(_fname, "wb+")
for l in f
l.each_byte{|c| ff.write((c | 0x80).chr) }
end
f.close()
ff.close()
end
About 2.8 times as fast.
def scramble( fname )
_fname = fname + ".scrambled"
mask = 0x8080808080808080
data = File.open( fname, "rb" ){|f| f.read }
pad = (8 - (data.size % 8)) % 8
data << "p" * pad
File.open( _fname, "wb"){|ff|
array = data.unpack('Q*')
array.map!{|x| x | mask }
ff.write( array.pack('Q*')[0 ... -pad] )
}
end- Hide quoted text -
- Show quoted text -

Okay but wake me up when you can make the original Ruby code I
submitted at the outset of this post run 120 times faster because that
is the difference between Ruby and optimized Python (when Psyco is
used to optimize the Python).

Whether or not Rubynauts feel like Python is perversion or not it is
amazing easy to achieve machine code speeds from Python as opposed to
the difficulty in getting Ruby code to run even 5x faster once
optimized.

I'm still confused about why someone wouldn't just have a C extension
for a Ruby module instead of going through all this claimed hard work.

We know the facts: a VM solution is better than an interpreted language;
a compiled solution is better than a VM.

Yes, we get it - Python, with the effort you put in, is faster. But,
it's simply a case of matching problems to languages. I don't think
anyone on this group is saying that Ruby is the answer for everything.
There are known issues and limitations to Ruby - some of these will be
resolved and some will not.

As I mentioned before (I think in this thread), Ruby serves its purpose
in many situations. You would not write the firmware of a mobile phone
in Ruby - nor would you in Python - nor would you in Java. You would
most likely write it in C/C++ and where needed, a bit of assembly.

We can discuss this endlessly. Why not write our web applications in
assembly? It is shown that Java takes more resources than C++ and I saw
a paper that showed that on average, C++ /may/ be quite a bit slower
than C and people claim assembly is about 10x faster than C in extreme
cases. So, let's just write everything in assembly language and get a
speedup of 2000 times or some absurd number like that.

Match languages to the problems and the resources. You can, with a C
extension, easily get the performance you talk about. The point is that
for anything complex, it is very likely that Ruby will let you get to
the point where you know where the bottlenecks are much quicker.

Cheers,
Mohit.
9/24/2007 | 11:08 AM.- Hide quoted text -

- Show quoted text -

I know of some people right now who are feverishly working to make
Ruby "the" answer to everything regardless of what the real problem
they are trying to resolve is and they are spending a whole lotta time
doing it too.

 

[Ilmari Heikkinen]

And here's a C version:

To give C its fair due, you might try with

unsigned long *buf;

and

buf |= 0x80808080;

(and the other necessary changes, of course).

And a version with 64-bit ints was even faster. But it had a bug too.

Anyone wish to contribute working versions ?

Guess not, so here's one:

$ time ./scramble64 trace trace.scramblec

real 0m0.022s
user 0m0.004s
sys 0m0.020s

$ export RUBYOPT= # -rubygems takes ~40ms
$ time ruby -rscramble -e 'scramble_nobu("trace")'

real 0m0.049s
user 0m0.016s
sys 0m0.032s

$ diff trace.scramble*
$

$ cat scramble64.c

#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>

#define BUF_SIZE 4096

int main(int argc, char** argv) {
int i, j, k, rsz, wsz;
FILE *in_f, *out_f;
unsigned long long isz;
struct stat st;
long long buf[BUF_SIZE];

if (argc != 3) {
printf("USAGE: %s INFILE OUTFILE\n", argv[0]);
return 1;
}

isz = sizeof(long long);

in_f = fopen(argv[1], "r");
if (NULL == in_f) {
printf("Failed to open input file %s\n", argv[0]);
return 2;
}

out_f = fopen(argv[2], "w");
if (NULL == out_f) {
printf("Failed to open output file %s\n", argv[1]);
fclose(in_f);
return 4;
}

fstat(fileno(in_f), &st);
for(i=0,j = st.st_size / (isz*BUF_SIZE); i<j; i++) {
rsz = fread(buf, isz, BUF_SIZE, in_f);
for(k=0; k<rsz; k++)
buf[k] |= 0x8080808080808080;
wsz = fwrite(buf, isz, rsz, out_f);
if (wsz != rsz) {
printf("Failed to write to output file\n");
break;
}
}
rsz = fread(buf, 1, isz*BUF_SIZE, in_f);
for(k=0; k<rsz; k++)
((char*)buf)[k] |= 0x80;
wsz = fwrite(buf, 1, rsz, out_f);
if (wsz != rsz) {
printf("Failed to write to output file\n");
}

fclose(in_f);
fclose(out_f);
return 0;
}