SyncEnumerator

S

Sean Russell

I was playing around with the SyncEnumerator from Ruby 1.8, and
noticed that it was taking a long time to iterate over elements.

The current implementation may be the "correct", or elegant, way to
implement it, but it is terribly inefficient. I rewrote the class,
and got the following results:

Construction
Old: 0.159291
New: 0.000352
Starting SyncEnumerator
.................................................................................
26 shold be 26 :: Time: 19.686455
Starting SyncEnumerator_
.................................................................................
26 shold be 26 :: Time: 0.028817

The new version is nearly three orders of magnitude faster for
construction, and is three orders of magnitude faster in execution.

Is there any reason why I shouldn't submit this as an RCR to replace
SyncEnumerator?


<code name='Implementation'>
class SyncEnumerator_
include Enumerable

# Creates a new SyncEnumerator which enumerates rows of given
# Enumerable objects.
def initialize(*enums)
@gens = enums
end

# Returns the number of enumerated Enumerable objects, i.e. the size
# of each row.
def size
@gens.size
end

# Returns the number of enumerated Enumerable objects, i.e. the size
# of each row.
def length
@gens.length
end

# Enumerates rows of the Enumerable objects.
def each
count = 0
maxsize = 0
@gens.each { |g| maxsize = g.size > maxsize ? g.size : maxsize }
until count >= maxsize
yield @gens.map { |g| g[count] }
count += 1
end
self
end
end
</code>


<code name='Benchmark'>
require 'generator'
require 'mysync'

N = 80

a = %w{ a b c d e f g h i j k l m n o p q r s t u v w x y z }
b = a.reverse
c = %w{ 1 2 3 4 5 6 7 8 9 10 11 12 }
d = c.reverse

puts "Construction"
t=Time.now; 100.times { SyncEnumerator.new( a,b,c,d ) }; puts "Old:
#{Time.now-t}"
t=Time.now; 100.times { SyncEnumerator_.new( a,b,c,d ) }; puts "New:
#{Time.now-t}"

for sync in [SyncEnumerator.new( a, b, c, d ),
SyncEnumerator_.new( a, b, c, d )]
puts "Starting #{sync.class.name}"
time = Time.now
count = 0
N.times { sync.each { |x,y| count += 1 } ; print "." ; STDOUT.flush
}
time = Time.now - time
puts "\n#{count / N} shold be #{a.size} :: Time: #{time}"
end

# SyncEnumerator_.new( a, b, c, d ).each { |x,y,z,w| puts "#{x} #{y}
#{z} #{w}" }
</code>


<code name='Unit Test'>
require 'mysync'
require 'test/unit'

class SyncTest < Test::Unit::TestCase
def test_all
a = %w{ a b c d e f g h i j k l m n o p q r s t u v w x y z }
b = %w{ 1 2 3 4 5 6 7 8 9 10 11 12 }
c = [ 1,2,3,4,5,6,7,9,9,10,11,12,13,14,15 ]

x = []
y = []
z = []
sync = SyncEnumerator_.new( a, b, c ).each { |m,n,o|
x << m
y << n
z << o
}
y.compact! # Get rid of end nils
z.compact!
assert_equal( a, x )
assert_equal( b, y )
assert_equal( c, z )
end
end
</code>
 
S

SER

I should have said "I refactored the class", and, mostly, I just
rewrote each(). After re-reading my post, I thought it was a bit
disingenuous to say that I "rewrote" the class.


--- SER
 
F

Florian Gross

Sean said:
I was playing around with the SyncEnumerator from Ruby 1.8, and
noticed that it was taking a long time to iterate over elements.

The current implementation may be the "correct", or elegant, way to
implement it, but it is terribly inefficient. I rewrote the class,
and got the following results:

Construction
Old: 0.159291
New: 0.000352
Starting SyncEnumerator
................................................................................
26 shold be 26 :: Time: 19.686455
Starting SyncEnumerator_
................................................................................
26 shold be 26 :: Time: 0.028817

The new version is nearly three orders of magnitude faster for
construction, and is three orders of magnitude faster in execution.

Is there any reason why I shouldn't submit this as an RCR to replace
SyncEnumerator?

I wonder how an implementation using .zip with a block would compare? Or
would that not do the same thing?
 
G

Graham Foster

Hello Sean,

Is there any reason why I shouldn't submit this as an RCR to replace
SyncEnumerator?
I'm a newbiew .. but your optimisation looks worthwhile in terms of
performance. You didn't get an answer.. have you submitted it to
replace the standard?
 
M

Mark Hubbart

Hi,

I wonder how an implementation using .zip with a block would compare? Or
would that not do the same thing?

Hmm... I don't have the OP's message in my archives...
Assuming it's a transparent change, improving performance without
breaking code, why not just submit it to ruby-core? People are always
posting bug patches there, I suspect that a performance patch wouldn't
be unwelcome.

cheers,
Mark
 
T

Tom Copeland

Assuming it's a transparent change, improving performance without
breaking code, why not just submit it to ruby-core? People are always
posting bug patches there, I suspect that a performance patch wouldn't
be unwelcome.

Or better still, file it here:

http://rubyforge.org/tracker/?atid=1700&group_id=426&func=browse

That'll send an email of the patch summary and description to ruby-core
as well. Also, then you can post updated version of the patch as you
get feedback, and other folks will be able to monitor the patch and get
notified of the updates.

Yours,

tom
 
S

Sean E. Russell

--nextPart157133070.hFDKLvAu90
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Or better still, file it here:

http://rubyforge.org/tracker/?atid=3D1700&group_id=3D426&func=3Dbrowse

That'll send an email of the patch summary and description to ruby-core
as well. Also, then you can post updated version of the patch as you
get feedback, and other folks will be able to monitor the patch and get
notified of the updates.

I've been a member of the Ruby community for I don't know how long now, and=
I=20
*still* haven't figured out when a patch should be submitted to ruby-core,=
=20
and when to RCR. :-/

=2D-=20
### SER =20
### Deutsch|Esperanto|Francaise|Linux|XML|Java|Ruby|Aikido
### http://www.germane-software.com/~ser jabber.com:ser ICQ:83578737=20
### GPG: http://www.germane-software.com/~ser/Security/ser_public.gpg

--nextPart157133070.hFDKLvAu90
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQBBjY4GP0KxygnleI8RAqoUAJwPiqdK6uL4ZtnRvXuv2Cd+bVtGIQCfY5Zd
4sNKTQ8W2nefBuB3FFXlXAM=
=gO1w
-----END PGP SIGNATURE-----

--nextPart157133070.hFDKLvAu90--
 
Y

Yukihiro Matsumoto

Hi,

In message "Re: SyncEnumerator"

|I've been a member of the Ruby community for I don't know how long now, and I
|*still* haven't figured out when a patch should be submitted to ruby-core,
|and when to RCR. :-/

If you think the change is to fix the bug (i.e. difference from the
behavior you assume I (matz) expect), send it to the ruby-core list.
If the change is to make Ruby better in your point of view, submit
RCR, then raise the discussion on the list.

matz.
 
D

David A. Black

Hi --

Hi,

In message "Re: SyncEnumerator"

|I've been a member of the Ruby community for I don't know how long now, and I
|*still* haven't figured out when a patch should be submitted to ruby-core,
|and when to RCR. :-/

If you think the change is to fix the bug (i.e. difference from the
behavior you assume I (matz) expect), send it to the ruby-core list.
If the change is to make Ruby better in your point of view, submit
RCR, then raise the discussion on the list.

I've been a member of the Ruby community for four years (minus roughly
13 hours; my first post to ruby-talk was November 7, 2000, at 21:52
EST :) and one thing I'm not sure of is.....

To what extent should RCR discussion be here, and to what extent
should it be on the message/comment space on RCRchive?


David
 
Y

Yukihiro Matsumoto

Hi,

In message "Re: RCR discussions (was: Re: SyncEnumerator)"

|To what extent should RCR discussion be here, and to what extent
|should it be on the message/comment space on RCRchive?

I prefer discussion on the list, especially for more something than
simple discussion.

* wider range of opinion can be gathered.
* I prefer push model of mail lists to web's pull model.
* summaries are often better organized on the web discussion,
though.

matz.
 
J

Jim Weirich

I've been a member of the Ruby community for four years (minus roughly
13 hours; my first post to ruby-talk was November 7, 2000, at 21:52
EST :)

Wow, how time flies!
and one thing I'm not sure of is.....

To what extent should RCR discussion be here, and to what extent
should it be on the message/comment space on RCRchive?

If you have a glimmer of an RCR, and have done your research (at least googled
the topic on the mail list), the the mailing list is a good place to solicite
feedback. Hopefully the research step will get you beyond the topics that
have been hashed out over and over (and over and over and ... well, you get
the idea).

You can read: http://onestepback.org/index.cgi/Tech/Ruby/WritingRcrs.rdoc for
a longer rant on this topic.
 
D

David A. Black

Hi --

Wow, how time flies!


If you have a glimmer of an RCR, and have done your research (at least googled
the topic on the mail list), the the mailing list is a good place to solicite
feedback. Hopefully the research step will get you beyond the topics that
have been hashed out over and over (and over and over and ... well, you get
the idea).

I would also emphasize the importance of reading the rejected RCRs on
RCRchive, along with the list archives.

I've read it -- actually it's been linked in to RCRchive as required
reading from the beginning :)

It just seems there are different scenarios in practice: submitting at
RCRchive first, then discussing here (which the scenario Matz was
endorsing this morning), discussing first and then submitting (which
happens a lot, even unendorsed :), submitting and using the comment
space on RCRchive, etc. I guess it's partly that discussion in any
forum sometimes happens to move toward discussion of possible language
changes, so in that sense pre-RCR discussion happens spontaneously.
(And it may not matter too much, as long as the basic tenets of
non-rehashing and non-resubmission of rejected ideas are followed.)


David
 
G

gabriele renzi

Yukihiro Matsumoto ha scritto:
n to RCR. :-/
If you think the change is to fix the bug (i.e. difference from the
behavior you assume I (matz) expect), send it to the ruby-core list.
If the change is to make Ruby better in your point of view, submit
RCR, then raise the discussion on the list.

matz.

but /just/ making something faster seems a ruby-core thing, cause it is
better in everyone's point of view (well, except for making sleep() faster)
 
Y

Yukihiro Matsumoto

Hi,

In message "Re: SyncEnumerator"

|but /just/ making something faster seems a ruby-core thing, cause it is
|better in everyone's point of view (well, except for making sleep() faster)

I thing making something faster without changing behavior would do
nothing with RCR from the beginning.

matz.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,708
Messages
2,569,346
Members
44,650
Latest member
LuckyVivo

Latest Threads

Top