Enumerators and generators

[Hal Fulton]

[Note: parts of this message were removed to make it a legal post.]

Hi, all...

I'm trying to grasp when/why one might use a generator instead of
an enumerator (or vice versa).

I've used generators, but enumerators are newer to me.

I've heard one person's take on this (thanks David!) but thought
I might ask more thoughts...

Thanks,
Hal Fulton

 

[Ricardo Panaggio]

Hello Hal

Generators "externalize" iteration, whereas Enumerator "internalize"
it (but this docs may have already told you).

Generators can be used to create a "controlled stream" of objects,
that you can operate as you where using a remove control: go to the
next (Generator#next), give me the position (Generator#pos), return to
the beginning (Generator#rewind), ...

Enumerator offers methods to iterate through the entire stream, like
iterate using a windows of size X (Enumerator#each_cons), iterate
using slices from the stream (Enumerator#each_slide), ...

They are quite different in use, although both are usable to iterate over data.

 

[Robert Klemme]

I'm trying to grasp when/why one might use a generator instead of
an enumerator (or vice versa).

It seems the alternative is Generator vs. Enumerable rather than
Generator vs. Enumerator. Enumerator is rather a special form of
Enumerable which allows to do some things more efficiently because it
delays the iteration. E.g.

# traditional one liner style: needs much memory:
matches = File.readlines("foo").select {|line| /keyword/ =~ line}

# Enumerator: only memory for the
matches = File.to_enumforeach, "foo").select {|line| /keyword/ =~ line}

# Same in 1.9 with implicit Enumerator creation
matches = File.foreach("foo").select {|line| /keyword/ =~ line}

It's a convenient replacement for the equally efficient (memory wise):

matches = []

File.foreach "foo" do |line|
matches << line if /keyword/ =~ line
end

Generator is a completely different iteration style as Ricardo explained.

Kind regards

robert

 

[David A. Black]

Hi --

Hello Hal

Generators "externalize" iteration, whereas Enumerator "internalize"
it (but this docs may have already told you).

Generators can be used to create a "controlled stream" of objects,
that you can operate as you where using a remove control: go to the
next (Generator#next), give me the position (Generator#pos), return to
the beginning (Generator#rewind), ...

Enumerators have #next and #rewind too, though (though not #pos). In
1.9, as far as I can tell, the generator.rb library has been removed,
and Generator is now a class inside Enumerator. An Enumerator::Generator
is created automatically, for the use of the enumerator, if you create
an enumerator with a block. Also, these Generators don't have #next and
friends; that's available via the enumerator. (I'm not sure what
happened to #pos.)

As I understand it, the main thing about generators is that "controlled
stream" thing, where you can roll your own sense of iteration, rather
than just piggy-backing on what some enumerable object with its own
ideas about iteration thinks. Here's a (very contrived) 1.9 example:

[dblack@ruby-versions ~]$ cat e.rb
message = nil

g = Enumerator::Generator.new do |yielder|
yielder << "Hi."
puts "I've been told to #{message}."
case message
when "leave"
yielder << "Bye."
when "stay"
yielder << "I'm still here!"
end
end

e = Enumerator.new(g)

puts e.next
message = "stay"
puts e.next

e.rewind

puts e.next
message = "leave"
puts e.next

[dblack@ruby-versions ~]$ ruby e.rb
Hi.
I've been told to stay.
I'm still here!
Hi.
I've been told to leave.
Bye.


David

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

The Ruby training with Black/Brown/McAnally
Compleat Philadelphia, PA, October 1-2, 2010
Rubyist http://www.compleatrubyist.com

 

[Ricardo Panaggio]

Hi --



Enumerators have #next and #rewind too, though (though not #pos). In
1.9, as far as I can tell, the generator.rb library has been removed,

I wasn't aware of it in 1.9. Good to now

and Generator is now a class inside Enumerator. An Enumerator::Generator
is created automatically, for the use of the enumerator, if you create
an enumerator with a block. Also, these Generators don't have #next and
friends; that's available via the enumerator. (I'm not sure what
happened to #pos.)

As I understand it, the main thing about generators is that "controlled
stream" thing, where you can roll your own sense of iteration, rather
than just piggy-backing on what some enumerable object with its own
ideas about iteration thinks. Here's a (very contrived) 1.9 example:

[dblack@ruby-versions ~]$ cat e.rb message =3D nil

g =3D Enumerator::Generator.new do |yielder|
=A0yielder << "Hi."
=A0puts "I've been told to #{message}."
=A0case message
=A0when "leave"
=A0 =A0yielder << "Bye."
=A0when "stay"
=A0 =A0yielder << "I'm still here!"
=A0end
end

e =3D Enumerator.new(g)

puts e.next
message =3D "stay"
puts e.next

e.rewind

puts e.next
message =3D "leave"
puts e.next

[dblack@ruby-versions ~]$ ruby e.rb Hi.
I've been told to stay.
I'm still here!
Hi.
I've been told to leave.
Bye.


David

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

=A0The =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Ruby training with Black/Brown= /McAnally
=A0Compleat =A0 =A0 =A0 =A0 =A0 =A0 =A0Philadelphia, PA, October 1-2, 201= 0
=A0Rubyist =A0 =A0 =A0 =A0 =A0 =A0 =A0 http://www.compleatrubyist.com

 

[Hal Fulton]

[Note: parts of this message were removed to make it a legal post.]

My next questions then are:

1. Does Enumerator::Generator have all the functionality of the
old generator.rb, just (it seems to me) a different usage?

2. In David's example, is "yielder" simply an array of objects? Do
we know or care? (If so, I'd be tempted to call it "list" because
"yielder" reminds of the keyword yield which I suppose is unrelated
here.)

Hal

On Thu, Jul 22, 2010 at 7:59 AM, Ricardo Panaggio <

Hi --



Enumerators have #next and #rewind too, though (though not #pos). In
1.9, as far as I can tell, the generator.rb library has been removed,

I wasn't aware of it in 1.9. Good to now

and Generator is now a class inside Enumerator. An Enumerator::Generator
is created automatically, for the use of the enumerator, if you create
an enumerator with a block. Also, these Generators don't have #next and
friends; that's available via the enumerator. (I'm not sure what
happened to #pos.)

As I understand it, the main thing about generators is that "controlled
stream" thing, where you can roll your own sense of iteration, rather
than just piggy-backing on what some enumerable object with its own
ideas about iteration thinks. Here's a (very contrived) 1.9 example:

[dblack@ruby-versions ~]$ cat e.rb message = nil

g = Enumerator::Generator.new do |yielder|
yielder << "Hi."
puts "I've been told to #{message}."
case message
when "leave"
yielder << "Bye."
when "stay"
yielder << "I'm still here!"
end
end

e = Enumerator.new(g)

puts e.next
message = "stay"
puts e.next

e.rewind

puts e.next
message = "leave"
puts e.next

[dblack@ruby-versions ~]$ ruby e.rb Hi.
I've been told to stay.
I'm still here!
Hi.
I've been told to leave.
Bye.


David

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

The Ruby training with Black/Brown/McAnally
Compleat Philadelphia, PA, October 1-2, 2010
Rubyist http://www.compleatrubyist.com

 

[David A. Black]

Hi --

My next questions then are:

1. Does Enumerator::Generator have all the functionality of the
old generator.rb, just (it seems to me) a different usage?

No; it doesn't have #next or #rewind, but enumerators
(which can easily/transparently wrap generators) do. Neither of them
appears to have #pos, unless it's available in some other way that I
haven't spotted. (I'm writing this in haste and without benefit of irb,
so I'm not digging further right now.)

2. In David's example, is "yielder" simply an array of objects? Do
we know or care? (If so, I'd be tempted to call it "list" because
"yielder" reminds of the keyword yield which I suppose is unrelated
here.)

It's actually an Enumerator::Yielder object, which I think only exists
for the purpose of being the receptacle for the objects to be yielded by
the generator.


David

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

The Ruby training with Black/Brown/McAnally
Compleat Philadelphia, PA, October 1-2, 2010
Rubyist http://www.compleatrubyist.com

 

[Roger Pack]

Hi, all...

I'm trying to grasp when/why one might use a generator instead of
an enumerator (or vice versa).

I've used generators, but enumerators are newer to me.

Basically what was a Generator in 1.8 is now an enumerator in 1.9, and
faster.

http://wiki.github.com/rdp/ruby_tutorials_core/enumerator

Cheers!
-r