Enumerators and generators

Discussion in 'Ruby' started by Hal Fulton, Jul 22, 2010.

  1. Hal Fulton

    Hal Fulton Guest

    [Note: parts of this message were removed to make it a legal post.]

    Hi, all...

    I'm trying to grasp when/why one might use a generator instead of
    an enumerator (or vice versa).

    I've used generators, but enumerators are newer to me.

    I've heard one person's take on this (thanks David!) but thought
    I might ask more thoughts...

    Thanks,
    Hal Fulton
     
    Hal Fulton, Jul 22, 2010
    #1
    1. Advertising

  2. Hello Hal

    Generators "externalize" iteration, whereas Enumerator "internalize"
    it (but this docs may have already told you).

    Generators can be used to create a "controlled stream" of objects,
    that you can operate as you where using a remove control: go to the
    next (Generator#next), give me the position (Generator#pos), return to
    the beginning (Generator#rewind), ...

    Enumerator offers methods to iterate through the entire stream, like
    iterate using a windows of size X (Enumerator#each_cons), iterate
    using slices from the stream (Enumerator#each_slide), ...

    They are quite different in use, although both are usable to iterate over data.

    On Wed, Jul 21, 2010 at 22:12, Hal Fulton <> wrote:
    > Hi, all...
    >
    > I'm trying to grasp when/why one might use a generator instead of
    > an enumerator (or vice versa).
    >
    > I've used generators, but enumerators are newer to me.
    >
    > I've heard one person's take on this (thanks David!) but thought
    > I might ask more thoughts...
    >
    > Thanks,
    > Hal Fulton
    >
     
    Ricardo Panaggio, Jul 22, 2010
    #2
    1. Advertising

  3. 2010/7/22 Hal Fulton <>:
    > I'm trying to grasp when/why one might use a generator instead of
    > an enumerator (or vice versa).


    It seems the alternative is Generator vs. Enumerable rather than
    Generator vs. Enumerator. Enumerator is rather a special form of
    Enumerable which allows to do some things more efficiently because it
    delays the iteration. E.g.

    # traditional one liner style: needs much memory:
    matches = File.readlines("foo").select {|line| /keyword/ =~ line}

    # Enumerator: only memory for the
    matches = File.to_enum:)foreach, "foo").select {|line| /keyword/ =~ line}

    # Same in 1.9 with implicit Enumerator creation
    matches = File.foreach("foo").select {|line| /keyword/ =~ line}

    It's a convenient replacement for the equally efficient (memory wise):

    matches = []

    File.foreach "foo" do |line|
    matches << line if /keyword/ =~ line
    end

    Generator is a completely different iteration style as Ricardo explained.

    Kind regards

    robert

    --
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Jul 22, 2010
    #3
  4. Hi --

    On Thu, 22 Jul 2010, Ricardo Panaggio wrote:

    > Hello Hal
    >
    > Generators "externalize" iteration, whereas Enumerator "internalize"
    > it (but this docs may have already told you).
    >
    > Generators can be used to create a "controlled stream" of objects,
    > that you can operate as you where using a remove control: go to the
    > next (Generator#next), give me the position (Generator#pos), return to
    > the beginning (Generator#rewind), ...


    Enumerators have #next and #rewind too, though (though not #pos). In
    1.9, as far as I can tell, the generator.rb library has been removed,
    and Generator is now a class inside Enumerator. An Enumerator::Generator
    is created automatically, for the use of the enumerator, if you create
    an enumerator with a block. Also, these Generators don't have #next and
    friends; that's available via the enumerator. (I'm not sure what
    happened to #pos.)

    As I understand it, the main thing about generators is that "controlled
    stream" thing, where you can roll your own sense of iteration, rather
    than just piggy-backing on what some enumerable object with its own
    ideas about iteration thinks. Here's a (very contrived) 1.9 example:

    [dblack@ruby-versions ~]$ cat e.rb
    message = nil

    g = Enumerator::Generator.new do |yielder|
    yielder << "Hi."
    puts "I've been told to #{message}."
    case message
    when "leave"
    yielder << "Bye."
    when "stay"
    yielder << "I'm still here!"
    end
    end

    e = Enumerator.new(g)

    puts e.next
    message = "stay"
    puts e.next

    e.rewind

    puts e.next
    message = "leave"
    puts e.next

    [dblack@ruby-versions ~]$ ruby e.rb
    Hi.
    I've been told to stay.
    I'm still here!
    Hi.
    I've been told to leave.
    Bye.


    David

    --
    David A. Black, Senior Developer, Cyrus Innovation Inc.

    The Ruby training with Black/Brown/McAnally
    Compleat Philadelphia, PA, October 1-2, 2010
    Rubyist http://www.compleatrubyist.com
     
    David A. Black, Jul 22, 2010
    #4
  5. On Thu, Jul 22, 2010 at 07:11, David A. Black <> wrote:
    > Hi --
    >
    > On Thu, 22 Jul 2010, Ricardo Panaggio wrote:
    >
    >> Hello Hal
    >>
    >> Generators "externalize" iteration, whereas Enumerator "internalize"
    >> it (but this docs may have already told you).
    >>
    >> Generators can be used to create a "controlled stream" of objects,
    >> that you can operate as you where using a remove control: go to the
    >> next (Generator#next), give me the position (Generator#pos), return to
    >> the beginning (Generator#rewind), ...

    >
    > Enumerators have #next and #rewind too, though (though not #pos). In
    > 1.9, as far as I can tell, the generator.rb library has been removed,


    I wasn't aware of it in 1.9. Good to now :)

    > and Generator is now a class inside Enumerator. An Enumerator::Generator
    > is created automatically, for the use of the enumerator, if you create
    > an enumerator with a block. Also, these Generators don't have #next and
    > friends; that's available via the enumerator. (I'm not sure what
    > happened to #pos.)
    >
    > As I understand it, the main thing about generators is that "controlled
    > stream" thing, where you can roll your own sense of iteration, rather
    > than just piggy-backing on what some enumerable object with its own
    > ideas about iteration thinks. Here's a (very contrived) 1.9 example:
    >
    > [dblack@ruby-versions ~]$ cat e.rb message =3D nil
    >
    > g =3D Enumerator::Generator.new do |yielder|
    > =A0yielder << "Hi."
    > =A0puts "I've been told to #{message}."
    > =A0case message
    > =A0when "leave"
    > =A0 =A0yielder << "Bye."
    > =A0when "stay"
    > =A0 =A0yielder << "I'm still here!"
    > =A0end
    > end
    >
    > e =3D Enumerator.new(g)
    >
    > puts e.next
    > message =3D "stay"
    > puts e.next
    >
    > e.rewind
    >
    > puts e.next
    > message =3D "leave"
    > puts e.next
    >
    > [dblack@ruby-versions ~]$ ruby e.rb Hi.
    > I've been told to stay.
    > I'm still here!
    > Hi.
    > I've been told to leave.
    > Bye.
    >
    >
    > David
    >
    > --
    > David A. Black, Senior Developer, Cyrus Innovation Inc.
    >
    > =A0The =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Ruby training with Black/Brown=

    /McAnally
    > =A0Compleat =A0 =A0 =A0 =A0 =A0 =A0 =A0Philadelphia, PA, October 1-2, 201=

    0
    > =A0Rubyist =A0 =A0 =A0 =A0 =A0 =A0 =A0 http://www.compleatrubyist.com
    >
    >
     
    Ricardo Panaggio, Jul 22, 2010
    #5
  6. Hal Fulton

    Hal Fulton Guest

    [Note: parts of this message were removed to make it a legal post.]

    My next questions then are:

    1. Does Enumerator::Generator have all the functionality of the
    old generator.rb, just (it seems to me) a different usage?

    2. In David's example, is "yielder" simply an array of objects? Do
    we know or care? (If so, I'd be tempted to call it "list" because
    "yielder" reminds of the keyword yield which I suppose is unrelated
    here.)

    Hal

    On Thu, Jul 22, 2010 at 7:59 AM, Ricardo Panaggio <
    > wrote:

    > On Thu, Jul 22, 2010 at 07:11, David A. Black <> wrote:
    > > Hi --
    > >
    > > On Thu, 22 Jul 2010, Ricardo Panaggio wrote:
    > >
    > >> Hello Hal
    > >>
    > >> Generators "externalize" iteration, whereas Enumerator "internalize"
    > >> it (but this docs may have already told you).
    > >>
    > >> Generators can be used to create a "controlled stream" of objects,
    > >> that you can operate as you where using a remove control: go to the
    > >> next (Generator#next), give me the position (Generator#pos), return to
    > >> the beginning (Generator#rewind), ...

    > >
    > > Enumerators have #next and #rewind too, though (though not #pos). In
    > > 1.9, as far as I can tell, the generator.rb library has been removed,

    >
    > I wasn't aware of it in 1.9. Good to now :)
    >
    > > and Generator is now a class inside Enumerator. An Enumerator::Generator
    > > is created automatically, for the use of the enumerator, if you create
    > > an enumerator with a block. Also, these Generators don't have #next and
    > > friends; that's available via the enumerator. (I'm not sure what
    > > happened to #pos.)
    > >
    > > As I understand it, the main thing about generators is that "controlled
    > > stream" thing, where you can roll your own sense of iteration, rather
    > > than just piggy-backing on what some enumerable object with its own
    > > ideas about iteration thinks. Here's a (very contrived) 1.9 example:
    > >
    > > [dblack@ruby-versions ~]$ cat e.rb message = nil
    > >
    > > g = Enumerator::Generator.new do |yielder|
    > > yielder << "Hi."
    > > puts "I've been told to #{message}."
    > > case message
    > > when "leave"
    > > yielder << "Bye."
    > > when "stay"
    > > yielder << "I'm still here!"
    > > end
    > > end
    > >
    > > e = Enumerator.new(g)
    > >
    > > puts e.next
    > > message = "stay"
    > > puts e.next
    > >
    > > e.rewind
    > >
    > > puts e.next
    > > message = "leave"
    > > puts e.next
    > >
    > > [dblack@ruby-versions ~]$ ruby e.rb Hi.
    > > I've been told to stay.
    > > I'm still here!
    > > Hi.
    > > I've been told to leave.
    > > Bye.
    > >
    > >
    > > David
    > >
    > > --
    > > David A. Black, Senior Developer, Cyrus Innovation Inc.
    > >
    > > The Ruby training with Black/Brown/McAnally
    > > Compleat Philadelphia, PA, October 1-2, 2010
    > > Rubyist http://www.compleatrubyist.com
    > >
    > >

    >
    >
     
    Hal Fulton, Jul 22, 2010
    #6
  7. Hi --

    On Fri, 23 Jul 2010, Hal Fulton wrote:

    > My next questions then are:
    >
    > 1. Does Enumerator::Generator have all the functionality of the
    > old generator.rb, just (it seems to me) a different usage?


    No; it doesn't have #next or #rewind, but enumerators
    (which can easily/transparently wrap generators) do. Neither of them
    appears to have #pos, unless it's available in some other way that I
    haven't spotted. (I'm writing this in haste and without benefit of irb,
    so I'm not digging further right now.)

    > 2. In David's example, is "yielder" simply an array of objects? Do
    > we know or care? (If so, I'd be tempted to call it "list" because
    > "yielder" reminds of the keyword yield which I suppose is unrelated
    > here.)


    It's actually an Enumerator::Yielder object, which I think only exists
    for the purpose of being the receptacle for the objects to be yielded by
    the generator.


    David

    --
    David A. Black, Senior Developer, Cyrus Innovation Inc.

    The Ruby training with Black/Brown/McAnally
    Compleat Philadelphia, PA, October 1-2, 2010
    Rubyist http://www.compleatrubyist.com
     
    David A. Black, Jul 22, 2010
    #7
  8. Hal Fulton

    Roger Pack Guest

    Roger Pack, Jul 22, 2010
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Matt Taylor
    Replies:
    6
    Views:
    449
    Victor Bazarov
    Jul 13, 2004
  2. Replies:
    6
    Views:
    4,226
    Pete Becker
    Apr 23, 2005
  3. Dave
    Replies:
    1
    Views:
    328
    Ron Natalie
    Nov 9, 2005
  4. Replies:
    5
    Views:
    129
  5. Omran Nazir

    Newbie question: Enumerators

    Omran Nazir, Nov 21, 2009, in forum: Ruby
    Replies:
    4
    Views:
    173
    Omran Nazir
    Nov 22, 2009
Loading...

Share This Page