Iterators

Discussion in 'Python' started by Chris Lyon, Jul 13, 2004.

  1. Chris Lyon

    Chris Lyon Guest

    coming from a Vb'ish sort of background, issues like iterators and
    list comprehensions appear at first site to be a confusion. However
    since I have a great deal of respect for the large brains in a jar
    (LBIAJ) that post on this list and construct my favourite language I
    realise I'm missing something.

    So can anyone point me at some applications, tutorials, websites and
    advantages that these mechanisms provide? that don't rely on having
    done LBIAJ courses such that I dare use them in my own code that
    thrives on the massive over application of lists and dictionaries that
    I believe I finally got my head round.

    Apologies to all for continually stirring up the mud at the bottom of
    the crystal clear river that is python but I fear being left behind
    when whatever is built on top of these constructs raises it's head in
    the forseable future.

    As ever I am attempting to use humour (badly) to voice issues that are
    assumed by many but complicated to a few.

    Chris
    Chris Lyon, Jul 13, 2004
    #1
    1. Advertising

  2. Chris Lyon

    Peter Hansen Guest

    Chris Lyon wrote:

    > So can anyone point me at some applications, tutorials, websites and
    > advantages that these mechanisms provide? that don't rely on having
    > done LBIAJ courses such that I dare use them in my own code that
    > thrives on the massive over application of lists and dictionaries that
    > I believe I finally got my head round.


    There is *nothing* wrong with continuing to use (or over-use :) lists
    and dictionaries. I fully intend to do so myself!

    > Apologies to all for continually stirring up the mud at the bottom of
    > the crystal clear river that is python but I fear being left behind
    > when whatever is built on top of these constructs raises it's head in
    > the forseable future.
    >
    > As ever I am attempting to use humour (badly) to voice issues that are
    > assumed by many but complicated to a few.


    I understand where you're coming from, and I'm sure you'll get
    some responses from others that will actually point you somewhere
    that can help improve your understanding in this area.

    I just wanted to reassure you that you don't need to be too
    concerned about the whole thing. I didn't figure out how to
    write a list comprehension for at least a year after they were
    first added to the language... I had too much productive work
    to do with lists. I have _not_ gone back to change any of the
    list-based code(*), and still only rarely reach for list
    comprehensions as my first option.

    There's nothing wrong with not being a leader in adopting new
    language features. If you try too hard to keep up, you'll
    probably "churn" too much and waste time rewriting code or
    something, when you could just be writing working code using
    the features you know and understand well.

    -Peter

    * The fact that I no longer work at that the company would, of
    course, mean that going back to change the code would result in
    criminal trespassing and probably industrial sabotage charges,
    but that's beside the point. ;-)
    Peter Hansen, Jul 13, 2004
    #2
    1. Advertising

  3. Chris Lyon wrote:
    > coming from a Vb'ish sort of background, issues like iterators and
    > list comprehensions appear at first site to be a confusion. However
    > since I have a great deal of respect for the large brains in a jar
    > (LBIAJ) that post on this list and construct my favourite language I
    > realise I'm missing something.


    Perhaps it's best explained by the problem these constructs were intended to
    solve:

    = list comprehension =
    There seems to be a lot of code like the following:

    # transforming a list
    result = []
    for element in sourcelist:
    result.append (transform (element))

    # filtering a list
    result = []
    for element in sourcelist:
    if passesFilter (element):
    result.append (element)

    # both filtering and transforming
    result = []
    for element in sourcelist:
    if passesFilter (element):
    result.append (transform (element))

    Some thought that this is a lot of typing and possibly obfuscates the intent
    of the code, so list comprehension syntax was borrowed from the Haskell
    programming language:

    result = [transform (element) for element in sourcelist]
    result = [element for element in sourcelist if passesFilter (element)]
    result = [transform (element) for element in sourcelist if passesFilter
    (element)]

    In the old days, this was done using map (for transformation) and filter
    (for filtering):
    result = map (transform, sourcelist)
    result = filter (passesFilter, sourcelist)
    result = map (transform, filter (passesFilter, sourcelist))

    This looks actually shorter than list comprehension. But assume that
    transform and passesFilter are member functions. Then it becomes
    result = map (lambda e: e.transform (), sourcelist)
    result = filter (lambda e: e.passesFilter (), sourcelist)
    result = map (lambda e: e.transform (), filter (lambda e: e.passesFilter (),
    sourcelist))
    versus
    result = [element.transform () for element in sourcelist]
    result = [element for element in sourcelist if element.passesFilter ()]
    result = [element.transform () for element in sourcelist if
    element.passesFilter ()]

    The main disadvantages of list comprehension:
    - you can't insert print statements
    - you can't transform first and filter last unless you nest list
    comprehension
    result = [outer for outer in [transform (inner) for inner in sourcelist]
    if passesFilter (outer)]

    2.4 will also introduce 'generator comprehension':
    - no [] around the construct
    - more efficient because the list isn't built in memory

    = Iterators =
    The code to read all lines of a file used to be as follows:
    while 1:
    line = stream.readline ()
    if not line:
    break # end of file
    pass # do something with line

    which is cumbersome to write.

    Or
    for line in stream.readlines ():
    pass # do something with line

    which will consume too much memory if the file is large.

    So what was needed was some kind of object:
    - that could be used in a for loop
    - doesn't put everything in memory

    Another example would be potentially large SQL queries, where you want to
    loop over all the rows.

    A simple implementation to iterate over the lines of a file
    class LineIterator:
    def __init__ (self, fname):
    self.stream = open (fname, 'r')

    def next (self):
    line = self.stream.readline ()
    if not line:
    raise StopIteration
    return line

    def __iter__ (self):
    return self

    Disadvantage of using iterators:
    - you can read them only once
    - print doesn't write anything useful

    Daniel
    Daniel Dittmar, Jul 13, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ken Sprague
    Replies:
    4
    Views:
    680
  2. Russ Perry Jr
    Replies:
    2
    Views:
    4,120
    Russ Perry Jr
    Aug 20, 2004
  3. Paul Chapman

    Idempotent ODBMS iterators

    Paul Chapman, Feb 16, 2005, in forum: Java
    Replies:
    0
    Views:
    428
    Paul Chapman
    Feb 16, 2005
  4. Marcin Kaliciñski

    Iterators and reverse iterators

    Marcin Kaliciñski, May 8, 2005, in forum: C++
    Replies:
    1
    Views:
    483
    Kai-Uwe Bux
    May 8, 2005
  5. , India
    Replies:
    10
    Views:
    1,071
    James Kanze
    Aug 8, 2009
Loading...

Share This Page