RE: Help with generators outside of loops.

Discussion in 'Python' started by Robert Brewer, Dec 7, 2004.

  1. Christopher J. Bottaro wrote:
    > I have a generator that works like this:
    >
    > for row in obj.ExecSQLQuery(sql, args):
    > # process the row
    >
    > Now there are some querys that run where I know the result
    > will only be a
    > single row. Is there anyway to get that single row from the generator
    > without having to use it in a for loop? I want to do
    > something like this:
    >
    > row = obj.ExecSQLQuery(sql, args)[0]
    >
    > But I'm guessing that you can't index into a generator as if
    > it is a list.


    row = obj.ExecSQLQuery(sql, args).next()


    Robert Brewer
    MIS
    Amor Ministries
    Robert Brewer, Dec 7, 2004
    #1
    1. Advertising

  2. In article <>,
    "Robert Brewer" <> wrote:

    > > But I'm guessing that you can't index into a generator as if
    > > it is a list.

    >
    > row = obj.ExecSQLQuery(sql, args).next()


    I've made it a policy in my own code to always surround explicit calls
    to next() with try ... except StopIteration ... guards.

    Otherwise if you don't guard the call and you get an unexpected
    exception from the next(), within a call chain that includes a for-loop
    over another generator, then that other for-loop will terminate without
    any error messages and the cause of its termination can be very
    difficult to track down.

    --
    David Eppstein
    Computer Science Dept., Univ. of California, Irvine
    http://www.ics.uci.edu/~eppstein/
    David Eppstein, Dec 8, 2004
    #2
    1. Advertising

  3. Robert Brewer wrote:
    > Christopher J. Bottaro wrote:
    >
    >>I have a generator that works like this:
    >>
    >>for row in obj.ExecSQLQuery(sql, args):
    >> # process the row
    >>
    >>Now there are some querys that run where I know the result
    >>will only be a
    >>single row. Is there anyway to get that single row from the generator
    >>without having to use it in a for loop? I want to do
    >>something like this:
    >>
    >>row = obj.ExecSQLQuery(sql, args)[0]
    >>
    >>But I'm guessing that you can't index into a generator as if
    >>it is a list.

    >
    >
    > row = obj.ExecSQLQuery(sql, args).next()


    I don't do much with SQL/databases stuff, but if you really know the
    result will be a single row, you can take advantage of tuple unpacking
    and do something like:

    row, = obj.ExecSQLQuery(sql, args)

    or

    [row] = obj.ExecSQLQuery(sql, args)

    This has the advantage that you'll get a ValueError if you happen to be
    wrong (and there are more or fewer values in the generator).

    >>> def g(n):

    .... for i in range(n):
    .... yield i
    ....
    >>> x, = g(1)
    >>> x

    0
    >>> x, = g(2)

    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    ValueError: too many values to unpack
    >>> x, = g(0)

    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    ValueError: need more than 0 values to unpack

    Steve
    Steven Bethard, Dec 8, 2004
    #3
  4. David Eppstein wrote:
    > In article <>,
    > "Robert Brewer" <> wrote:
    >
    >
    >>>But I'm guessing that you can't index into a generator as if
    >>>it is a list.

    >>
    >>row = obj.ExecSQLQuery(sql, args).next()

    >
    >
    > I've made it a policy in my own code to always surround explicit calls
    > to next() with try ... except StopIteration ... guards.
    >
    > Otherwise if you don't guard the call and you get an unexpected
    > exception from the next(), within a call chain that includes a for-loop
    > over another generator, then that other for-loop will terminate without
    > any error messages and the cause of its termination can be very
    > difficult to track down.


    Isn't the handling of StopIteration confined in the very moment of
    calling .next() ? This was what I expected... and from a simple test
    looks also what is happening...

    >>> for x in xrange(10):

    if x == 8:
    raise StopIteration()
    print x


    0
    1
    2
    3
    4
    5
    6
    7

    Traceback (most recent call last):
    File "<pyshell#7>", line 3, in -toplevel-
    raise StopIteration()
    StopIteration


    i.e. the loop didn't stop silently

    Andrea
    Andrea Griffini, Dec 8, 2004
    #4
  5. guarding for StopIteration (WAS: Help with generators outside ofloops.)

    David Eppstein wrote:
    > I've made it a policy in my own code to always surround explicit calls
    > to next() with try ... except StopIteration ... guards.
    >
    > Otherwise if you don't guard the call and you get an unexpected
    > exception from the next(), within a call chain that includes a for-loop
    > over another generator, then that other for-loop will terminate without
    > any error messages and the cause of its termination can be very
    > difficult to track down.


    Just to clarify here, the only time code raising a StopIteration will
    cause a for-loop to exit silently is if the StopIteration is raised in
    an __iter__ method, e.g.:

    >>> def g():

    .... raise StopIteration
    ....
    >>> class C(object):

    .... def __iter__(self):
    .... for i in range(3):
    .... yield i
    .... g()
    ....
    >>> for i in C():

    .... print i
    ....
    0
    >>>


    A StopIteration raised within the body of a for-loop will not cause it
    to terminate silently; the StopIteration exception will be propagated
    upward:

    >>> def g():

    .... raise StopIteration
    ....
    >>> def f(n):

    .... for i in range(n):
    .... print i
    .... g()
    ....
    >>> f(3)

    0
    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    File "<interactive input>", line 4, in f
    File "<interactive input>", line 2, in g
    StopIteration
    >>>



    Steve
    Steven Bethard, Dec 8, 2004
    #5
  6. Re: guarding for StopIteration (WAS: Help with generators outsideof loops.)

    Steven Bethard wrote:
    > Just to clarify here, the only time code raising a StopIteration will
    > cause a for-loop to exit silently is if the StopIteration is raised in
    > an __iter__ method, e.g.:


    That was a little imprecise. What I should have said is "the only time
    code raising a StopIteration will cause a for-loop to exit silently is
    if the StopIteration is raised in the code that is executed under the
    iteration protocol."

    So, using the legacy iterator protocol:

    >>> class C:

    .... def __getitem__(self, index):
    .... if index > 3:
    .... raise IndexError
    .... if index == 1:
    .... raise StopIteration
    .... return index
    ....
    >>> for i in C():

    .... print i
    ....
    0

    Or using a separate iterator object:

    >>> class C(object):

    .... def __iter__(self):
    .... return I()
    ....
    >>> class I(object):

    .... def __init__(self):
    .... self.count = -1
    .... def next(self):
    .... self.count += 1
    .... if self.count == 1:
    .... raise StopIteration
    .... return self.count
    ....
    >>> for i in C():

    .... print i
    ....
    0

    Or using a generator to create the iterator object:

    >>> class C(object):

    .... def __iter__(self):
    .... for i in range(3):
    .... if i == 1:
    .... raise StopIteration
    .... yield i
    ....
    >>> for i in C():

    .... print i
    ....
    0

    Basically, each of these raises a StopIteration in the equivalent of the
    ..next method. If a function that raises a StopIteration is put into any
    of the places where my code says 'raise StopIteration', then that
    StopIteration will silently terminate the loop.

    I don't believe there should be any other places where raising a
    StopIteration would silently terminate a loop.

    Steve
    Steven Bethard, Dec 8, 2004
    #6
  7. Steven Bethard wrote:
    > I don't do much with SQL/databases stuff, but if you really know the
    > result will be a single row, you can take advantage of tuple unpacking
    > and do something like:
    >
    > row, = obj.ExecSQLQuery(sql, args)
    >
    > or
    >
    > [row] = obj.ExecSQLQuery(sql, args)
    >
    > This has the advantage that you'll get a ValueError if you happen to be
    > wrong (and there are more or fewer values in the generator).
    >
    > >>> def g(n):

    > ... for i in range(n):
    > ... yield i
    > ...
    > >>> x, = g(1)
    > >>> x

    > 0
    > >>> x, = g(2)

    > Traceback (most recent call last):
    > File "<interactive input>", line 1, in ?
    > ValueError: too many values to unpack
    > >>> x, = g(0)

    > Traceback (most recent call last):
    > File "<interactive input>", line 1, in ?
    > ValueError: need more than 0 values to unpack
    >
    > Steve


    Wow, good advice. One question, how is the generator class implemented so
    that if assigned to a tuple/list, it knows what to do? Is it possible to
    overload the assignment operator kinda like in C++?

    Thank you all who replied to the original post, it was very helpful.
    Christopher J. Bottaro, Dec 8, 2004
    #7
  8. Christopher J. Bottaro wrote:
    > Wow, good advice. One question, how is the generator class implemented so
    > that if assigned to a tuple/list, it knows what to do? Is it possible to
    > overload the assignment operator kinda like in C++?


    Tuple unpacking works with generators because generators implement the
    iterator protocol. Any object that implements the iterator protocol can
    be used in tuple unpacking:

    >>> class I(object):

    .... def __init__(self, n):
    .... self.n = n
    .... self.i = -1
    .... def next(self):
    .... self.i += 1
    .... if self.i == self.n:
    .... raise StopIteration
    .... return self.i
    .... def __iter__(self):
    .... return self
    ....
    >>> x, y, z = I(3)
    >>> x, y, z

    (0, 1, 2)

    So you're not really overloading the assignment operator; you're taking
    advantage of something the assignment operator already does.

    Steve
    Steven Bethard, Dec 8, 2004
    #8
  9. Re: guarding for StopIteration (WAS: Help with generators outside of loops.)

    In article <62ztd.532216$D%.125153@attbi_s51>,
    Steven Bethard <> wrote:

    > David Eppstein wrote:
    > > I've made it a policy in my own code to always surround explicit calls
    > > to next() with try ... except StopIteration ... guards.
    > >
    > > Otherwise if you don't guard the call and you get an unexpected
    > > exception from the next(), within a call chain that includes a for-loop
    > > over another generator, then that other for-loop will terminate without
    > > any error messages and the cause of its termination can be very
    > > difficult to track down.

    >
    > Just to clarify here, the only time code raising a StopIteration will
    > cause a for-loop to exit silently is if the StopIteration is raised in
    > an __iter__ method, e.g.:


    Sure. But in the situation I was attempting to describe, the __iter__
    method calls the code of the outer generator, which (perhaps nested
    several levels deep) may call an inner generator. In this case, the
    inner generator's StopIteration can terminate the outer loop.

    --
    David Eppstein
    Computer Science Dept., Univ. of California, Irvine
    http://www.ics.uci.edu/~eppstein/
    David Eppstein, Dec 8, 2004
    #9
  10. In article <Dzytd.477960$>,
    Andrea Griffini <> wrote:

    > Isn't the handling of StopIteration confined in the very moment of
    > calling .next() ? This was what I expected... and from a simple test
    > looks also what is happening...


    Not if someone farther back in the call chain is looking for
    StopIteration. Which could be the case if the call chain includes a
    for-loop that is calling the next() method of another generator.

    --
    David Eppstein
    Computer Science Dept., Univ. of California, Irvine
    http://www.ics.uci.edu/~eppstein/
    David Eppstein, Dec 8, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mayer

    help with generators

    Mayer, May 19, 2005, in forum: Python
    Replies:
    4
    Views:
    352
    Steven Bethard
    May 19, 2005
  2. Janus
    Replies:
    11
    Views:
    415
    Harald van =?UTF-8?B?RMSzaw==?=
    Aug 10, 2006
  3. zuzu
    Replies:
    12
    Views:
    286
    Florian Gross
    Aug 27, 2004
  4. Krzysztof Poc

    outside type, outside function

    Krzysztof Poc, Feb 3, 2012, in forum: C++
    Replies:
    1
    Views:
    266
    Victor Bazarov
    Feb 7, 2012
  5. Me
    Replies:
    2
    Views:
    226
Loading...

Share This Page