PEP 288 ponderings

Discussion in 'Python' started by Steven Bethard, Jan 2, 2005.

  1. PEP 288 was mentioned in one of the lambda threads and so I ended up
    reading it for the first time recently. I definitely don't like the
    idea of a magical __self__ variable that isn't declared anywhere. It
    also seemed to me like generator attributes don't really solve the
    problem very cleanly. An example from the PEP[1]:

    def mygen():
    while True:
    print __self__.data
    yield None

    g = mygen()
    g.data = 1
    g.next() # prints 1
    g.data = 2
    g.next() # prints 2

    I looked in the archives but couldn't find a good discussion of why
    setting an attribute on the generator is preferable to passing the
    argument to next. Isn't this example basically equivalent to:

    class mygen(object):
    def next(self, data):
    print data
    return None

    g = mygen()
    g.next(1) # prints 1
    g.next(2) # prints 2

    Note that I didn't even define an __iter__ method since it's never used
    in the example.

    Another example from the PEP:

    def filelike(packagename, appendOrOverwrite):
    data = []
    if appendOrOverwrite == 'w+':
    data.extend(packages[packagename])
    try:
    while True:
    data.append(__self__.dat)
    yield None
    except FlushStream:
    packages[packagename] = data

    ostream = filelike('mydest','w')
    ostream.dat = firstdat; ostream.next()
    ostream.dat = firstdat; ostream.next()
    ostream.throw(FlushStream)

    This could be rewritten as:

    class filelike(object):
    def __init__(self, packagename, appendOrOverwrite):
    self.data = []
    if appendOrOverwrite == 'w+':
    self.data.extend(packages[packagename])
    def next(self, dat):
    self.data.append(dat)
    return None
    def close(self):
    packages[packagename] = self.data

    ostream = filelike('mydest','w')
    ostream.next(firstdat)
    ostream.next(firstdat)
    ostream.close()

    So, I guess I have two questions:

    (1) What's the benefit of the generator versions of these functions over
    the class-based versions?

    (2) Since in all the examples there's a one-to-one correlation between
    setting a generator attribute and calling the generator's next function,
    aren't these generator attribute assignments basically just trying to
    define the 'next' parameter list?

    If this is true, I would have expected that a more useful idiom would
    look something like:

    def mygen():
    while True:
    data, = nextargs()
    print data
    yield None

    g = mygen()
    g.next(1) # prints 1
    g.next(2) # prints 2

    where the nextargs function retrieves the arguments of the most recent
    call to the generator's next function.

    With a little sys._getframe hack, you can basically get this behavior now:

    py> class gen(object):
    .... def __init__(self, gen):
    .... self.gen = gen
    .... def __iter__(self):
    .... return self
    .... def next(self, *args):
    .... return self.gen.next()
    .... @staticmethod
    .... def nextargs():
    .... return sys._getframe(2).f_locals['args']
    ....
    py> def mygen():
    .... while True:
    .... data, = gen.nextargs()
    .... print data
    .... yield None
    ....
    py> g = gen(mygen())
    py> g.next(1)
    1
    py> g.next(2)
    2

    Of course, it's still a little magical, but I think I like it a little
    better because you can see, looking only at 'mygen', when 'data' is
    likely to change value...


    Steve

    [1] http://www.python.org/peps/pep-0288.html
     
    Steven Bethard, Jan 2, 2005
    #1
    1. Advertising

  2. Steven Bethard

    Ian Bicking Guest

    Steven Bethard wrote:
    > PEP 288 was mentioned in one of the lambda threads and so I ended up
    > reading it for the first time recently. I definitely don't like the
    > idea of a magical __self__ variable that isn't declared anywhere. It
    > also seemed to me like generator attributes don't really solve the
    > problem very cleanly. An example from the PEP[1]:
    >
    > def mygen():
    > while True:
    > print __self__.data
    > yield None
    >
    > g = mygen()
    > g.data = 1
    > g.next() # prints 1
    > g.data = 2
    > g.next() # prints 2


    I don't get why this isn't good enough:

    def mygen(data):
    while True:
    print data[0]
    yield None

    data = [None]
    g = mygen(data)
    data[0] = 1
    g.next()
    data[0] = 1
    g.next()

    Using a one-element list is kind of annoying, because it isn't clear out
    of context that it's just a way of creating shared state. But it's
    okay, work right now, and provides the exact same functionality. The
    exception part of PEP 288 still seems interesting.

    --
    Ian Bicking / / http://blog.ianbicking.org
     
    Ian Bicking, Jan 2, 2005
    #2
    1. Advertising

  3. Steven Bethard

    Nick Coghlan Guest

    Ian Bicking wrote:
    > Using a one-element list is kind of annoying, because it isn't clear out
    > of context that it's just a way of creating shared state. But it's
    > okay, work right now, and provides the exact same functionality.


    Uh, isn't shared state what classes were invented for?

    Py> class mygen(object):
    .... def __init__(self, data):
    .... self.data = data
    .... def __iter__(self):
    .... while 1:
    .... print self.data
    .... yield None
    ....
    Py> g = mygen(0)
    Py> giter = iter(g)
    Py> giter.next()
    0
    Py> g.data = 1
    Py> giter.next()
    1

    Cheers,
    Nick.


    --
    Nick Coghlan | | Brisbane, Australia
    ---------------------------------------------------------------
    http://boredomandlaziness.skystorm.net
     
    Nick Coghlan, Jan 2, 2005
    #3
  4. [Steven Bethard]
    > (1) What's the benefit of the generator versions of these functions over
    > the class-based versions?


    Generators are easier to write, are clearer, and run faster.

    They automatically
    * create a distinct generator-iterator object upon each invocation
    * create the next() and idempotent __iter__() methods.
    * raise StopIteration upon termination
    * remain stopped if next() is called too many times
    * save their own local variable state, avoiding the need for self.var references
    * resume execution at the point of the last yield



    > (2) Since in all the examples there's a one-to-one correlation between
    > setting a generator attribute and calling the generator's next function,
    > aren't these generator attribute assignments basically just trying to
    > define the 'next' parameter list?


    They are not the same. The generator needs some way to receive the values. The
    function arguments cannot be used because they are needed to create the
    generator-iterator. The yield statements likewise won't work because the first
    yield is not encountered until well after the first next() call.

    The given examples are minimal and are intended only to demonstrate the idea.



    > I definitely don't like the
    > idea of a magical __self__ variable that isn't declared anywhere.


    It is no more magical than f.__name__ or f.__doc__ for functions. The concept
    is almost identical to that for threading.local(). Also, the __self__ argument
    is a non-issue because there are other alternate approaches such as providing a
    function that retrieves the currently running generator. Which approach is
    ultimately taken is a matter of aesthetics -- the PEP itself concentrates on the
    core idea instead of debating syntax.

    The real issue with PEP 288's idea for generator attributes is that the current
    C implementation doesn't readily accommodate this change. Major surgery would
    be required :-(

    The more important part of the PEP is the idea for generator exceptions. The
    need arises in the context of flushing/closing resources upon generator
    termination.



    Raymond Hettinger
     
    Raymond Hettinger, Jan 2, 2005
    #4
  5. Raymond Hettinger wrote:
    > [Steven Bethard]
    >>(2) Since in all the examples there's a one-to-one correlation between
    >>setting a generator attribute and calling the generator's next function,
    >>aren't these generator attribute assignments basically just trying to
    >>define the 'next' parameter list?

    >
    > They are not the same. The generator needs some way to receive the values. The
    > function arguments cannot be used because they are needed to create the
    > generator-iterator. The yield statements likewise won't work because the first
    > yield is not encountered until well after the first next() call.


    Yeah, I wasn't trying to claim that passing the arguments to .next() is
    equivalent to generator attributes, only that the point at which new
    values for the generator state variables are provided correspond with
    calls to .next(). So if there was a means within a generator of getting
    access to the arguments passed to .next(), generator attributes would be
    unnecessary for the examples provided.

    > The given examples are minimal and are intended only to demonstrate the idea.


    Do you have an example where the generator state isn't updated in
    lock-step with .next() calls? I'd be interested to look at an example
    of this...

    >>I definitely don't like the
    >>idea of a magical __self__ variable that isn't declared anywhere.

    >
    > It is no more magical than f.__name__ or f.__doc__ for functions.


    I'm not sure this is quite a fair parallel. The difference here is that
    f.__name__ and f.__doc__ are accessed as attributes of the f object,
    and the __name__ and __doc__ attributes are created as a result of
    function creation. The proposed __self__ is (1) not an attribute that
    becomes available, rather, a new binding local to the function, and (2)
    not created as a result of generator object creation but created as a
    result of calling .next() on the generator object.

    > Also, the __self__ argument is a non-issue because there are other alternate
    > approaches such as providing a function that retrieves the currently
    > running generator.


    Is there a discussion of some of these alternate suggested approaches
    somewhere you could point me to?

    > The more important part of the PEP is the idea for generator exceptions. The
    > need arises in the context of flushing/closing resources upon generator
    > termination.


    I wonder if maybe it would be worth moving this part to a separate PEP.
    It seems like it could probably stand on its own merit, and having it
    in with the generator attributes PEP means it isn't likely to be
    accepted separately.

    Of course, I would probably declare a class and provide a .close()
    method. =)

    Steve
     
    Steven Bethard, Jan 2, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Francis Avila
    Replies:
    9
    Views:
    369
    Francis Avila
    Nov 20, 2003
  2. Christoph Becker-Freyseng

    PEP for new modules (I read PEP 2)

    Christoph Becker-Freyseng, Jan 15, 2004, in forum: Python
    Replies:
    3
    Views:
    390
    Gerrit Holl
    Jan 16, 2004
  3. Lie
    Replies:
    25
    Views:
    772
    Dafydd Hughes
    Dec 18, 2007
  4. uridium

    FYI--Hanging .NET v1.0.3705.288 Web Service

    uridium, Sep 6, 2003, in forum: ASP .Net Web Services
    Replies:
    0
    Views:
    140
    uridium
    Sep 6, 2003
  5. Replies:
    44
    Views:
    372
    Hal Fulton
    Aug 23, 2006
Loading...

Share This Page