How to unget a line when reading from a file/streamiterator/generator?

[python]

Is there an elegant way to unget a line when reading from a file/stream
iterator/generator?

By "unget" I mean a way to push back a line into the source stream and
backtrack the iterator/generator one step?

The only alternative I can see is to put my line reading in a while-True
loop (vs. a for-loop) that manually calls my file/stream
iterator/generator's .next() method while manually handling the
StopIteration exception. Doesn't sound too elegant.

Is there a Pythonic design pattern/best practice that I can apply here?

Thank you,
Malcolm

 

[Arnaud Delobelle]

Is there an elegant way to unget a line when reading from a file/stream
iterator/generator?

By "unget" I mean a way to push back a line into the source stream and
backtrack the iterator/generator one step?

The only alternative I can see is to put my line reading in a while-True
loop (vs. a for-loop) that manually calls my file/stream
iterator/generator's .next() method while manually handling the
StopIteration exception. Doesn't sound too elegant.

Is there a Pythonic design pattern/best practice that I can apply here?

It seems to me that the answer depends on what you do with your
unget'ed line (i.e. there is no included general purpose 'unget'
replacement). If you provided some (pseudo-)code, someone might be
able to find an elegant way to perform the task without the need to
push back an element into an iterator.

As a last resort, you could always wrap your iterator into a some
'Backtrackable' class.

class Backtrackable(object):
def __init__(self, iterator):
self.iterator = iter(iterator)
self._back = False
self._last = None
def __iter__(self):
return self
def next(self):
if self._back:
next = self._last
else:
self._last = next = self.iterator.next()
self._back = False
return next
def back(self):
if self._back:
raise('Cannot backtrack twice')
self._back = True

b = Backtrackable([1,2,3])
b.next() 1
b.next() 2
b.back()
b.next() 2
b.next()

3

 

[George Sakkis]

Is there an elegant way to unget a line when reading from a file/stream
iterator/generator?

By "unget" I mean a way to push back a line into the source stream and
backtrack the iterator/generator one step?

The only alternative I can see is to put my line reading in a while-True
loop (vs. a for-loop) that manually calls my file/stream
iterator/generator's .next() method while manually handling the
StopIteration exception. Doesn't sound too elegant.

Is there a Pythonic design pattern/best practice that I can apply here?

Thank you,
Malcolm

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502304

HTH,
George

 

[python]

George,

Is there an elegant way to unget a line when reading from a file/stream iterator/generator?

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502304

That's exactly what I was looking for!

For those following this thread, the above recipe creates a generic
object that wraps any iterator with an 'unget' ("push") capability.
Clean and elegant!

Thank you,
Malcolm

 

[George Sakkis]

George,


http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502304

That's exactly what I was looking for!

For those following this thread, the above recipe creates a generic
object that wraps any iterator with an 'unget' ("push") capability.
Clean and elegant!

Thank you,
Malcolm

A small suggestion: since unget is expected to be called infrequently,
next should better be faster for the common case instead of penalizing
it with a try/except:

def next(self):
if not self.pushed_back:
return self.it.next()
else:
return self.pushed_back.pop()

George

 

[python]

Duncan,

If speed is an issue then it may be better to avoid the test altogether ... <snipped>

Thanks for your suggestion.

Regards,
Malcolm