How to write a file generator

[Billy Mays]

I want to make a generator that will return lines from the tail of
/var/log/syslog if there are any, but my function is reopening the file
each call:

def getLines():
with open('/var/log/syslog', 'rb') as f:
while True:
line = f.readline()
if line:
yield line
else:
raise StopIteration


I know the problem lies with the StopIteration, but I'm not sure how to
tell the caller that there are no more lines for now.

 

[bruno.desthuilliers]

I want to make a generator that will return lines from the tail of
/var/log/syslog if there are any

Err... I must have missed something, but python files are their own
iterators.

Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
pythonrc start
pythonrc done.... print line
....
(snip unintersting syslog stuff))

, but my function is reopening the file
each call:

How do you know, and how do you call your function ?

def getLines():
     with open('/var/log/syslog', 'rb') as f:
         while True:
             line = f.readline()
             if line:
                 yield line
             else:
                 raise StopIteration

I know the problem lies with the StopIteration, but I'm not sure how to
tell the caller that there are no more lines for now.

If you want the generator to wait until new content is available, just
remove the raise part - but you'll have a blocking call... Else, I
don't see what behaviour you are expecting exactly.

 

[Thomas Jollans]

I want to make a generator that will return lines from the tail of
/var/log/syslog if there are any, but my function is reopening the file
each call:

def getLines():
with open('/var/log/syslog', 'rb') as f:
while True:
line = f.readline()
if line:
yield line
else:
raise StopIteration


I know the problem lies with the StopIteration, but I'm not sure how to
tell the caller that there are no more lines for now.

http://stackoverflow.com/questions/1475950/tail-f-in-python-with-no-time-sleep

 

[Billy Mays]

http://stackoverflow.com/questions/1475950/tail-f-in-python-with-no-time-sleep

That was actually the behavior I was trying to avoid. If there is no
data to be read, the call will hang. That function is actually called
by a webserver (from wsgiref) so it cannot hang indefinitely.

 

[sturlamolden]

I know the problem lies with the StopIteration, but I'm not sure how to
tell the caller that there are no more lines for now.

Try 'yield None' instead of 'raise StopIteration'.

Sturla

 

[Thomas Jollans]

That was actually the behavior I was trying to avoid. If there is no
data to be read, the call will hang. That function is actually called
by a webserver (from wsgiref) so it cannot hang indefinitely.

In that case, what Bruno said.

 

[Thomas Jollans]

That was actually the behavior I was trying to avoid. If there is no
data to be read, the call will hang. That function is actually called
by a webserver (from wsgiref) so it cannot hang indefinitely.

What Terry said, then. (Not Bruno, I confused that. Sorry for sending a
mail both short and wrong.)

 

[Thomas Rachel]

Am 12.07.2011 16:46 schrieb Billy Mays:

I want to make a generator that will return lines from the tail of
/var/log/syslog if there are any, but my function is reopening the file
each call

....

I have another solution: an object which is not an iterator, but an
iterable.

class Follower(object):
def __init__(self, file):
self.file = file
def __iter__(self):
while True:
l = self.file.readline()
if not l: return
yield l

if __name__ == '__main__':
f = Follower(open("/var/log/messages"))
while True:
for i in f: print i,
print "foo"
import time
time.sleep(4)

Here, you iterate over the object until it is exhausted, but you can
iterate again to get the next entries.


Thomas