Re: Tailing a log file?

Discussion in 'Python' started by Kenji Noguchi, Jun 22, 2007.

  1. something like this? unix tail command does more fancy stuff
    like it waits for timeout, and check if the file is truncated
    or depending on incoming data it sleeps seconds , etc etc.

    #!/usr/bin/env python
    import sys, select

    while True:
    ins, outs, errs = select.select([sys.stdin],[],[])
    for i in ins:
    print i.readline()


    2007/6/22, Evan Klitzke <>:
    > On 6/22/07, Evan Klitzke <> wrote:
    > > Everyone,
    > >
    > > I'm interested in writing a python program that reads from a log file
    > > and then executes actions based on the lines. I effectively want to
    > > write a loop that does something like this:
    > >
    > > while True:
    > > log_line = log_file.readline()
    > > do_something(log_line)
    > >
    > > Where the readline() method blocks until a new line appears in the
    > > file, unlike the standard readline() method which returns an empty
    > > string on EOF. Does anyone have any suggestions on how to do this?
    > > Thanks in advance!

    >
    > I checked the source code for tail and they actually poll the file by
    > using fstat and sleep to check for changes in the file size. This
    > didn't seem right so I thought about it more and realized I ought to
    > be using inotify. So I guess I answered my own question.
    >
    > --
    > Evan Klitzke <>
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Kenji Noguchi, Jun 22, 2007
    #1
    1. Advertising

  2. Kenji Noguchi

    Falcolas Guest

    On Jun 22, 12:50 pm, "Kenji Noguchi" <> wrote:
    > > I checked the source code for tail and they actually poll the file by
    > > using fstat and sleep to check for changes in the file size. This
    > > didn't seem right so I thought about it more and realized I ought to
    > > be using inotify. So I guess I answered my own question.


    I built something which worked on the "check stat and sleep" method.
    Using notify or select may work, but it's not portible to Windows
    systems, which may or may not be an issue for you.

    #! /usr/bin/env python

    import os
    import sys
    import time
    from stat import *

    class Watchfile(object):

    def __init__(self, f_loc):

    self.file_loc = f_loc

    self.prev_lm = 0
    self.last_read_pos = 0

    f = open(self.file_loc, "r")
    for l in f:
    pass
    self.last_read_pos = f.tell()
    f.close()

    self.prev_lm = os.stat(self.file_loc)[ST_MTIME]

    def changed(self):

    lm_time = os.stat(self.file_loc)[ST_MTIME]

    if lm_time > self.prev_lm:
    return True
    else:
    return False

    def get_since_last_read(self):

    f = open(self.file_loc, "r")

    f.seek(self.last_read_pos)
    lines = f.readlines()
    self.last_read_pos = f.tell()

    f.close

    self.prev_lm = os.stat(self.file_loc)[ST_MTIME]
    return lines

    if __name__ == "__main__":
    x_file = Watchfile("<file path>")
    y_file = Watchfile("<file path>")

    while True:

    if x_file.changed():

    lines = x_file.get_since_last_read()

    # do something

    if y_file.changed():

    lines = y_file.get_since_last_read()

    # do something else

    try:
    time.sleep(5)
    except KeyboardInterrupt:
    break
     
    Falcolas, Jun 22, 2007
    #2
    1. Advertising

  3. Kenji Noguchi

    Guest

    On Jun 22, 2:50 pm, "Kenji Noguchi" <> wrote:
    > something like this? unix tail command does more fancy stuff
    > like it waits for timeout, and check if the file is truncated
    > or depending on incoming data it sleeps seconds , etc etc.
    >
    > #!/usr/bin/env python
    > import sys, select
    >
    > while True:
    > ins, outs, errs = select.select([sys.stdin],[],[])
    > for i in ins:
    > print i.readline()


    select doesn't work on regular files (only pipes/sockets/etc).
    inotify is platform-specific; the portable way to do this is to poll,
    like tail does. If you know you're on a platform that has inotify or
    something like it, it's probably the more efficient (and lower
    latency) route to go.
     
    , Jun 22, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tony Stock

    tailing a log file using c ...

    Tony Stock, Nov 20, 2003, in forum: C Programming
    Replies:
    10
    Views:
    692
    CBFalconer
    Nov 21, 2003
  2. lebo
    Replies:
    0
    Views:
    346
  3. jalkadir
    Replies:
    6
    Views:
    434
    jalkadir
    Oct 15, 2005
  4. Evan Klitzke

    Tailing a log file?

    Evan Klitzke, Jun 22, 2007, in forum: Python
    Replies:
    0
    Views:
    290
    Evan Klitzke
    Jun 22, 2007
  5. SamXiao

    Tailing a series of log files

    SamXiao, Jun 12, 2013, in forum: Java
    Replies:
    8
    Views:
    262
    Arved Sandstrom
    Jun 16, 2013
Loading...

Share This Page