multiple threads with Logging: ValueError: I/O operation on closedfile

Discussion in 'Python' started by scriptlearner@gmail.com, Nov 8, 2008.

  1. Guest

    OS: Solaris 9
    Python Version: 2.4.4

    I need to log certain data in a worker thread; however, I am getting
    an error now when I use two worker threads.
    I think the problem comes from the line
    logging.info('Thread Object (%d):(%d), Time:%s in seconds %d'%
    (self.no,self.duration,time.ctime(),time.time()))
    when multiple worker thread is trying to update the log files.
    What did I do wrong? Should I lock the log file before writing to
    it? Thanks.

    Traceback (most recent call last):
    File "/tools/python-2_4_4.i386.solaris.5_9/lib/python2.4/logging/
    __init__.py", line 737, in emit
    self.stream.write(fs % msg)
    ValueError: I/O operation on closed file
    Traceback (most recent call last):
    File "/tools/python-2_4_4.i386.solaris.5_9/lib/python2.4/logging/
    __init__.py", line 737, in emit
    self.stream.write(fs % msg)
    ValueError: I/O operation on closed file


    class Worker(threading.Thread):
    def __init__(self,no,duration):
    threading.Thread.__init__(self)
    self.no = no
    self.duration = duration

    def run(self):
    end = time.time() + self.duration

    while(end > time.time()):
    logging.info('Thread Object (%d):(%d), Time:%s in seconds
    %d'%(self.no,self.duration,time.ctime(),time.time()))
    time.sleep(10)


    def main():
    children = []
    logging.basicConfig(level=logging.INFO,
    format='%(asctime)s %(levelname)s %
    (message)s',
    filename='logs/myapp.log',
    filemode='w')
    args = parseArgs()

    for i in range(args.threads):
    logging.info('i=%d'%(i))
    children.append(Worker(i,args.duration))
    children.start()
    time.sleep(0.1)
    , Nov 8, 2008
    #1
    1. Advertising

  2. Vinay Sajip Guest

    Re: multiple threads with Logging: ValueError: I/O operation onclosed file

    On Nov 8, 10:52 pm, wrote:
    > OS: Solaris 9
    > Python Version: 2.4.4
    >
    > I need to log certain data in a worker thread; however, I am getting
    > an error now when I use two worker threads.
    > I think the problem comes from the linelogging.info('Thread Object (%d):(%d), Time:%s in seconds %d'%
    > (self.no,self.duration,time.ctime(),time.time()))
    > when multiple worker thread is trying to update the log files.
    > What did I do wrong? Should I lock the log file before writing to
    > it? Thanks.
    >
    > Traceback (most recent call last):
    > File "/tools/python-2_4_4.i386.solaris.5_9/lib/python2.4/logging/
    > __init__.py", line 737, in emit
    > self.stream.write(fs % msg)
    > ValueError: I/O operation on closed file
    > Traceback (most recent call last):
    > File "/tools/python-2_4_4.i386.solaris.5_9/lib/python2.4/logging/
    > __init__.py", line 737, in emit
    > self.stream.write(fs % msg)
    > ValueError: I/O operation on closed file
    >
    > class Worker(threading.Thread):
    > def __init__(self,no,duration):
    > threading.Thread.__init__(self)
    > self.no = no
    > self.duration = duration
    >
    > def run(self):
    > end = time.time() + self.duration
    >
    > while(end > time.time()):
    > logging.info('Thread Object (%d):(%d), Time:%s in seconds
    > %d'%(self.no,self.duration,time.ctime(),time.time()))
    > time.sleep(10)
    >
    > def main():
    > children = []
    > logging.basicConfig(level=logging.INFO,
    > format='%(asctime)s %(levelname)s %
    > (message)s',
    > filename='logs/myapp.log',
    > filemode='w')
    > args = parseArgs()
    >
    > for i in range(args.threads):
    > logging.info('i=%d'%(i))
    > children.append(Worker(i,args.duration))
    > children.start()
    > time.sleep(0.1)


    It would be helpful if you could post a complete script which shows
    the problem. You should be doing a join for each of the threads
    spawned in the main thread, as otherwise the main thread will exit
    after the for loop and cause atexit processing to be done (which
    causes logging handlers to be closed).

    Regards,

    Vinay Sajip
    Vinay Sajip, Nov 10, 2008
    #2
    1. Advertising

  3. Vinay Sajip Guest

    Re: multiple threads with Logging: ValueError: I/O operation onclosed file

    On Nov 8, 10:52 pm, wrote:
    > OS: Solaris 9
    > Python Version: 2.4.4
    >
    > I need to log certain data in a worker thread; however, I am getting
    > an error now when I use two worker threads.
    > I think the problem comes from the linelogging.info('Thread Object (%d):(%d), Time:%s in seconds %d'%
    > (self.no,self.duration,time.ctime(),time.time()))
    > when multiple worker thread is trying to update the log files.
    > What did I do wrong? Should I lock the log file before writing to
    > it? Thanks.
    >
    > Traceback (most recent call last):
    > File "/tools/python-2_4_4.i386.solaris.5_9/lib/python2.4/logging/
    > __init__.py", line 737, in emit
    > self.stream.write(fs % msg)
    > ValueError: I/O operation on closed file
    > Traceback (most recent call last):
    > File "/tools/python-2_4_4.i386.solaris.5_9/lib/python2.4/logging/
    > __init__.py", line 737, in emit
    > self.stream.write(fs % msg)
    > ValueError: I/O operation on closed file
    >
    > class Worker(threading.Thread):
    > def __init__(self,no,duration):
    > threading.Thread.__init__(self)
    > self.no = no
    > self.duration = duration
    >
    > def run(self):
    > end = time.time() + self.duration
    >
    > while(end > time.time()):
    > logging.info('Thread Object (%d):(%d), Time:%s in seconds
    > %d'%(self.no,self.duration,time.ctime(),time.time()))
    > time.sleep(10)
    >
    > def main():
    > children = []
    > logging.basicConfig(level=logging.INFO,
    > format='%(asctime)s %(levelname)s %
    > (message)s',
    > filename='logs/myapp.log',
    > filemode='w')
    > args = parseArgs()
    >
    > for i in range(args.threads):
    > logging.info('i=%d'%(i))
    > children.append(Worker(i,args.duration))
    > children.start()
    > time.sleep(0.1)


    Take a look at this example test script to see how to use logging in a
    multi-threaded environment:

    http://dpaste.com/hold/89734/

    Regards,

    Vinay Sajip
    Vinay Sajip, Nov 10, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. j vickroy
    Replies:
    5
    Views:
    783
    Peter Otten
    Nov 14, 2003
  2. dj
    Replies:
    5
    Views:
    4,164
    Vinay Sajip
    Apr 14, 2009
  3. Adam Mercer
    Replies:
    0
    Views:
    134
    Adam Mercer
    Jun 12, 2013
  4. Peter Otten
    Replies:
    0
    Views:
    169
    Peter Otten
    Jun 12, 2013
  5. Adam Mercer
    Replies:
    0
    Views:
    120
    Adam Mercer
    Jun 12, 2013
Loading...

Share This Page