fd leak in parent process for logger.

[ravindrapai34]

Hi All,
I have server process which spawns a process for each request. Where parent process is leaking fd for logger. Please find example code.

from threading import Thread
from multiprocessing import Process
from time import sleep
import logging
from uuid import uuid4

class ChildFile(object):
def __init__(self):
self.logger = logging.getLogger('test')
fh = logging.FileHandler('/tmp/test'+str(uuid4()))
fh.setLevel(logging.INFO)
self.logger.addHandler(fh)
self.fd = open('test2', 'wb')

def run(self):
self.logger.info('dummy run')

def child_file_creator():
a = ChildFile()
child_process = Process(target=a.run)
child_process.start()
child_process.join()

if __name__ == '__main__':
print 'parent process run'
while True:
child_file_creator()
sleep(10)

1) after child process exits.
2) For parent process, still fd remains open.
one can check out using,

cd /proc/23223/fd

ravindra@ravindra-Ideapad-Z570:/proc/23223/fd$ ls -ltr
total 0
l-wx------ 1 ravindra ravindra 64 Nov 11 15:10 6 -> /tmp/test62bba7f1-223c-4c17-a483-f6d92ab67222
l-wx------ 1 ravindra ravindra 64 Nov 11 15:10 5 -> /tmp/test2946cdf6-7e4c-4979-b56a-fd2cc6333398
l-wx------ 1 ravindra ravindra 64 Nov 11 15:10 4 -> /tmp/test0488579b-10d7-4635-abb0-a31a0ea79eeb
lr-x------ 1 ravindra ravindra 64 Nov 11 15:10 3 -> /dev/urandom
lrwx------ 1 ravindra ravindra 64 Nov 11 15:10 2 -> /dev/pts/19
lrwx------ 1 ravindra ravindra 64 Nov 11 15:10 1 -> /dev/pts/19
lrwx------ 1 ravindra ravindra 64 Nov 11 15:10 0 -> /dev/pts/19

3) while normal file descriptor of open 'test2' is closed. But fd attached to logger is leaking.

How can I close for the same for logger object.

Thanks and Regards,
Ravindra M

 

[Roy Smith]

Hi All,
I have server process which spawns a process for each request. Where
parent process is leaking fd for logger. Please find example code.

You've got a lot of code here. The first thing to do when trying to
debug a problem like this is get rid of anything that's not essential to
demonstrating the problem. You import Thread, but don't appear to use
it. Get rid of the import line. You also do this in a loop. Can you
demonstrate the same problem with a single pass?

Anyway, my guess here is that the logging package is keeping a reference
to your log file when you create each FileHandler. Try configuring the
logger in your run() method instead of __init__(). I believe (untested)
that will make the file get opened in the child process.