Asaf Las said:
Yes the question was about CPython. But i am not after CPython leaks
though detecting these would be good, but my own mistakes leading to
accumulation of data in mutable structures.
there will be few processes running python code standalone communicating
across servers and every activity will be spread over time so
i have to persistently keep record of activity and remove it later when
activity is finished.
I had a similar concern. My main worry, which turned out to be well-founded,
was that I would create an object as a result of some user input, but when
the user had finished with it, and in theory it could be garbage-collected,
in practice it would not be due to some obscure circular reference
somewhere.
For short-running tasks this is not a cause for concern, but for a
long-running server these can build up over time and end up causing a
problem.
My solution was to log every time an object was created, with some
self-identifying piece of information, and then log when it was deleted,
with the same identifier. After running the program for a while I could then
analyse the log and ensure that each creation had a corresponding deletion.
The tricky bit was logging the deletion. It is a known gotcha in Python that
you cannot rely on the __del__ method, and indeed it can cause a circular
reference in itself which prevents the object from being garbage-collected.
I found a solution somewhere which explained the use of a 'delwatcher'
class. This is how it works -
class MainObject:
def __init__(self, identifier):
self._del = delwatcher('MainObject', identifier)
class delwatcher:
def __init__(self, obj_type, identifier):
self.obj_type = obj_type
self.identifier = identifier
log('{}: id={} created'.format(self.obj_type, self.identifier))
def __del__(self):
log('{}: id={} deleted'.format(self.obj_type, self.identifier))
In this case calling __del__() is safe, as no reference to the main object
is held.
If you do find that an object is not being deleted, it is then
trial-and-error to find the problem and fix it. It is probably a circular
reference
HTH
Frank Millman