finding memory leak in edgewall trac 0.11

[rupert.thurner]

what would be a good means of finding where the 0.11 version of
edgewall trac uses excessive memory. see
http://groups.google.com/group/trac-dev/browse_thread/thread/116e519da54f16b
for some details, where jonas suggested
http://wingolog.org/archives/2007/11/27/reducing-the-footprint-of-python-applications
as reading.

tiran already gave some hints on http://bugs.python.org/issue1871, but
also suggested to ask the general mailing list:

Do you have classes with a __del__ method which may create reference
cycles? The GC can't break cycles when a __del__ method is involved.

Are you keeping references to tracebacks, exception objects (except
Exception, err) or frames (sys._getframe())

many thanks,

rupert.

 

[Christian Heimes]

what would be a good means of finding where the 0.11 version of
edgewall trac uses excessive memory. see
http://groups.google.com/group/trac-dev/browse_thread/thread/116e519da54f16b
for some details, where jonas suggested
http://wingolog.org/archives/2007/11/27/reducing-the-footprint-of-python-applications
as reading.

tiran already gave some hints on http://bugs.python.org/issue1871, but
also suggested to ask the general mailing list:

Do you have classes with a __del__ method which may create reference
cycles? The GC can't break cycles when a __del__ method is involved.

Are you keeping references to tracebacks, exception objects (except
Exception, err) or frames (sys._getframe())

I forgot one important point in my reply. The GC module contains some
useful methods for debugging. Check gc.garbage. It should be empty.

http://docs.python.org/lib/module-gc.html

Christian

 

[Christian Heimes]

what would be a good means of finding where the 0.11 version of
edgewall trac uses excessive memory. see
http://groups.google.com/group/trac-dev/browse_thread/thread/116e519da54f16b
for some details, where jonas suggested
http://wingolog.org/archives/2007/11/27/reducing-the-footprint-of-python-applications
as reading.

tiran already gave some hints on http://bugs.python.org/issue1871, but
also suggested to ask the general mailing list:

Do you have classes with a __del__ method which may create reference
cycles? The GC can't break cycles when a __del__ method is involved.

Are you keeping references to tracebacks, exception objects (except
Exception, err) or frames (sys._getframe())

I forgot one important point in my reply. The GC module contains some
useful methods for debugging. Check gc.garbage. It should be empty.

http://docs.python.org/lib/module-gc.html

Christian

 

[Jeroen Ruigrok van der Werven]

Hi Christian,

I forgot one important point in my reply. The GC module contains some
useful methods for debugging. Check gc.garbage. It should be empty.

Yeah, we're messing around with that stuff as well as many other ways of
trying to track issues, but it can really be looking for a needle in a
haystack to be honest.
There's so much output that, I guess, make sense only when you're semi-deep
into the Python internals to even make heads or tails out of it. =\
And even third-party code is not helping much to reduce the clutter and
provide insight.

 

[Christian Heimes]

Hi Christian,

I forgot one important point in my reply. The GC module contains some
useful methods for debugging. Check gc.garbage. It should be empty.

Yeah, we're messing around with that stuff as well as many other ways of
trying to track issues, but it can really be looking for a needle in a
haystack to be honest.
There's so much output that, I guess, make sense only when you're semi-deep
into the Python internals to even make heads or tails out of it. =\
And even third-party code is not helping much to reduce the clutter and
provide insight.

Under normal circumstances gc.garbage should be an empty list. In
general it's a bad sign if gc.garbage contains lots of objects.

I found several potential leaks in trac:

$ find -name \*.py | xargs grep __del__
../trac/versioncontrol/svn_fs.py: def __del__(self):
../trac/versioncontrol/svn_fs.py: def __del__(self):
../trac/db/pool.py: def __del__(self):

$ find -name \*.py | xargs grep frame
../trac/web/main.py:
[...]
../trac/core.py: frame = sys._getframe(1)
../trac/core.py: locals_ = frame.f_locals

I recommend that you either replace __del__ with a weak reference
callback or to remove it. Referencing a frame, traceback or f_locals is
going to leak, too. You *must* explicitly del every frame and locals
variable.

Christian

 

[Christian Heimes]

Hi Christian,

I forgot one important point in my reply. The GC module contains some
useful methods for debugging. Check gc.garbage. It should be empty.

Yeah, we're messing around with that stuff as well as many other ways of
trying to track issues, but it can really be looking for a needle in a
haystack to be honest.
There's so much output that, I guess, make sense only when you're semi-deep
into the Python internals to even make heads or tails out of it. =\
And even third-party code is not helping much to reduce the clutter and
provide insight.

Under normal circumstances gc.garbage should be an empty list. In
general it's a bad sign if gc.garbage contains lots of objects.

I found several potential leaks in trac:

$ find -name \*.py | xargs grep __del__
../trac/versioncontrol/svn_fs.py: def __del__(self):
../trac/versioncontrol/svn_fs.py: def __del__(self):
../trac/db/pool.py: def __del__(self):

$ find -name \*.py | xargs grep frame
../trac/web/main.py:
[...]
../trac/core.py: frame = sys._getframe(1)
../trac/core.py: locals_ = frame.f_locals

I recommend that you either replace __del__ with a weak reference
callback or to remove it. Referencing a frame, traceback or f_locals is
going to leak, too. You *must* explicitly del every frame and locals
variable.

Christian

 

[rupert.thurner]

Hi Christian,

I forgot one important point in my reply. The GC module contains some
useful methods for debugging. Check gc.garbage. It should be empty.

Yeah, we're messing around with that stuff as well as many other ways of
trying to track issues, but it can really be looking for a needle in a
haystack to be honest.
There's so much output that, I guess, make sense only when you're semi-deep
into the Python internals to even make heads or tails out of it. =\
And even third-party code is not helping much to reduce the clutter and
provide insight.

Under normal circumstances gc.garbage should be an empty list. In
general it's a bad sign if gc.garbage contains lots of objects.

I found several potential leaks in trac:

$ find -name \*.py | xargs grep __del__
./trac/versioncontrol/svn_fs.py:    def __del__(self):
./trac/versioncontrol/svn_fs.py:    def __del__(self):
./trac/db/pool.py:    def __del__(self):

$ find -name \*.py | xargs grep frame
./trac/web/main.py:
[...]
./trac/core.py:        frame = sys._getframe(1)
./trac/core.py:        locals_ = frame.f_locals

I recommend that you either replace __del__ with a weak reference
callback or to remove it. Referencing a frame, traceback or f_locals is
going to leak, too. You *must* explicitly del every frame and locals
variable.

Christian

many thanks! as the main change was replacing clearsilver with genshi,
this means one could do the same thing with genshi, http://genshi.edgewall.org/?

$ find -name \*.py | xargs grep frame
./genshi/filters/html.py: 'dir', 'disabled', 'enctype', 'for',
'frame', 'headers', 'height',
./genshi/input.py: _EMPTY_ELEMS = frozenset(['area', 'base',
'basefont', 'br', 'col', 'frame',
./genshi/output.py: 'http://www.w3.org/TR/html4/frameset.dtd'
./genshi/output.py: 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-
frameset.dtd'
./genshi/output.py: * "html-transitional" for the HTML 4.01
frameset DTD
./genshi/output.py: * "xhtml-frameset" for the XHTML 1.0
frameset DTD
./genshi/output.py: 'html-frameset': DocType.HTML_FRAMESET,
./genshi/output.py: 'xhtml-frameset': cls.XHTML_FRAMESET,
./genshi/output.py: _EMPTY_ELEMS = frozenset(['area', 'base',
'basefont', 'br', 'col', 'frame',
./genshi/template/base.py: _ctxt2dict = lambda ctxt: ctxt.frames[0]
./genshi/template/base.py: self.frames = deque([data])
./genshi/template/base.py: self.pop = self.frames.popleft
./genshi/template/base.py: self.push = self.frames.appendleft
./genshi/template/base.py: return repr(list(self.frames))
./genshi/template/base.py: for frame in self.frames:
./genshi/template/base.py: if key in frame:
./genshi/template/base.py: del frame[key]
./genshi/template/base.py: value, frame = self._find(key)
./genshi/template/base.py: if frame is None:
./genshi/template/base.py: self.frames[0][key] = value
./genshi/template/base.py: """Retrieve a given variable's value
and the frame it was found in.
./genshi/template/base.py: for frame in self.frames:
./genshi/template/base.py: if key in frame:
./genshi/template/base.py: return frame[key], frame
./genshi/template/base.py: for frame in self.frames:
./genshi/template/base.py: if key in frame:
./genshi/template/base.py: return frame[key]
./genshi/template/base.py: for frame in self.frames:
./genshi/template/base.py: keys += [key for key in frame if
key not in keys]
./genshi/template/directives.py: # Store the function reference
in the bottom context frame so that it
./genshi/template/directives.py: ctxt.frames[-1][self.name] =
function
./genshi/template/directives.py: frame = {}
./genshi/template/directives.py: ctxt.push(frame)
./genshi/template/tests/directives.py: frame =
exc_traceback.tb_next
./genshi/template/tests/directives.py: frames = []
./genshi/template/tests/directives.py: while frame.tb_next:
./genshi/template/tests/directives.py: frame =
frame.tb_next
./genshi/template/tests/directives.py:
frames.append(frame)
./genshi/template/tests/directives.py:
frames[-1].tb_frame.f_code.co_name)
./genshi/template/tests/directives.py:
frames[-1].tb_frame.f_code.co_filename)
./genshi/template/tests/directives.py: self.assertEqual(2,
frames[-1].tb_lineno)
./genshi/template/tests/eval.py: frame =
exc_traceback.tb_next
./genshi/template/tests/eval.py: frames = []
./genshi/template/tests/eval.py: while frame.tb_next:
./genshi/template/tests/eval.py: frame = frame.tb_next
./genshi/template/tests/eval.py: frames.append(frame)
./genshi/template/tests/eval.py:
frames[-3].tb_frame.f_code.co_name)
./genshi/template/tests/eval.py:
frames[-3].tb_frame.f_code.co_filename)
./genshi/template/tests/eval.py: self.assertEqual(50,
frames[-3].tb_lineno)
./genshi/template/tests/eval.py: frame =
exc_traceback.tb_next
./genshi/template/tests/eval.py: while frame.tb_next:
./genshi/template/tests/eval.py: frame = frame.tb_next
./genshi/template/tests/eval.py: code =
frame.tb_frame.f_code
./genshi/template/tests/eval.py: self.fail("never found
the frame I was looking for")
./genshi/template/tests/eval.py: self.assertEqual(50,
frame.tb_lineno)
./genshi/template/tests/eval.py: frame =
exc_traceback.tb_next
./genshi/template/tests/eval.py: while frame.tb_next:
./genshi/template/tests/eval.py: frame = frame.tb_next
./genshi/template/tests/eval.py: code =
frame.tb_frame.f_code
./genshi/template/tests/eval.py: self.fail("never found
the frame I was looking for")
./genshi/template/tests/eval.py: self.assertEqual(50,
frame.tb_lineno)


rupert

 

[rupert.thurner]

Jeroen Ruigrok van der Werven wrote:

Hi Christian,
-On [20080119 16:16], Christian Heimes ([email protected]) wrote:
I forgot one important point in my reply. The GC module contains some
useful methods for debugging. Check gc.garbage. It should be empty.
Yeah, we're messing around with that stuff as well as many other ways of
trying to track issues, but it can really be looking for a needle in a
haystack to be honest.
There's so much output that, I guess, make sense only when you're semi-deep
into the Python internals to even make heads or tails out of it. =\
And even third-party code is not helping much to reduce the clutter and
provide insight.

Under normal circumstances gc.garbage should be an empty list. In
general it's a bad sign if gc.garbage contains lots of objects.

I found several potential leaks in trac:

$ find -name \*.py | xargs grep __del__
./trac/versioncontrol/svn_fs.py:    def __del__(self):
./trac/versioncontrol/svn_fs.py:    def __del__(self):
./trac/db/pool.py:    def __del__(self):

$ find -name \*.py | xargs grep frame
./trac/web/main.py:
[...]
./trac/core.py:        frame = sys._getframe(1)
./trac/core.py:        locals_ = frame.f_locals

I recommend that you either replace __del__ with a weak reference
callback or to remove it. Referencing a frame, traceback or f_locals is
going to leak, too. You *must* explicitly del every frame and locals
variable.

Christian

many thanks! as the main change was replacing clearsilver with genshi,
this means one could do the same thing with genshi,http://genshi.edgewall.org/?

$ find -name \*.py | xargs grep frame
./genshi/filters/html.py:        'dir', 'disabled', 'enctype', 'for', ...

- Show quoted text -

i forgot to mention that i cannot see any explicit sys._getframe(), or
__del__ in the genshi code, while the ones in trac-core seemed to be
there in 0.10.4.

rupert

 

[Christian Heimes]

i forgot to mention that i cannot see any explicit sys._getframe(), or
__del__ in the genshi code, while the ones in trac-core seemed to be
there in 0.10.4.

Does the code keep a reference to a traceback object or an attribute of
a traceback object?

Christian

 

[rupert.thurner]

Does the code keep a reference to a traceback object or an attribute of
a traceback object?

Christian

if there is no other possibility but doing it with sys.exc_traceback,
sys.last_traceback, sys.exc_info, the answer is no for genshi.

 

[rupert.thurner]

Does the code keep a reference to a traceback object or an attribute of
a traceback object?

Christian

could you have one look at http://genshi.edgewall.org/browser/trunk/genshi/_speedups.c
?

rupert.