garbage collector and slowdown (guillaume weymeskirch)

[guillaume weymeskirch]

Hello everybody,


To test the python 2.5 garbage collector, I wrote a trivial script
allocating dummy objects of various sizes, then forgetting them in a loop.
The garbage collector seems working well, limiting the memory used.

But I've noticed a near linear slowdown of the execution : after a few
minutes - and several millions of allocated and freed objects, each
iteration take more and more time to execute.

Does anybody have noticed this ? Could python suffers from some memory
fragmentation in long running processes ?

I've got the same slowdown in python 2.5.2 on a 64 bits gentoo linux
box, and on a winxp machine.
FYI, the following script shows this : it prints the seconds taken for
each iteration.

--- cut here ---

import random,sys
from timeit import Timer

class Reference(object):
refcount = {}

def __init__(self):
name = self.__class__.__name__
c = self.refcount.setdefault(name,0)
self.refcount[name] = c + 1


class ClassicSmall(Reference):
def __init__(self):
super(ClassicSmall,self).__init__()
self.a = 0;
self.b = 20000;
self.c = random.randint(0,500)
self.d = random.randint(0,500)


class ClassicBig(Reference):
def __init__(self):
super(ClassicBig,self).__init__()
self.mylist = range(1000)


class Tree(Reference):
def __init__(self,left,right):
super(Tree,self).__init__()
self.left = left
self.right = right
self.data = ''.join([chr(x) for x in range(65,128)])


def doit():
smalls = []
bigs = []
trees = []
for i in xrange(30000):
smalls.append(ClassicSmall())
bigs.append(ClassicBig())
trees.append(Tree(1,2))


if __name__ == '__main__':
t = Timer("doit()", "from __main__ import doit; gc.enable()")
min = 0.9e300; max=0.
try:
while True:
d = t.timeit(1)
if d < min:
min = d
if d > max:
max = d
print d
except:
pass
print Reference.refcount
print "max=%f min=%f " % (max,min)