GC performance with lists

[jonas]

While working on some python wrapping, I've run into some problems
where the GC seems to take an unreasonable amount of time to run. The
code below is a demonstration:

import gc
#gc.disable()

data = []
for i in xrange(100000):

shortdata = []
for j in range(57):
mytuple = (j, i+1, i+2, i+3, i+4, i+5, i+6)
shortdata.append(mytuple)
data.extend(shortdata)

print len(data)

with gc disabled (the second line) the code runs in 15 seconds, with
it enabled it runs in 2:15, or ~9x slower. I expected some gc
overhead, but not an order of magnitude! Am I doing something
obviously wrong in the above code?

Thanks,
...Eric

 

[Zentrader]

On Sep 4, 7:06 am, (e-mail address removed) wrote:
One thing to do is to calc i+1 etc before the j loop instead of on
every iteration. That is, calculate 600,000 times instead of
6*57*100,000=34,200,00, And in today's world, it probably won't make
a lot of difference, This is not related to gc but is a good
programming practice IMHO. Personal preference is to use a varialble
to store the result If I have to calculate something more than twice.

 

[Zentrader]

-----Original Message-----
From: [email protected] [mailtoython-
[email protected]] On Behalf Of (e-mail address removed)
Sent: Tuesday, September 04, 2007 8:07 AM
To: (e-mail address removed)
Subject: GC performance with lists

While working on some python wrapping, I've run into some problems
where the GC seems to take an unreasonable amount of time to run. The
code below is a demonstration:

import gc
#gc.disable()

data = []
for i in xrange(100000):

shortdata = []
for j in range(57):
mytuple = (j, i+1, i+2, i+3, i+4, i+5, i+6)
shortdata.append(mytuple)
data.extend(shortdata)

print len(data)

Isn't this the type of program that psyco is supposed to do well
with? Perhaps someone else knows a little bit more.