How to free memory ( ie garbage collect) at run time with Python 2.5.1(windows)

[rfv-370]

have made the following small test:

Before starting my test my UsedPhysicalMemory(PF): 555Mb

tf=range(0,10000000) PF: 710Mb ( so 155Mb for my List)
tf=[0,1,2,3,4,5] PF: 672Mb (Why? Why the remaining 117Mb is not freed?)
del tf PF: 672Mb (unused memory not freed)

So changing the list contents and/or deleting the list changes
nothing...from a memory point of view.

This is a problem as I have several applications/threads competing for
memory share. (ie wxpython app).

So how can I force Python to clean the memory and free the memory that
is not used?


PS: gc.collect() does nothing - To return memory I need to kill my
process (ie the interpreter).

Any ideas is welcome.

Thanx

Robert

 

[Alex Martelli]

have made the following small test:

Before starting my test my UsedPhysicalMemory(PF): 555Mb

tf=range(0,10000000) PF: 710Mb ( so 155Mb for my List)
tf=[0,1,2,3,4,5] PF: 672Mb (Why? Why the remaining 117Mb is
not freed?) del tf PF: 672Mb (unused memory
not freed)

Integer objects that are once generated are kept around in a "free list"
against the probability that they might be needed again in the future (a
few other types of objects similarly keep a per-type free-list, but I
think int is the only one that keeps an unbounded amount of memory
there). Like any other kind of "cache", this free-list (in normal
cases) hoards a bit more memory than needed, but results in better
runtime performance; anomalous cases like your example can however
easily "bust" this too-simple heuristic.

So how can I force Python to clean the memory and free the memory that
is not used?

On Windows, with Python 2.5, I don't know of a good approach (on Linux
and other Unix-like systems I've used a strategy based on forking, doing
the bit that needs a bazillion ints in the child process, ending the
child process; but that wouldn't work on Win -- no fork).

I suggest you enter a feature request to let gc grow a way to ask every
type object to prune its cache, on explicit request from the Python
program; this will not solve the problem in Python 2.5, but work on 3.0
is underway and this is just the right time for such requests.


Alex

 

[MC]

Hi!

For windows, I had a soft for this (reduce_memory.exe). Sorry, it's not
write with Python.

If you want, I will give a URL for download it.

 

[Marc 'BlackJack' Rintsch]

have made the following small test:

Before starting my test my UsedPhysicalMemory(PF): 555Mb

tf=range(0,10000000) PF: 710Mb ( so 155Mb for my List)
tf=[0,1,2,3,4,5] PF: 672Mb (Why? Why the remaining 117Mb is not freed?)
del tf PF: 672Mb (unused memory not freed)

So changing the list contents and/or deleting the list changes
nothing...from a memory point of view.

From the OS memory point of view to be precise. Although here the integer
cache applies as Alex pointed out, in the general case memory might be
kept allocated by Python to be re-used later. The Python allocator is
better suited for frequent allocation and deallocation of many small
objects than `malloc()` from the C standard library is. And even a
`free()` in the C standard library doesn't guarantee that memory is given
back to the OS and reported as free memory again.

Ciao,
Marc 'BlackJack' Rintsch

 

[Peter Otten]

Integer objects that are once generated are kept around in a "free list"
against the probability that they might be needed again in the future (a

Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.False

Why don't x and y point to the same object then?

Peter

 

[Peter Otten]

Integer objects that are once generated are kept around in a "free list"
against the probability that they might be needed again in the future (a

Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.False

Why don't x and y point to the same object then?

Peter

Oops, I think I got it now. The actual objects are freed, only the memory is
kept around for reuse with other ints...

Peter

 

[Méta-MCI \(MVP\)]

Re!

Sended by direct (private) e-mail

@+

Michel Claveau

 

[Méta-MCI \(MVP\)]

Aïe!

gmail said : "illegal attachment" (because .exe?)
I will try to send a zipped-file...

@+

Michel Claveau

 

[Francesco Guerrieri]

Integer objects that are once generated are kept around in a "free list"
against the probability that they might be needed again in the future (a

Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

x = 1000
y = 1000
x is y

False
Why don't x and y point to the same object then?
Peter

Oops, I think I got it now. The actual objects are freed, only the memory is
kept around for reuse with other ints...

On my (windows) machine, only integer up to 256 are cached...
I made two dictionaries with mapping from i to id(i) and then
compared. They were equal up to 256.
In short, try your example with 256 and 257 and see what happens

francesco

 

[Bruno Desthuilliers]

Alex Martelli a écrit :

On Windows, with Python 2.5, I don't know of a good approach (on Linux
and other Unix-like systems I've used a strategy based on forking, doing
the bit that needs a bazillion ints in the child process, ending the
child process; but that wouldn't work on Win -- no fork).

IIRC, Windows has it's own way to let you launch other processes, so a
similar strategy might apply here...

 

[Méta-MCI \(MVP\)]

Hi!

I've send the soft.
But, I have no news ; good news?

@+

Michel Claveau