Memory Management in python 2.5

[cesar.ortiz]

Hi, I am starting to have a look to a python program that does not free
memory (I am using python 2.4.3). As I have read about a new memory
management in python 2.5 (http://evanjones.ca/python-memory.html) I
decided to try the program with the new version.
With the new version of python the memory consumption is the same. Now
I am asking myself if python 2.5 has any improving in memory
management or maybe not yet. Thank you.

-- Cesar

 

[Max M]

(e-mail address removed) skrev:

Hi, I am starting to have a look to a python program that does not free
memory (I am using python 2.4.3). As I have read about a new memory
management in python 2.5 (http://evanjones.ca/python-memory.html) I
decided to try the program with the new version.
With the new version of python the memory consumption is the same. Now
I am asking myself if python 2.5 has any improving in memory
management or maybe not yet. Thank you.

In previous versions Python collected memory, but never released it
again. It simply kept on to it, so it could reuse it again later.

From 2.5 onwards it should release most of the unused memory. However
it doesn't use less memory. The peak memory usage should be the same as
before. So for one-off programs that starts up and runs once, there
should not be much gain.


--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science

 

[Fredrik Lundh]

Hi, I am starting to have a look to a python program that does not free
memory (I am using python 2.4.3). As I have read about a new memory
management in python 2.5 (http://evanjones.ca/python-memory.html) I
decided to try the program with the new version.
With the new version of python the memory consumption is the same. Now
I am asking myself if python 2.5 has any improving in memory
management or maybe not yet.

the new mechanism only helps if you temporarily create large structures and release
them quickly; it does not help if you're *fragmenting* the memory. Python requests
memory from the underlying system in blocks (called "arenas"), and it can only re-
turn them if they're entirely empty.

(this is discussed on the page you link to, and the various posts it links to).

</F>

 

[cesar.ortiz]

I just checked the comsuptiom with the 'top' unix util. I am procesing
html docs and the amount of memory rises continiously.
I am using a lot of lists and docs. Some of them with objects. Do i
have to make any special thing in order to get them released back to
the Memory Manager? For instantec.. is it enough to do a clear() in a
dictionary?

-- Cesar

 

[Fredrik Lundh]

I just checked the comsuptiom with the 'top' unix util. I am procesing
html docs and the amount of memory rises continiously.

what library are you using for this ?

I am using a lot of lists and docs. Some of them with objects. Do i
have to make any special thing in order to get them released back to
the Memory Manager? For instantec.. is it enough to do a clear() in a
dictionary?

the garbage collector should take care of that, unless you're doing something
fishy. maybe you're keeping pointers to documents you've already processed
in some data structure somewhere?

</F>