Zipimport leaks memory?


A

ashwin.u.rao

Hi,

We are currently trying to identify and fix all the memory leaks by just doing Py_Initialize-PyRun_SimpleFile(some simple script)-Py_Finalize and found that there are around 70 malloc-ed blocks which are not freed. One of the significant contributor to this number is the 'files' object in ZipImporter. I am not able to identify the reason for this leak and was wondering if anyone on this list would help me out here.

So, here goes :

Since we have a zip file in our sys.path, this object is initialized and added to the zip_directory_cache dict during Py_Initialize. One point to note here is that there is no DECREF on the 'files' object after adding it to the zip_directory_cache dict.

When a module in a directory is imported(encoding.alias) then the reference count of 'files' is increased. When this module is unloaded during Py_Finalize-PyImport_Cleanup, the ref count of files object is decremented properly. So at the end of Py_Finalize the files object still has one reference count which is a result of it being an entry in the zip_directory_cache.

To summarize :

ZipImporter->files - zipimporter_init function:

Py_Initialize -
Read files from zip - ref_count -1
Add files to zip_directory_cache - ref count - 2
Import some modules from different directory in zip - ref count - 3

Py_Finalze -
PyImport_Cleanup - sub-directory - ref-count - 2
PyImport_Cleanup - main directory - ref-count - 1

So the reference count of 'files' is still 1 after Py_Finalize. The zip_directory_cache is the dict that has this 'files' object and this dict is added to the module during zipimport module initialization. This dict is destroyed during _PyModule_Clear(called by PyImport_Cleanup) but the 'files' object's reference count is not decremented.

Please excuse me if I have misinterpreted the whole logic. Any pointers towards solving this would be great.

Thank you.

~Ashwin
 
Ad

Advertisements


Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top