problem with multiple Py_Initialize/Py_Finalize calls

[Andrew Trevorrow]

Hi folks,

I've only been using Python for a few weeks, so please be gentle.
Lovely language by the way -- I really enjoying using it -- but
I've struck a very nasty problem when running Python scripts from
my app.

Summary:

Multiple Py_Initialize/Py_Finalize calls result in bad memory leaks,
and Py_Finalize gradually gets slower and slower.

The problem occurs on both Mac OS 10.3.9 and Windows 2000.
I'm using Python 2.3 on the Mac and 2.4.2 on Windows.

More details:

I'm co-writing an open source, cross-platform Life app called Golly
(more details at http://golly.sourceforge.net/ if you're interested).
The GUI is written in C++ using wxWidgets, in case that matters.

I've added scripting capability, ie. embedded Python, so users can
run .py scripts from within the app to do various fancy things.
This is all working very nicely, but then I noticed serious memory
problems after running many scripts (or the same script many times).
In fact, running a single complicated script that repeatedly builds
very large lists (representing large Life patterns) can cause a
leak of many megabytes!

After much hair-pulling and experimentation I decided to write a
very simple routine that doesn't even execute a script:

void TestPython()
{
for (int i=0; i<1000; i++) {
Py_Initialize();
Py_Finalize();
}
}

After running this routine on my Mac, the app's memory use increased
by about 12.9MB; ie. a 12.9K leak per Py_Initialize/Py_Finalize pair.
Similar result on Windows: the app's memory increased by 11MB.

Curiously, on both machines, the Py_Finalize() call takes longer and
longer to complete whatever it's doing. The above routine takes a
few *minutes* to complete on my 400MHz Mac.

Note that if I execute a script between Py_Initialize and Py_Finalize
by calling

PyRun_SimpleString("execfile('foo.py')");

then much larger memory leaks will occur. In fact it seems that
Py_Finalize is not actually deallocating any memory.

Any idea what is going wrong? Am I doing something stupid?
If Py_Initialize/Py_Finalize should not be called multiple times
then how can I tell Python to free all the memory used by an
executed script?

Yours in desperation, Andrew
(e-mail address removed)