Python memory management

[Marcelo A. Camelo]

Hi!

I will be presenting Python to an audience of game developers, mostly
C/C++ programmers. In my presentation I will talk about using python and
C/C++ extension instead of pure C/C++ to write commercial games and
content creation tools.

In a number of times I have presented these ideas to this same kind of
audience and they were received with cold skepticism, despite the fact
that some, including me, are already doing this with quite some success.

I've noticed that, from all their concerns, the most difficult to tackle
(for me, at least) is memory management.

Currently, the most popular platform for game applications are videogame
consoles. Memory on these special purpose machines are a premium
resource and one should used it judiciously. Python's RTE footprint and
memory fragmentation issues make it a hard sell for game programmers
(many tend to prefer LUA).

I would like to address this topic in my presentation, showing the real
picture about python memory habits and how to gain control over memory
management when using Python. So, I'm looking the references on the
subject. Also, I would like to hear comments from those with experience
in tweaking python's memory management.

Cheers,


Marcelo A. Camelo, M. Eng. - Project Leader
ESSS - Engineering Simulation and Scientific Software

E-mail: (e-mail address removed)
Phone: +55-48-334-8922

 

[Michael Hudson]

Currently, the most popular platform for game applications are videogame
consoles. Memory on these special purpose machines are a premium
resource and one should used it judiciously. Python's RTE footprint and
memory fragmentation issues make it a hard sell for game programmers
(many tend to prefer LUA).

Um, *I*, at least, would say they have a point. I love Python and
think it has many, many virtues, but being miserly with RAM is not one
of them (at least, for the current implementation).

If you're on a real PC, things are very different, of course.

Cheers,
mwh

 

[Steven Taschuk]

Quoth Marcelo A. Camelo:
[...]

Yes, I will be saying that. But, if I can't change the way Python
behaves, at least I want a way to diagnose what he is doing. I've found
some references about Pymalloc, now the default Python memory manager,
and how it can report statistics about memory usage, but the
documentation is a bit light on details. Any pointer to some resources
on the subject? [...]

There's Misc/SpecialBuilds.txt in the source distribution, which
documents several build-time options which could be used for such
investigations. (It assumes some familiarity with the internals.)

 

[Aahz]

I've noticed that, from all their concerns, the most difficult to tackle
(for me, at least) is memory management.

Currently, the most popular platform for game applications are videogame
consoles. Memory on these special purpose machines are a premium
resource and one should used it judiciously. Python's RTE footprint and
memory fragmentation issues make it a hard sell for game programmers
(many tend to prefer LUA).

As Michael Hudson said, those are real issues. Nevertheless, the
increasing power of game consoles renders the first issue less
important, and PyMalloc (standard in Python 2.3) does much to alleviate
the second issue.