Hooking into Python's memory management

[Daniel Neilson]

Hello,
I'm hoping that there will be someone here with sufficient expertise
to answer a question on Python 3 for me.

I work in the Computer Science department at a large Canadian
University. We are currently doing a feasibility analysis for switching
to using Python in our first year major-stream courses.

Part of our first year curriculum requires that students be exposed to
explicit dynamic memory allocation in the form of C++'s new/delete, C's
malloc/free, etc. I realize that Python is garbage collected, and that
there is never a need to explicitly allocate & deallocate objects.
However, I am trying to determine whether or not it is possible to
simulate that behaviour within Python via a module for the purposes of
instruction.

For these purposes, I would like to know whether it is possible within
Python 3 to write a Python-only module that, essentially, hooks into the
"constructor" and "destructor" of many of the Python built-in types
(specifically list, dictionary, set, tuple, and string) so that the
module can:
1) Maintain a list of object id()'s for objects that have been
created. Ideally, this list would also contain the file & line number
where the object was created.
2) Provide a "deallocate" function that will remove a given object's
id() from the list from (1).
3) Print out an error message if the python script terminates with a
non-empty list from (1). Preferably with a list of objects that are
still "allocated."

Baring a Python-only module, would this behaviour be possible to add
via a C-language module?

A module that hooked in to all memory allocation, and inspected the
type of the object being allocated to conditionally add the object's
id() to the list would also suffice.

In either case, if such a module is possible, any pointers you could
provide regarding how to implement such a module would be appreciated.

Thank you for your time,
Daniel

 

[Gregory Ewing]

1) Maintain a list of object id()'s for objects that have been created.
Ideally, this list would also contain the file & line number where the
object was created.

2) Provide a "deallocate" function that will remove a given object's
id() from the list from (1).

3) Print out an error message if the python script terminates with a
non-empty list from (1). Preferably with a list of objects that are
still "allocated."

I don't think this will work out quite the way you seem to be
imagining. Consider that Python creates a lot of objects behind
the scenes without you doing anything specific to "allocate"
them. Examples include the dicts holding module and class
namespaces, all the strings representing names in loaded code,
ints, floats and strings representing literals in the code,
etc.

If you automatically add any object of any of these types
to the "allocated" list, these implicitly-created objects will
all still be present in it when the program terminates, and
the student will be told off for failing to deallocate them.

An alternative might be to provide an allocate() function which
the student is expected to use on any explicitly-created object.
But there wouldn't be any way of enforcing this.

Personally I think the whole idea is misguided, and it would be
better to teach these concepts in the context of a language
where they actually matter, which these days more or less
means C. If teaching C in the first year in parallel with
Python is considered too steep, then leave explicit memory
management until later in the curriculum.

(It's really a shame that Pascal is not taught any more. It
provided a fairly clean environment for teaching about things
like this, without getting so close to the machine that you
get your nosed rubbed in segfaults.)

 

[scattered]

Hello,
  I'm hoping that there will be someone here with sufficient expertise
to answer a question on Python 3 for me.

  I work in the Computer Science department at a large Canadian
University. We are currently doing a feasibility analysis for switching
to using Python in our first year major-stream courses.

  Part of our first year curriculum requires that students be exposed to
explicit dynamic memory allocation in the form of C++'s new/delete, C's
malloc/free, etc. I realize that Python is garbage collected, and that
there is never a need to explicitly allocate & deallocate objects.
However, I am trying to determine whether or not it is possible to
simulate that behaviour within Python via a module for the purposes of
instruction.

  For these purposes, I would like to know whether it is possible within
Python 3 to write a Python-only module that, essentially, hooks into the
"constructor" and "destructor" of many of the Python built-in types
(specifically list, dictionary, set, tuple, and string) so that the
module can:
  1) Maintain a list of object id()'s for objects that have been
created. Ideally, this list would also contain the file & line number
where the object was created.
  2) Provide a "deallocate" function that will remove a given object's
id() from the list from (1).
  3) Print out an error message if the python script terminates with a
non-empty list from (1). Preferably with a list of objects that are
still "allocated."

  Baring a Python-only module, would this behaviour be possible to add
via a C-language module?

  A module that hooked in to all memory allocation, and inspected the
type of the object being allocated to conditionally add the object's
id() to the list would also suffice.

  In either case, if such a module is possible, any pointers you could
provide regarding how to implement such a module would be appreciated.

Thank you for your time,
  Daniel

Maybe you can simulate a heap. Create a heap class where a heap-object
maintains an internal list of fixed length (whose length is determined
by a constructor)for its heap and a list of free blocks in this heap.
It can have methods for allocating and deallocating objects in the
heap. Perhaps some sort of dictionary with strings representing
pointers as keys and indices of corresponding allocated blocks as
values. Subscript-out of range errors in the internal heap would
correspond to segmentation errors. The heap-class could catch this
error and inform the student. It could also have methods which catch
things like dangling pointers and memory leaks. Perhaps a method to
display a schematic of the heap. Maybe something as simple as printing
something like

|||||____|||__|||______||||____________

where ||| represents used blocks and ____ represents free space.
Displaying this could show the problem of heap-fragmentation.

Once you get the interface down - assign some homework where the
students need to implement a simple algorithm which requires dynamic
memory alocation. The catch is that the students are forbidden from
using things like Python lists directly in their code but instead have
to get by with using a single instance of this heap object for their
data storage needs. Make a bit of a game out of it.

I obviously haven't worked out the details, but I suspect that this
sort of thing is both easier and more pedagogically sound (since you
can tailor the error messages) then what you were looking for.