Hooking into Python's memory management

Discussion in 'Python' started by Daniel Neilson, May 4, 2011.

  1. Hello,
    I'm hoping that there will be someone here with sufficient expertise
    to answer a question on Python 3 for me.

    I work in the Computer Science department at a large Canadian
    University. We are currently doing a feasibility analysis for switching
    to using Python in our first year major-stream courses.

    Part of our first year curriculum requires that students be exposed to
    explicit dynamic memory allocation in the form of C++'s new/delete, C's
    malloc/free, etc. I realize that Python is garbage collected, and that
    there is never a need to explicitly allocate & deallocate objects.
    However, I am trying to determine whether or not it is possible to
    simulate that behaviour within Python via a module for the purposes of
    instruction.

    For these purposes, I would like to know whether it is possible within
    Python 3 to write a Python-only module that, essentially, hooks into the
    "constructor" and "destructor" of many of the Python built-in types
    (specifically list, dictionary, set, tuple, and string) so that the
    module can:
    1) Maintain a list of object id()'s for objects that have been
    created. Ideally, this list would also contain the file & line number
    where the object was created.
    2) Provide a "deallocate" function that will remove a given object's
    id() from the list from (1).
    3) Print out an error message if the python script terminates with a
    non-empty list from (1). Preferably with a list of objects that are
    still "allocated."

    Baring a Python-only module, would this behaviour be possible to add
    via a C-language module?

    A module that hooked in to all memory allocation, and inspected the
    type of the object being allocated to conditionally add the object's
    id() to the list would also suffice.

    In either case, if such a module is possible, any pointers you could
    provide regarding how to implement such a module would be appreciated.

    Thank you for your time,
    Daniel
     
    Daniel Neilson, May 4, 2011
    #1
    1. Advertising

  2. Daniel Neilson wrote:

    > 1) Maintain a list of object id()'s for objects that have been created.
    > Ideally, this list would also contain the file & line number where the
    > object was created.


    > 2) Provide a "deallocate" function that will remove a given object's
    > id() from the list from (1).


    > 3) Print out an error message if the python script terminates with a
    > non-empty list from (1). Preferably with a list of objects that are
    > still "allocated."


    I don't think this will work out quite the way you seem to be
    imagining. Consider that Python creates a lot of objects behind
    the scenes without you doing anything specific to "allocate"
    them. Examples include the dicts holding module and class
    namespaces, all the strings representing names in loaded code,
    ints, floats and strings representing literals in the code,
    etc.

    If you automatically add any object of any of these types
    to the "allocated" list, these implicitly-created objects will
    all still be present in it when the program terminates, and
    the student will be told off for failing to deallocate them.

    An alternative might be to provide an allocate() function which
    the student is expected to use on any explicitly-created object.
    But there wouldn't be any way of enforcing this.

    Personally I think the whole idea is misguided, and it would be
    better to teach these concepts in the context of a language
    where they actually matter, which these days more or less
    means C. If teaching C in the first year in parallel with
    Python is considered too steep, then leave explicit memory
    management until later in the curriculum.

    (It's really a shame that Pascal is not taught any more. It
    provided a fairly clean environment for teaching about things
    like this, without getting so close to the machine that you
    get your nosed rubbed in segfaults.)

    --
    Greg
     
    Gregory Ewing, May 5, 2011
    #2
    1. Advertising

  3. Daniel Neilson

    scattered Guest

    On May 4, 12:51 pm, Daniel Neilson <> wrote:
    > Hello,
    >   I'm hoping that there will be someone here with sufficient expertise
    > to answer a question on Python 3 for me.
    >
    >   I work in the Computer Science department at a large Canadian
    > University. We are currently doing a feasibility analysis for switching
    > to using Python in our first year major-stream courses.
    >
    >   Part of our first year curriculum requires that students be exposed to
    > explicit dynamic memory allocation in the form of C++'s new/delete, C's
    > malloc/free, etc. I realize that Python is garbage collected, and that
    > there is never a need to explicitly allocate & deallocate objects.
    > However, I am trying to determine whether or not it is possible to
    > simulate that behaviour within Python via a module for the purposes of
    > instruction.
    >
    >   For these purposes, I would like to know whether it is possible within
    > Python 3 to write a Python-only module that, essentially, hooks into the
    > "constructor" and "destructor" of many of the Python built-in types
    > (specifically list, dictionary, set, tuple, and string) so that the
    > module can:
    >   1) Maintain a list of object id()'s for objects that have been
    > created. Ideally, this list would also contain the file & line number
    > where the object was created.
    >   2) Provide a "deallocate" function that will remove a given object's
    > id() from the list from (1).
    >   3) Print out an error message if the python script terminates with a
    > non-empty list from (1). Preferably with a list of objects that are
    > still "allocated."
    >
    >   Baring a Python-only module, would this behaviour be possible to add
    > via a C-language module?
    >
    >   A module that hooked in to all memory allocation, and inspected the
    > type of the object being allocated to conditionally add the object's
    > id() to the list would also suffice.
    >
    >   In either case, if such a module is possible, any pointers you could
    > provide regarding how to implement such a module would be appreciated.
    >
    > Thank you for your time,
    >   Daniel


    Maybe you can simulate a heap. Create a heap class where a heap-object
    maintains an internal list of fixed length (whose length is determined
    by a constructor)for its heap and a list of free blocks in this heap.
    It can have methods for allocating and deallocating objects in the
    heap. Perhaps some sort of dictionary with strings representing
    pointers as keys and indices of corresponding allocated blocks as
    values. Subscript-out of range errors in the internal heap would
    correspond to segmentation errors. The heap-class could catch this
    error and inform the student. It could also have methods which catch
    things like dangling pointers and memory leaks. Perhaps a method to
    display a schematic of the heap. Maybe something as simple as printing
    something like

    |||||____|||__|||______||||____________

    where ||| represents used blocks and ____ represents free space.
    Displaying this could show the problem of heap-fragmentation.

    Once you get the interface down - assign some homework where the
    students need to implement a simple algorithm which requires dynamic
    memory alocation. The catch is that the students are forbidden from
    using things like Python lists directly in their code but instead have
    to get by with using a single instance of this heap object for their
    data storage needs. Make a bit of a game out of it.

    I obviously haven't worked out the details, but I suspect that this
    sort of thing is both easier and more pedagogically sound (since you
    can tailor the error messages) then what you were looking for.
     
    scattered, May 5, 2011
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dave A
    Replies:
    2
    Views:
    6,866
    Dave A
    Nov 29, 2005
  2. Kevin Jackson

    ASP.NET 1.1 hooking into Control.Init

    Kevin Jackson, Oct 16, 2006, in forum: ASP .Net
    Replies:
    0
    Views:
    301
    Kevin Jackson
    Oct 16, 2006
  3. Sean
    Replies:
    1
    Views:
    647
    Salt_Peter
    Dec 6, 2006
  4. Replies:
    1
    Views:
    339
    Roedy Green
    Jun 22, 2008
  5. Mike
    Replies:
    0
    Views:
    417
Loading...

Share This Page