Memoizing and WeakValueDictionary

[Paul McGuire]

Just wanted to share some experience I had in doing some memory and
performance tuning of a graphics script. I've been running some long-
running scripts on high-resolution images, and added memoizing to
optimize/minimize object creation (my objects are immutable, so better
to reuse objects from a cache than constantly create and discard
instances). This helped my script early on, but as the data files got
larger and larger, and runs got longer and longer, the object memoize
cache started to suck up some serious memory, and performance started
to degrade for a different reason - low memory -> page thrashing ->
program crawling.

It occurred to me that in my program, a given object may get used for
a while, but once its use declines, it doesn't come back for a while.
At first blush, I thought about using some sort of LRU cache, but
first I tried the WeakValueDictionary from the wkref module. This
worked great!

Now I get fairly optimal reuse of my immutable instances, but my
object cache doesn't grow without bounds.

Does this sound like a correct interpretation of this behavior? If
so, it would seem that WeakValueDictionary would be a good
recommendation to go along with any memoizing implementations.

 

[hsoft]

Just wanted to share some experience I had in doing some memory and
performance tuning of a graphics script.  I've been running some long-
running scripts on high-resolution images, and added memoizing to
optimize/minimize object creation (my objects are immutable, so better
to reuse objects from a cache than constantly create and discard
instances).  This helped my script early on, but as the data files got
larger and larger, and runs got longer and longer, the object memoize
cache started to suck up some serious memory, and performance started
to degrade for a different reason - low memory -> page thrashing ->
program crawling.

It occurred to me that in my program, a given object may get used for
a while, but once its use declines, it doesn't come back for a while.
At first blush, I thought about using some sort of LRU cache, but
first I tried the WeakValueDictionary from the wkref module.  This
worked great!

Now I get fairly optimal reuse of my immutable instances, but my
object cache doesn't grow without bounds.

Does this sound like a correct interpretation of this behavior?  If
so, it would seem that WeakValueDictionary would be a good
recommendation to go along with any memoizing implementations.

It depends on what you use memoizing for. If your memoized function
return data from a dataset, then yeah, you want to use a weakref
scheme because when such data is removed from the dataset, you want
the memory associated to it to be freed. However, if you use memoizing
on a function that creates the return value, you don't want to use a
weakref scheme, because it makes the memoizing useless: If the caller
of the memoized function doesn't keep a reference to the returned
value, there will be no memoizing taking place for the next call of
the function with the same arguments.

To make it short: it depends.