Memory Management

S

Shiraz

I am trying to set aside some memory for my cache (hash/array etc) in
my script.
I use the ulimit and then monitor used mem with Devel::Size and
destroy when i am close to my limit. Any ideas on how to do this
better? Can anyone please point me to a memory management routing that
is better than what i am using.
Thanks
 
M

Michael Carman

I am trying to set aside some memory for my cache (hash/array etc) in my
script. I use the ulimit and then monitor used mem with Devel::Size and
destroy when i am close to my limit. Any ideas on how to do this better?

You don't really do memory management in Perl. Proper programming can reduce the
amount of memory that you use, though. Have you read "perldoc -q memory?"

-mjc
 
S

Shiraz

You don't really do memory management in Perl. Proper programming can reduce the
amount of memory that you use, though. Have you read "perldoc -q memory?"

-mjc

Maybe I worded my question wrong and it should be Memory Allocation
and monitor.

I know I can allocate memory space by using ` $#arr = 999 ` but i am
not sure how much memory it consumes. Basically my goal is to figure
out how to set the max memory a variable can use and how to monitor
when the max has been reached.

I did read the perldoc but that didnt tell me anything about reserving
memory, just to be carefull with it.
 
M

Michael Carman

I know I can allocate memory space by using ` $#arr = 999 ` but i am
not sure how much memory it consumes. Basically my goal is to figure
out how to set the max memory a variable can use and how to monitor
when the max has been reached.

'$#arr = 999' will set the size of the array to 1000 elements. That could either
extend or truncate the @arr depending on what length it was beforehand. It
doesn't create a limit, though. Perl will extend the array if (e.g.) you were to
push() another item onto the end.

You can use Devel::Size to find out how much memory a variable is actually
using, but that's independent of the length. An array of 1000 integers would
take less space than an array of 1000 strings. Array elements can be any scalar
value which means you can create an array of hashes of hashes of arrays of...

If you *really* need to continuously monitor the size of an array you could
create a tie() class that used Devel::Size to check the memory consumption every
time you added or modified an element and warn() when it exceeds some threshold.
It would absolutely clobber your performance, though, and I doubt that you
really need to do that. We can probably provide better advice if you tell us
what your real task is.

-mjc
 
S

Shiraz

'$#arr = 999' will set the size of the array to 1000 elements. That could either
extend or truncate the @arr depending on what length it was beforehand. It
doesn't create a limit, though. Perl will extend the array if (e.g.) you were to
push() another item onto the end.

You can use Devel::Size to find out how much memory a variable is actually
using, but that's independent of the length. An array of 1000 integers would
take less space than an array of 1000 strings. Array elements can be any scalar
value which means you can create an array of hashes of hashes of arrays of...

If you *really* need to continuously monitor the size of an array you could
create a tie() class that used Devel::Size to check the memory consumption every
time you added or modified an element and warn() when it exceeds some threshold.
It would absolutely clobber your performance, though, and I doubt that you
really need to do that. We can probably provide better advice if you tell us
what your real task is.

-mjc



I need to keep a hash of arrays in memory and keep adding things to it
and when the hash has so many things in it that it can cause a perl
out of memory error, i want to prune it. almost like a cache.
 
M

Michael Carman

I need to keep a hash of arrays in memory and keep adding things to it and
when the hash has so many things in it that it can cause a perl out of memory
error, i want to prune it. almost like a cache.

So whatever gets flushed from the cache can be recalculated if necessary? You
haven't said how you're filling it, but take a look at the Memoize and
Memoize::Expire modules on CPAN. They *might* do what you want.

If not, my advice is to create a class for your cache with get/set methods. When
it gets too big, start undef'ing entries. Devel::Size will give you accurate
results but be slow. If your array refs are of roughly equal size, you could use
scalar keys on the hash for a faster (but crude) estimate.

-mjc
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top