Memory Management

Discussion in 'Perl Misc' started by Shiraz, Apr 27, 2007.

  1. Shiraz

    Shiraz Guest

    I am trying to set aside some memory for my cache (hash/array etc) in
    my script.
    I use the ulimit and then monitor used mem with Devel::Size and
    destroy when i am close to my limit. Any ideas on how to do this
    better? Can anyone please point me to a memory management routing that
    is better than what i am using.
    Thanks
    Shiraz, Apr 27, 2007
    #1
    1. Advertising

  2. On 4/27/2007 1:26 PM, Shiraz wrote:
    > I am trying to set aside some memory for my cache (hash/array etc) in my
    > script. I use the ulimit and then monitor used mem with Devel::Size and
    > destroy when i am close to my limit. Any ideas on how to do this better?


    You don't really do memory management in Perl. Proper programming can reduce the
    amount of memory that you use, though. Have you read "perldoc -q memory?"

    -mjc
    Michael Carman, Apr 27, 2007
    #2
    1. Advertising

  3. Shiraz

    Shiraz Guest

    On Apr 27, 5:22 pm, Michael Carman <> wrote:
    > On 4/27/2007 1:26 PM, Shiraz wrote:
    >
    > > I am trying to set aside some memory for my cache (hash/array etc) in my
    > > script. I use the ulimit and then monitor used mem with Devel::Size and
    > > destroy when i am close to my limit. Any ideas on how to do this better?

    >
    > You don't really do memory management in Perl. Proper programming can reduce the
    > amount of memory that you use, though. Have you read "perldoc -q memory?"
    >
    > -mjc


    Maybe I worded my question wrong and it should be Memory Allocation
    and monitor.

    I know I can allocate memory space by using ` $#arr = 999 ` but i am
    not sure how much memory it consumes. Basically my goal is to figure
    out how to set the max memory a variable can use and how to monitor
    when the max has been reached.

    I did read the perldoc but that didnt tell me anything about reserving
    memory, just to be carefull with it.
    Shiraz, Apr 27, 2007
    #3
  4. On 4/27/2007 5:36 PM, Shiraz wrote:
    >
    > I know I can allocate memory space by using ` $#arr = 999 ` but i am
    > not sure how much memory it consumes. Basically my goal is to figure
    > out how to set the max memory a variable can use and how to monitor
    > when the max has been reached.


    '$#arr = 999' will set the size of the array to 1000 elements. That could either
    extend or truncate the @arr depending on what length it was beforehand. It
    doesn't create a limit, though. Perl will extend the array if (e.g.) you were to
    push() another item onto the end.

    You can use Devel::Size to find out how much memory a variable is actually
    using, but that's independent of the length. An array of 1000 integers would
    take less space than an array of 1000 strings. Array elements can be any scalar
    value which means you can create an array of hashes of hashes of arrays of...

    If you *really* need to continuously monitor the size of an array you could
    create a tie() class that used Devel::Size to check the memory consumption every
    time you added or modified an element and warn() when it exceeds some threshold.
    It would absolutely clobber your performance, though, and I doubt that you
    really need to do that. We can probably provide better advice if you tell us
    what your real task is.

    -mjc
    Michael Carman, Apr 28, 2007
    #4
  5. Shiraz

    Shiraz Guest

    On Apr 27, 9:41 pm, Michael Carman <> wrote:
    > On 4/27/2007 5:36 PM, Shiraz wrote:
    >
    >
    >
    > > I know I can allocate memory space by using ` $#arr = 999 ` but i am
    > > not sure how much memory it consumes. Basically my goal is to figure
    > > out how to set the max memory a variable can use and how to monitor
    > > when the max has been reached.

    >
    > '$#arr = 999' will set the size of the array to 1000 elements. That could either
    > extend or truncate the @arr depending on what length it was beforehand. It
    > doesn't create a limit, though. Perl will extend the array if (e.g.) you were to
    > push() another item onto the end.
    >
    > You can use Devel::Size to find out how much memory a variable is actually
    > using, but that's independent of the length. An array of 1000 integers would
    > take less space than an array of 1000 strings. Array elements can be any scalar
    > value which means you can create an array of hashes of hashes of arrays of...
    >
    > If you *really* need to continuously monitor the size of an array you could
    > create a tie() class that used Devel::Size to check the memory consumption every
    > time you added or modified an element and warn() when it exceeds some threshold.
    > It would absolutely clobber your performance, though, and I doubt that you
    > really need to do that. We can probably provide better advice if you tell us
    > what your real task is.
    >
    > -mjc




    I need to keep a hash of arrays in memory and keep adding things to it
    and when the hash has so many things in it that it can cause a perl
    out of memory error, i want to prune it. almost like a cache.
    Shiraz, Apr 28, 2007
    #5
  6. On 4/27/2007 10:18 PM, Shiraz wrote:
    >
    > I need to keep a hash of arrays in memory and keep adding things to it and
    > when the hash has so many things in it that it can cause a perl out of memory
    > error, i want to prune it. almost like a cache.


    So whatever gets flushed from the cache can be recalculated if necessary? You
    haven't said how you're filling it, but take a look at the Memoize and
    Memoize::Expire modules on CPAN. They *might* do what you want.

    If not, my advice is to create a class for your cache with get/set methods. When
    it gets too big, start undef'ing entries. Devel::Size will give you accurate
    results but be slow. If your array refs are of roughly equal size, you could use
    scalar keys on the hash for a faster (but crude) estimate.

    -mjc
    Michael Carman, Apr 28, 2007
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. penguinista

    Embeded perl & memory management

    penguinista, Jun 30, 2004, in forum: Perl
    Replies:
    0
    Views:
    592
    penguinista
    Jun 30, 2004
  2. Welman Jordan

    memory management problem during debugging

    Welman Jordan, Jan 5, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    307
    S. Justin Gengo
    Jan 5, 2004
  3. Floris van Haaster

    Project management / bug management

    Floris van Haaster, Sep 23, 2005, in forum: ASP .Net
    Replies:
    3
    Views:
    1,229
    Jon Paal
    Sep 23, 2005
  4. pouet
    Replies:
    2
    Views:
    737
    Will Hartung
    Jul 30, 2004
  5. Matt Oefinger
    Replies:
    0
    Views:
    205
    Matt Oefinger
    Jun 25, 2003
Loading...

Share This Page