Max size of an array used in perl.

[rajendra]

Hello All,
I would like to know the max size that an array can store in terms of bytes.

With Rgds,
Raj

 

[Paul Lalli]

Hello All,
I would like to know the max size that an array can store in terms of bytes.

There is no built-in limit in the language. It is limited only by the
amount of memory your computer system has available.

Paul Lalli

 

[gf]

There is no built-in limit in the language. It is limited only by the
amount of memory your computer system has available.

And how willing you are to let your machine go to a crawl if you
exceed the available physical RAM.

Oses these days aren't limited by RAM. They'll allocate disk as swap
space, which frees RAM to hold more, which then gives Perl the go-
ahead to load more data, which then gets swapped back out to disk.
Disk is slower than RAM so the machine will go to a crawl.

And, unfortunately, there are some things that are easy to do in Perl
that can put you in that situation, like slurping big data files
without checking to see how big they are first or trying to hold huge
structures in memory instead of reverting to using a database.

Back to the original poster's question -

Knowing that Perl will create an array big enough to fill memory
should prompt more specific questions, like "how many array entries
can I effectively access if they are of type X?", and "Is there a
better way to store and access data type Y?" and let us know what
you're going to store or access or try to do with it.

"How big" is too generic and leads to generic answers that don't
really help you.

 

[Joe Smith]

Hello All,
I would like to know the max size that an array can store in terms of bytes.

As much as your memory system allows.

WinXP% uname -a
CYGWIN_NT-5.1 wiggin 1.5.23(0.156/4/2) 2006-12-19 10:52 i686 Cygwin
WinXP% perl -le '$,=$a=",";for(;{print time()-$^T,$b++, length $a; $a.=$a}'
0,0,1
0,1,2
0,2,4
0,3,8
0,4,16
0,5,32
0,6,64
0,7,128
0,8,256
0,9,512
0,10,1024
0,11,2048
0,12,4096
0,13,8192
0,14,16384
0,15,32768
0,16,65536
0,17,131072
0,18,262144
0,19,524288
0,20,1048576
0,21,2097152
0,22,4194304
0,23,8388608
0,24,16777216
0,25,33554432
1,26,67108864
1,27,134217728
1,28,268435456
2,29,536870912
Out of memory!

There it took 2 seconds to go from 256M to 512M

On an 800 MHz P-III with 256M RAM:
0,25,33554432
0,26,67108864
1,27,134217728
2,28,268435456
21,29,536870912
155,30,1073741824
panic: realloc at -e line 1.

That is 21 seconds to double after exceeding physical memory and
155 seconds to double after that. After 1GB, it used up all the
available swap space on that poor little machine.

I haven't tried that test on a 16GB machine where perl is compiled with use64bitall.
-Joe