Handling a 450,000x450,000 array with Perl

Discussion in 'Perl Misc' started by francescomoi@usa.com, Oct 30, 2006.

  1. Guest

    Hi.

    I'd like to create a program to handle data (natural numbers)
    within a 450,000 x 450,000 array. I'm considering Perl, but
    I don't know is it's powerful enough to manage it.

    Thank you very much.
     
    , Oct 30, 2006
    #1
    1. Advertising

  2. Ingo Menger Guest

    wrote:
    > Hi.
    >
    > I'd like to create a program to handle data (natural numbers)
    > within a 450,000 x 450,000 array. I'm considering Perl, but
    > I don't know is it's powerful enough to manage it.


    Perl is, but you'll find few computers that could store 202 billion
    "natural numbers".
    Remember, a PC has a RAM of around 2 billion bytes, give or take a few.
    You could hold a maximum of 16 billion natural numbers as long as they
    are 0 or 1. But, 16 billion is still not 202 billion.

    In any case, this will be a system where native integers have 64 bits.
     
    Ingo Menger, Oct 30, 2006
    #2
    1. Advertising

  3. Guest

    Hi.

    Thank you very much for your quick answers. I try to handle data from a
    450x450 meter surface.

    I've got 100,000 elements distributed on this surface, and each
    square millimeter can hold one or more elements. So the array to create
    is a sparse one: most of square millimeters holds zero elements, and
    the rest, 20 or less.

    I'm trying to find the square meter with more elements inside.

    Thank you very much.

    On Oct 30, 3:24 pm, "Ingo Menger" <> wrote:
    > wrote:
    > > Hi.

    >
    > > I'd like to create a program to handle data (natural numbers)
    > > within a 450,000 x 450,000 array. I'm considering Perl, but
    > > I don't know is it's powerful enough to manage it.Perl is, but you'll find few computers that could store 202 billion

    > "natural numbers".
    > Remember, a PC has a RAM of around 2 billion bytes, give or take a few.
    > You could hold a maximum of 16 billion natural numbers as long as they
    > are 0 or 1. But, 16 billion is still not 202 billion.
    >
    > In any case, this will be a system where native integers have 64 bits.
     
    , Oct 30, 2006
    #3
  4. Ted Zlatanov Guest

    On 30 Oct 2006, bugbear@trim_papermule.co.uk_trim wrote:

    wrote:
    >> Thank you very much for your quick answers. I try to handle data from a
    >> 450x450 meter surface.
    >>
    >> I've got 100,000 elements distributed on this surface, and each
    >> square millimeter can hold one or more elements. So the array to create
    >> is a sparse one: most of square millimeters holds zero elements, and
    >> the rest, 20 or less.
    >>
    >> I'm trying to find the square meter with more elements inside.

    >
    > Ah. OK.
    >
    > Your question should really be "what data structure should
    > I use to..."
    >
    > It's not really a perl question at all, it's
    > really an algorithm and data structure question.


    Well no, some languages have sparse arrays and some don't. So this is
    definitely a language-specific question, the data structure is pretty
    obvious.

    For Perl 5, you want either a relevant CPAN module or just use hashes
    that combine x and y uniquely. See
    http://search.cpan.org/search?query=sparse&mode=all for some ideas.

    Ted
     
    Ted Zlatanov, Oct 30, 2006
    #4
  5. Guest

    wrote:
    > Hi.
    >
    > Thank you very much for your quick answers. I try to handle data from a
    > 450x450 meter surface.
    >
    > I've got 100,000 elements distributed on this surface, and each
    > square millimeter can hold one or more elements. So the array to create
    > is a sparse one: most of square millimeters holds zero elements, and
    > the rest, 20 or less.
    >
    > I'm trying to find the square meter with more elements inside.


    So then, I assume the square meters' must be rotationally aligned parallel
    to the axes of the millimeter scale. But can one square meter "slide" by
    a millimeter in order to make another square meter, or does it have to
    slide by a full meter? I.e. can the potential square meters overlap each
    other?

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Oct 30, 2006
    #5
  6. -berlin.de Guest

    bugbear <bugbear@trim_papermule.co.uk_trim> wrote in comp.lang.perl.misc:
    > wrote:
    > > Hi.
    > >
    > > Thank you very much for your quick answers. I try to handle data from a
    > > 450x450 meter surface.
    > >
    > > I've got 100,000 elements distributed on this surface, and each
    > > square millimeter can hold one or more elements. So the array to create
    > > is a sparse one: most of square millimeters holds zero elements, and
    > > the rest, 20 or less.
    > >
    > > I'm trying to find the square meter with more elements inside.


    "The square meter" as in, divide the surface into meter-squares like
    a checkerboard and find the one with the most elements? Or as in,
    find the one-meter square that holds the most elements? The first
    ones must start on integer meter-coordinates, the other can start
    anywhere. The second search space is much larger.

    > Ah. OK.
    >
    > Your question should really be "what data structure should
    > I use to..."
    >
    > It's not really a perl question at all, it's
    > really an algorithm and data structure question.


    There is, however, a likely Perl answer: Use a hash. It is often
    the most straight-forward way to represent a sparse anything.

    Here one could use blank-separated integer coordinates (in mm) for
    keys and the (nonzero) number of elements for values. Values of
    zero are *not* entered. Some care may be needed not to autovivify
    these non-entries.

    Anno
     
    -berlin.de, Oct 30, 2006
    #6
  7. Peter Scott Guest

    On Mon, 30 Oct 2006 05:10:07 -0800, francescomoi wrote:
    > I'd like to create a program to handle data (natural numbers)
    > within a 450,000 x 450,000 array. I'm considering Perl, but
    > I don't know is it's powerful enough to manage it.


    Any time you want to work with a matrix of lots of numbers in Perl,
    consider http://pdl.perl.org/.

    --
    Peter Scott
    http://www.perlmedic.com/
    http://www.perldebugged.com/
     
    Peter Scott, Oct 31, 2006
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. rote
    Replies:
    3
    Views:
    439
    Mark Rae [MVP]
    Jan 24, 2008
  2. lector

    storing 1,000,000 records

    lector, Apr 6, 2008, in forum: C Programming
    Replies:
    19
    Views:
    482
    Barry Schwarz
    Apr 8, 2008
  3. Replies:
    1
    Views:
    454
  4. Replies:
    0
    Views:
    603
  5. bad_knee

    regex to convert 1000000 -> 1,000,000 ?

    bad_knee, Nov 14, 2003, in forum: Perl Misc
    Replies:
    39
    Views:
    342
    Tad McClellan
    Nov 19, 2003
Loading...

Share This Page