size limits of arrays/vectors ?

Discussion in 'Java' started by William Zumwalt, Apr 3, 2004.

  1. Hey all,

    I'm writing a science application that was previously written in Fortan
    and it has arrays that literally had millions of elements. Fortunately
    the elements were integers. I'm moving it to Java because of the GUI and
    portability is a must. Can Java handle Vectors (or at least primitive
    arrays) of this size? The machine certainly has the resources.
    William Zumwalt, Apr 3, 2004
    #1
    1. Advertising

  2. William Zumwalt

    Lee Weiner Guest

    In article <>, William
    Zumwalt <> wrote:
    >I'm writing a science application that was previously written in Fortan
    >and it has arrays that literally had millions of elements. Fortunately
    >the elements were integers. I'm moving it to Java because of the GUI and
    >portability is a must. Can Java handle Vectors (or at least primitive
    >arrays) of this size? The machine certainly has the resources.


    It took me about 90 seconds to test this out, probably less time than it took
    you to post the question.

    public static void main ( String[] args )
    {
    Vector vec = new Vector();
    for( int i = 0; i < 2500000; i++ )
    vec.add( new Integer( i ) );
    }

    This one worked. When I changed the number to 3,000,000, I got an OutOfMemory
    Exception.

    Lee Weiner
    lee AT leeweiner DOT org
    Lee Weiner, Apr 3, 2004
    #2
    1. Advertising

  3. William Zumwalt

    Roedy Green Guest

    On Sat, 03 Apr 2004 01:13:17 GMT, William Zumwalt
    <> wrote or quoted :

    >I'm writing a science application that was previously written in Fortan
    >and it has arrays that literally had millions of elements. Fortunately
    >the elements were integers. I'm moving it to Java because of the GUI and
    >portability is a must. Can Java handle Vectors (or at least primitive
    >arrays) of this size? The machine certainly has the resources.


    Java ints are 32 bits so arrays and Vectors can hold +2,147,483,647
    elements [2^31-1] aka Integer.MAX_VALUE.

    You eventually bang into the addressing limit of your hardware.

    Sun is offering 64 bit workstations at a discount to help you over
    that hump.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
    Roedy Green, Apr 3, 2004
    #3
  4. William Zumwalt

    Roedy Green Guest

    On Sat, 03 Apr 2004 02:48:26 GMT, (Lee Weiner) wrote or
    quoted :

    >When I changed the number to 3,000,000, I got an OutOfMemory
    >Exception.


    which could be extended with a java.exe command line option. See
    http://mindprod.com/jgloss/javaexe.html

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
    Roedy Green, Apr 3, 2004
    #4
  5. William Zumwalt

    Chris Uppal Guest

    Lee Weiner wrote:

    > Vector vec = new Vector();
    > for( int i = 0; i < 2500000; i++ )
    > vec.add( new Integer( i ) );


    Argh!!! Don't even *consider* using Vectors of Integers to hold huge datasets.

    Consider the memory reqs (assuming the typical 8-byte overhead for an object
    header):

    For an array of 2500000 ints:
    2,500,000 * 4
    = 10 MBytes (plus a little bit for the array header).

    For a Vector (or any other collection) of 2500000 Integers:
    2,500,000 * (4 + 8) for the Integer objects.
    2,500,000 * 4 for the internal array of refs to the Integers
    = 50 MBytes (plus a little bit for the Vector itself).

    And the difference will be even bigger on a 64-bit JVM.

    That's not to mention the massive extra load that 2.5 million extra object will
    put on the garbage collector.

    Nor does it mention the horrible way that 2.5 million pointers will largely
    destroy any cache coherence in memory accesses.

    (Of course, the same observations go for Vectors of Floats, Doubles, etc).

    Lee, William, I'm sorry if I'm saying things that are already obvious to you.
    But I've seen several recent posts by people pulling exactly this kind of trick
    (and then wondering why they had problems!), so I don't think the point can be
    made strongly enough.

    (And it also suggests that there's something wrong with the way that we are
    teaching programming, but that's another story).

    -- chris
    Chris Uppal, Apr 3, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Joe Van Meer
    Replies:
    2
    Views:
    632
    Peter O'Reilly
    May 5, 2004
  2. Paminu

    No limits in arrays??

    Paminu, Oct 12, 2005, in forum: C Programming
    Replies:
    17
    Views:
    513
    Keith Thompson
    Oct 13, 2005
  3. Replies:
    3
    Views:
    694
    Shadowman
    Mar 26, 2008
  4. Philipp
    Replies:
    21
    Views:
    1,127
    Philipp
    Jan 20, 2009
  5. Guest
    Replies:
    0
    Views:
    439
    Guest
    Sep 14, 2005
Loading...

Share This Page