Confused: Java6 32-bit vs. 64-bit

Discussion in 'Java' started by Hendrik Maryns, May 2, 2007.

  1. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    Hi all.

    I have this program that does some very memory intense computations
    converting logical formulae into tree automata. The automata get that
    big that after a while I get OutOfMemoryError. I have this particular
    formula that gave me problems the first time I started testing the
    program. I then rewrote part of it to make it consume less memory, and
    after 6 months of work, that formula still gives problems. However,
    just monday, suddenly it managed in computing the formula, with ‘only’
    1.2G of RAM.

    However, this only works if I tell Eclipse to use the 32-bit java6 jre,
    which I installed additionally to the 64-bit jdk that I use for development.

    More strangely, I cannot reproduce this behavior consistently: the first
    few times I tried today, it didn’t manage the formula, with neither
    32-bit not 64-bit versions. Now, after some switching around, I get the
    same result: the 32-bit jre manages the formula, the 64-bit one doesn’t.

    The result is reproducible on the command line.

    I also have a 1.5 version 64-bit, which seems to fail as well. And now
    that I am in a testing phase, I quickly installed a 32-bit 1.5, which
    didn’t manage either.

    In short, my question: does anyone have an explanation for this? Are
    there some things I can learn from this? Which part of my program
    should be tuned, such that maybe it becomes usable with less memory? Etc.

    Many thanks, H.
    - --
    Hendrik Maryns
    http://tcl.sfs.uni-tuebingen.de/~hendrik/
    ==================
    http://aouw.org
    Ask smart questions, get good answers:
    http://www.catb.org/~esr/faqs/smart-questions.html
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.5 (GNU/Linux)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

    iD8DBQFGOI50e+7xMGD3itQRAh9MAJ0XgLkiSvtkhJbjaFoz387ls1T4CACfRZDc
    xCsHREW4+v6FPtZMLFDbB3k=
    =UuJa
    -----END PGP SIGNATURE-----
     
    Hendrik Maryns, May 2, 2007
    #1
    1. Advertising

  2. Hendrik Maryns wrote:
    > -----BEGIN PGP SIGNED MESSAGE-----
    > Hash: SHA1
    >
    > Hi all.
    >
    > I have this program that does some very memory intense computations
    > converting logical formulae into tree automata. The automata get that
    > big that after a while I get OutOfMemoryError. I have this particular
    > formula that gave me problems the first time I started testing the
    > program. I then rewrote part of it to make it consume less memory, and
    > after 6 months of work, that formula still gives problems. However,
    > just monday, suddenly it managed in computing the formula, with ‘only’
    > 1.2G of RAM.
    >
    > However, this only works if I tell Eclipse to use the 32-bit java6 jre,
    > which I installed additionally to the 64-bit jdk that I use for development.
    >
    > More strangely, I cannot reproduce this behavior consistently: the first
    > few times I tried today, it didn’t manage the formula, with neither
    > 32-bit not 64-bit versions. Now, after some switching around, I get the
    > same result: the 32-bit jre manages the formula, the 64-bit one doesn’t.
    >
    > The result is reproducible on the command line.
    >
    > I also have a 1.5 version 64-bit, which seems to fail as well. And now
    > that I am in a testing phase, I quickly installed a 32-bit 1.5, which
    > didn’t manage either.
    >
    > In short, my question: does anyone have an explanation for this? Are
    > there some things I can learn from this? Which part of my program
    > should be tuned, such that maybe it becomes usable with less memory? Etc.


    Well, one obvious lesson is that bigger pointers take more space. Why
    use a 64 bit JDK if your intended memory allocation is small enough for
    a 32 bit one?

    Does anyone know the heap management algorithms for the JRE? Some memory
    allocation systems can lose space to fragmentation, so that the exact
    allocation order can affect the amount of memory used.

    Also, in a multi-threaded program the sets of objects that exist
    simultaneously can vary from run to run.

    The big picture is that your program, as currently constructed, is right
    on the edge, and will be very fragile unless you do something drastic to
    either increase the available memory or reduce the memory it uses.

    Patricia
     
    Patricia Shanahan, May 2, 2007
    #2
    1. Advertising

  3. On 02.05.2007 15:56, Patricia Shanahan wrote:
    > Hendrik Maryns wrote:
    >> -----BEGIN PGP SIGNED MESSAGE-----
    >> Hash: SHA1
    >>
    >> Hi all.
    >>
    >> I have this program that does some very memory intense computations
    >> converting logical formulae into tree automata. The automata get that
    >> big that after a while I get OutOfMemoryError. I have this particular
    >> formula that gave me problems the first time I started testing the
    >> program. I then rewrote part of it to make it consume less memory, and
    >> after 6 months of work, that formula still gives problems. However,
    >> just monday, suddenly it managed in computing the formula, with
    >> ‘only’
    >> 1.2G of RAM.
    >>
    >> However, this only works if I tell Eclipse to use the 32-bit java6 jre,
    >> which I installed additionally to the 64-bit jdk that I use for
    >> development.
    >>
    >> More strangely, I cannot reproduce this behavior consistently: the first
    >> few times I tried today, it didn’t manage the formula, with neither
    >> 32-bit not 64-bit versions. Now, after some switching around, I get the
    >> same result: the 32-bit jre manages the formula, the 64-bit one
    >> doesn’t.
    >>
    >> The result is reproducible on the command line.
    >>
    >> I also have a 1.5 version 64-bit, which seems to fail as well. And now
    >> that I am in a testing phase, I quickly installed a 32-bit 1.5, which
    >> didn’t manage either.
    >>
    >> In short, my question: does anyone have an explanation for this? Are
    >> there some things I can learn from this? Which part of my program
    >> should be tuned, such that maybe it becomes usable with less memory? Etc.

    >
    > Well, one obvious lesson is that bigger pointers take more space. Why
    > use a 64 bit JDK if your intended memory allocation is small enough for
    > a 32 bit one?
    >
    > Does anyone know the heap management algorithms for the JRE? Some memory
    > allocation systems can lose space to fragmentation, so that the exact
    > allocation order can affect the amount of memory used.


    I suggest to look into GC related JVM settings, e.g. -XX:NewRatio might
    help.

    http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp

    Some more links
    http://www-128.ibm.com/developerworks/ibm/library/i-gctroub/
    http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html

    > Also, in a multi-threaded program the sets of objects that exist
    > simultaneously can vary from run to run.
    >
    > The big picture is that your program, as currently constructed, is right
    > on the edge, and will be very fragile unless you do something drastic to
    > either increase the available memory or reduce the memory it uses.


    Definitively.

    robert
     
    Robert Klemme, May 2, 2007
    #3
  4. Hendrik Maryns

    Tom Hawtin Guest

    Hendrik Maryns wrote:
    >
    > I have this program that does some very memory intense computations
    > converting logical formulae into tree automata. The automata get that
    > big that after a while I get OutOfMemoryError. I have this particular


    > More strangely, I cannot reproduce this behavior consistently: the first
    > few times I tried today, it didn’t manage the formula, with neither
    > 32-bit not 64-bit versions. Now, after some switching around, I get the
    > same result: the 32-bit jre manages the formula, the 64-bit one doesn’t.


    As Patricia pointed out, 64-bit pointers will use more memory the 32-bit
    pointers (4 bytes each).

    The OOMEs are inconsistent because it is thrown when there is still free
    memory available. This may seem nuts, but it isn't. If OOME really only
    did get thrown when there was no memory left, as memory gets short the
    garbage collector would have to run more often. The process would just
    max out CPU usage and get nowhere. Better to throw OOME when memory gets
    reasonably short, and allow backing out of an operation and be able to
    continue other operations and save data.

    > there some things I can learn from this? Which part of my program
    > should be tuned, such that maybe it becomes usable with less memory? Etc.


    Use some kind of memory profiling to work out where you memory is going.
    jmap and jhat are included in the JDK. Even back of an envelope
    calculations may well help.

    The problem is probably in very few classes. Don't waste time optimising
    classes that aren't actually taking up much memory. Sound obvious, but
    it's a very easy and common trap to fall into.

    Generally you will want fewer larger objects (or arrays) rather than
    lots of small ones.

    Tom Hawtin
     
    Tom Hawtin, May 2, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Timasmith
    Replies:
    0
    Views:
    342
    Timasmith
    Nov 13, 2006
  2. Replies:
    3
    Views:
    1,859
    Timothy Bendfelt
    Jan 19, 2007
  3. Replies:
    9
    Views:
    1,056
    Juha Nieminen
    Aug 22, 2007
  4. Bernd Eggink
    Replies:
    5
    Views:
    447
    Bernd Eggink
    Jan 1, 2009
  5. Jeff.M
    Replies:
    6
    Views:
    204
    Lasse Reichstein Nielsen
    May 4, 2009
Loading...

Share This Page