Virtual Memory

[Ulrich Sondermann]

I am running a program that uses the BigInteger class and the modPow method
on a 100,000+ binary bits integer. The problem is that instead of allocating
memory and staying there, the operation seems to be using virtual memory
instread. My program is as slow as I/O, which really sucks for what is
supposed to be a CPU intensive operation. This happenning both on my iMac G5
and under Windows XP home. Is there any way to stop using virtual memory?
When the numbers are 50,000+ bits this problem goes away as only ram seems
to be used.

Thanx,
Ulie

 

[Roedy Green]

I am running a program that uses the BigInteger class and the modPow method
on a 100,000+ binary bits integer. The problem is that instead of allocating
memory and staying there, the operation seems to be using virtual memory
instread. My program is as slow as I/O, which really sucks for what is
supposed to be a CPU intensive operation. This happenning both on my iMac G5
and under Windows XP home. Is there any way to stop using virtual memory?
When the numbers are 50,000+ bits this problem goes away as only ram seems
to be used.

In windows there is no memory other than virtual memory, except for
the OS itself. When you use too much of it, the system thrashes with
swapping.

You can experiment with setting the virtual ram size on the command
line smaller or larger to see the effect on performance.

You might run a profiler on your code to see what you are doing that
is gobbling up ram. It is not just one BigInteger in there. You may
have dozens of intermediates you are holding onto that could be GCed.

All you can do is try to figure out how to use less virtual RAM and
how to free up as much real ram as you can by shutting down everything
else you can think of also running. See the task manager for a list of
candidates.

 

[Chris Uppal]

I am running a program that uses the BigInteger class and the modPow
method on a 100,000+ binary bits integer. The problem is that instead of
allocating memory and staying there, the operation seems to be using
virtual memory instread.

Unpleasant, but a bit puzzling since a 100K bit integer isn't really using all
that much space in comparison with modern RAM sizes (even a hundred of the
things would only use 1.25 MBytes) -- perhaps you are creating, and keeping
long-lived references to, many more BigIntegers than you realize ?

As far as I know, there is no way to tell Java not to use virtual memory, since
that decision is up to the OS. If you are in control of the machines where
this code is running then you could try turning off virtual memory (I can't see
any point in VM for a desk-top class machine anyway).

One thing that might be worth trying is to work out the maximum memory that
your program /should/ need (assuming that you are not creating more BigIntegers
than you think) and run Java with the appropriate flags to tell it to use at
most that much memory. That may cut down RAM use to the point where the OS can
recognise that the application doesn't need VM. Alternatively it may fail
with an out-of-memory error if your application isn't behaving as you have
designed it to (i.e. if you have a bug that causes excessive memory use but
does not result in erroneous calculations).

-- chris

 

[Thomas Hawtin]

As far as I know, there is no way to tell Java not to use virtual memory, since
that decision is up to the OS. If you are in control of the machines where

(Recent versions of) Solaris allow processes to be kept in RAM.

Also various processors allow very large page sizes, so that
non-swapping applications don't thrash the TLA unnecessarily.

this code is running then you could try turning off virtual memory (I can't see
any point in VM for a desk-top class machine anyway).

My machine swaps like hell. (And wont take any more RAM.)

Tom Hawtin

 

[Chris Uppal]

(Recent versions of) Solaris allow processes to be kept in RAM.

Also various processors allow very large page sizes, so that
non-swapping applications don't thrash the TLA unnecessarily.

But can you control that from Java ? Which is what the OP seemed to be asking
for.

My machine swaps like hell. (And wont take any more RAM.)

Hmm. What do you typically have running ?

-- chris