Class with only methods - less memory?

[redefined.horizons]

I am working on a program that creates hundreds, if not thousands, of
objects in memory. This often causes the program to run out of memory
and shut down.

I am looking for a solution to this. I am curious how much memory is
required from objects that belong to a class that has only method
definitions, and no member variables or other objects included in its
definition.

I want to know if I can solve my memory problem by creating an object
that accesses its "data" or member variables off of the hard disk
instead of from RAM. I know this will impose a serious speed or
performance penatly. However, if this will work I can take some steps
to get around the speed problem.

Will I reduce RAM usage by creating hundreds or thousands of objects
from a class that only contains method definitions?

Thanks for the help.

Scott Huey

 

[Patricia Shanahan]

I am working on a program that creates hundreds, if not thousands, of
objects in memory. This often causes the program to run out of memory
and shut down.

I am looking for a solution to this. I am curious how much memory is
required from objects that belong to a class that has only method
definitions, and no member variables or other objects included in its
definition.

I want to know if I can solve my memory problem by creating an object
that accesses its "data" or member variables off of the hard disk
instead of from RAM. I know this will impose a serious speed or
performance penatly. However, if this will work I can take some steps
to get around the speed problem.

Will I reduce RAM usage by creating hundreds or thousands of objects
from a class that only contains method definitions?

Thanks for the help.

Scott Huey

You are probably already aware of this, but just in case... JVMs often
have a command line parameter to tell them to use more heap space. For
example, -Xmxn for Sun's JVMs.

Patricia

 

[redefined.horizons]

Thanks Patricia,

I've done that, but the data sets I'm working with are huge, and I need
find a way to access the information in the objects from the hard disk,
one at a time.

I thought objects created from a class definition with only methods
would have a very small method footprint.

I'm not sure how Java works internally, but I imagine it only needs one
"Method table" or "function table" per class, not per object. I would
think it is really the member variables that make up most of an
object's memory footprint.

Scott Huey

 

[as4109]

(e-mail address removed) ha escrito:

I am working on a program that creates hundreds, if not thousands, of
objects in memory. This often causes the program to run out of memory
and shut down.

I am looking for a solution to this. I am curious how much memory is
required from objects that belong to a class that has only method
definitions, and no member variables or other objects included in its
definition.

Object vptr size (probably the same as a native pointer) +
garbage-collection overhead size (probably at least the size of a
native pointer); so at least 8 bytes on a 32-bit machine. There may
also be overhead for the synchronization primitives, or the need to do
synchronization may mean that an Object has to be aligned to a native
cache-line (probably 8 or 16 bytes, possibly more).

On a 32-bit machine, those limitations as well as limitations on what
part of the memory is available to user programs may mean you cannot
have more than about 12 million objects in a single JVM. That would
require 3 GB of memory.

I want to know if I can solve my memory problem by creating an object
that accesses its "data" or member variables off of the hard disk
instead of from RAM. I know this will impose a serious speed or
performance penatly. However, if this will work I can take some steps
to get around the speed problem.

If you have the disk-space, your operating-system will probably allow
you to create a swap-file big enough (3 GB) to allow that JVM to exist
in virtual memory. I just tried it: first I tried running "java
-Xms1100M" and got the error

Error occurred during initialization of VM
Could not reserve enough space for object heap

Then I added a 1-GB swap file and tried the same command; it was
successful. The exact way to add virtual memory depends on your
operating system.

Will I reduce RAM usage by creating hundreds or thousands of objects
from a class that only contains method definitions?

You'll save some memory, but what can you do with a class that
contains only method definitions, and no state? Some ideas to ponder:

1. Use "java.lang.ref.WeakReference" or "java.util.WeakHashMap", or
both, or something similar.

2. Store your data-structure in SQL tables.

3. java.io.RandomAccessFile

4. Store your data in some other Map, but only put entries in the map
for things that are not in their default state.

5. byte[] can store 8 true-false values per byte of memory it uses

Exactly what you choose depends on your application, and you aren't
very specific here.

*David*

 

[Daniel Dyer]

I've done that, but the data sets I'm working with are huge, and I need
find a way to access the information in the objects from the hard disk,
one at a time.

I thought objects created from a class definition with only methods
would have a very small method footprint.

If there are no fields whatsoever, then there is nothing to be gained by
having more than one instance of the class (unless its methods are not
re-entrant and you need concurrent invocations, in which case you could
consider pooling multiple instances). If you do put your data on disk,
you could use just one instance of the class to process all of your data.

Not knowing anything about your program, is the Flyweight Pattern
something that might help?

Dan.

 

[Patricia Shanahan]

If there are no fields whatsoever, then there is nothing to be gained by
having more than one instance of the class (unless its methods are not
re-entrant and you need concurrent invocations, in which case you could
consider pooling multiple instances). If you do put your data on disk,
you could use just one instance of the class to process all of your data.

Not knowing anything about your program, is the Flyweight Pattern
something that might help?

Also consider whether the data would fit in memory packed densely in
primitive arrays. It all depends.

Patricia

 

[Mark Space]

If there are no fields whatsoever, then there is nothing to be gained by
having more than one instance of the class (unless its methods are not
re-entrant and you need concurrent invocations, in which case you could
consider pooling multiple instances). If you do put your data on disk,

This is an interesting idea. Does making a copy of a class with
non-reentrant methods make the class thread safe? I'm guessing no way.

I think the original poster was asking if the method byte-code (is that
the correct term?) is copied when the class (any class) was instanced,
or if just the local variables are copied. I'd assume he's correct and
only variables are copied, but I don't know. I'd assume that the
reference created for a "method only" object is just 8 bytes in most
implementations, or a bit more.

I think you should investigate some ways of managing memory, rather than
try to count bytes for JVM internal structures. Weak references might
help out (WeakReference). Soft references might be useful as well
(SoftReference). Both are derived from java.lang.ref.Reference.
java.lang.ref also includes a ReferenceQueue object to help you manage
your references.

 

[Mark Space]

I think you should investigate some ways of managing memory, rather than
try to count bytes for JVM internal structures. Weak references might
help out (WeakReference). Soft references might be useful as well
(SoftReference). Both are derived from java.lang.ref.Reference.
java.lang.ref also includes a ReferenceQueue object to help you manage
your references.

Oops, looks like *Dave* beat me too it. Next time read the whole
thread..

 

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

Object vptr size (probably the same as a native pointer) +
garbage-collection overhead size (probably at least the size of a
native pointer); so at least 8 bytes on a 32-bit machine. There may
also be overhead for the synchronization primitives, or the need to do
synchronization may mean that an Object has to be aligned to a native
cache-line (probably 8 or 16 bytes, possibly more).

On a 32-bit machine, those limitations as well as limitations on what
part of the memory is available to user programs may mean you cannot
have more than about 12 million objects in a single JVM. That would
require 3 GB of memory.

I can not follow your math.

And a simple test on my PC shows 100 million objects in 1.3 GB
java heap space.

Arne

 

[Daniel Dyer]

This is an interesting idea. Does making a copy of a class with
non-reentrant methods make the class thread safe? I'm guessing no way.

Actually, if there is no instance state whatsoever, you are right, since
the only thing I can think of that would make it non-re-entrant would be
accessing static data somewhere, and pooling wouldn't help that. Pooling
would be more useful when there is some instance state that has a wider
scope than the individual method invocation.

Dan.

 

[Chris Uppal]

I am working on a program that creates hundreds, if not thousands, of
objects in memory. This often causes the program to run out of memory
and shut down.

Unless those object are /very/ big (at least 100k or a megabyte each) creating
hundreds or thousands of them will not cause the JVM to run out of memory.
(Assuming you are running on a real computer, not some sort of handheld
device).

I am looking for a solution to this. I am curious how much memory is
required from objects that belong to a class that has only method
definitions, and no member variables or other objects included in its
definition.

It depends on the JVM implementation, but I believe that recent Sun JVMs will
typically require a minimum of 8-bytes per object, plus another 4 bytes if the
object is an array, plus another <unknown-but-small> bytes if one of several
unusual things has happened to the object during its lifetimes. Included in
the list of "unusual things" are (as far as I know, and I'm /not/ an expert)
entering a synchronised block/method that is /actually/ contended, and being
asked for its identity hash.

I want to know if I can solve my memory problem by creating an object
that accesses its "data" or member variables off of the hard disk
instead of from RAM. I know this will impose a serious speed or
performance penatly. However, if this will work I can take some steps
to get around the speed problem.

Certainly you can do that, "it's just code (tm)". BTW, if you are tempted to
use file mapping for this, be aware that the memory used for mapped files is
only reclaimed by the GC, you cannot control when the resources are released
back to the OS yourself (unlike closing a file, say).

Will I reduce RAM usage by creating hundreds or thousands of objects
from a class that only contains method definitions?

Yes, if you do it right. I'm guessing that you may be worried that the methods
are duplicated for each object ? If that is your worry, then you needn't be
worried ;-) The methods are attached to the class and are shared by all the
instances with no additional cost per object.

-- chris