Would you like to calculate memory usage of an object?

[Morten Simonsen]

Hello everyone

I have made a program to calculate the memory usage of an arbitrary object.
Usually this is done by making 10000 of a specific object and count the
memory usage in the runtime, before and after creation. My program is
nothing like this, it is designed to receive one single object, and do the
appropriate computations from that single object. The closest comparion
I can come up with, is a memory profiler, allthough that one is probably
made up in another way and a lot more complex in every way, also in
terms of resource demands. My main goal has been to make a 'device'
to track the memory usage of my various services in our production
environment. I have tested the program on many kind of objects, and
it works very fine.

There are 2 reasons why I write this mail:

1. I would be glad if anyone else could use this program.
2. I need help to improve the program, because it's not perfect

If you are interested in this 'project' please respond. I have not yet made
any request towards Jakarta, allthough I think it could fit there
somewhere. But, until I'm pretty sure this could be something worthwhile,
I prefer to try this first. Go to my
<a href="http://home.broadpark.no/~mosimons/">website</a> for
downloading and more information.

Morten Simonsen

 

[Chris Smith]

I have made a program to calculate the memory usage of an arbitrary object.
Usually this is done by making 10000 of a specific object and count the
memory usage in the runtime, before and after creation. My program is
nothing like this, it is designed to receive one single object, and do the
appropriate computations from that single object.

Unfortunately, there is no such thing as "the appropriate computations".
If you intend for your code to predict memory usage on the version of
IBM's Java runtime that's bundled with Websphere 5.1 on some specific
hardware platform, then you should be rather specific about this!
Pretending that this code correctly predicts memory use on any arbitrary
platform is just plain wrong. Object overheads and reference sizes are
almost certain to vary by implementation and platform. Data alignment
within an object and packing within an array may vary widely as well.

There are other questions as well, regarding whether you intend to
account for internal data structures within the allocator/garbage
collector that are used to track your objects or reserve their space,
but those are so subtle as to be moot until you document the clear
platform-dependence of the existing code.

There's not really a way to "fix" this. Just please document the target
platform, especially since it's not even the most common runtime
environment that your code is designed to work with.

--
www.designacourse.com
The Easiest Way to Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Roedy Green]

1. I would be glad if anyone else could use this program.
2. I need help to improve the program, because it's not perfect

Any technique relying only on measuring available RAM is going to be
easily fooled.

I agree, there is a need for such a tool. It would work like this:

You create an object.

// ram used including overhead, padding,
int size = Sniff.size ( thatObject );

You might invent some other related methods:

// ram used for actual data, excluding overhead
// this should be the same on all JVMs and match
// the intuitive size.
int databytes = Sniff.dataBytes( thatObject );


// size of this object and everything it links to, including
// overhead.
int sizeTree = Sniff.sizeTree ( thatObject );

There are now interfaces in Java designed for profilers and debuggers
Perhaps your mini profiler can tap into them.

see http://mindprod.com/jgloss/profiler.html
http://mindprod.com/jgloss/debugger.html

 

[Morten Simonsen]

Hi

Unfortunately, there is no such thing as "the appropriate computations".

I can really feel the pessimism you share with us here. It's almost enough
to bring me down, but not entirely. Still there is a spark of optimism,
allthough
I admit it's not a simple task. As you said, the implementation of the
various JVM
makes it harder to make a good memory calculator, but I suspect there are
ways to overcome even that. I dream of a way of initializing the memory
calculator. In this initialization phase we will compute (in a traditional
way) the
"basic sizes", like object-references, primitive-sizes and so on.

When I wrote the mail, I was hoping that someone may find it interesting,
and
work with me. Therefore, I was trying to attract some attention, rather than
focusing on every problem that I'm facing. Looking back, I see that maybe
I should have made something clear: This tool is not meant to do (and can
never do) and *exact* computation of the memory usage of an object. Internal
structures in the JVM (as the internal String-pool) prevent that. However,
I believe we can make computations which are "close" to reality, or if even
that
fails (it may in some cases) we should be able to compare two equvivalent
objects with each other.

My situation is like this: I am responsible for a framework in which we run
hundreds of services (WebServices). They all run in the same runtime, and I
want
a simple and quick way to pick those services which are the memory leakers.
Until now I have not come up with any tool that could help me solve this,
because
last time I tried with OptimizeIt in my production environment, it
terminated
the production application. I don't like that, and I'm seeking a somewhat
lighter
application.

I believe we really need this tool, and it should be free, simple to use and
reasonably correct. But I sure need some help.

Morten Simonsen

 

[Morten Simonsen]

// size of this object and everything it links to, including
// overhead.
int sizeTree = Sniff.sizeTree ( thatObject );

I think my program does something like this. Indeed the return
object is a graph which is easy to inspect and print out.

There are now interfaces in Java designed for profilers and debuggers
Perhaps your mini profiler can tap into them.

see http://mindprod.com/jgloss/profiler.html
http://mindprod.com/jgloss/debugger.html

I'm not quite sure how I can use any existing profiler or what interfaces
I could use. Maybe I need more guidance. However, I had a long look
at you JGloss-site, and it was pretty impressive. I especially liked your
Bali syntax. I think you have some point there, although it may be that the
waste of time on programming is slightly overrated. However, that
should not be an excuse not to do improvements, especially if they
are backwards compatible.

Morten Simonsen