Byte code execution count

[Kenneth P. Turvey]

I'm thinking about working on a project with a friend of mine, but there
are some issues that I'm not sure how to handle. I'm going to break them
into a few different posts since the issues are unrelated to each other
and people may choose to ignore some of them that way.

I understand that Java doesn't allow one to find out how much CPU time was
used by a process. That's fine. This isn't really a good measure for
what we want anyway since it would be vastly different from CPU to CPU.
What we really want is a way to measure how much work was done by a given
CPU on a given job.

What I was thinking would be a good measure was a simple count of the java
byte code instructions executed on a given machine. Is there anyway to
determine how many byte code instructions have been executed by a given
thread since it started?

I'm sure there is a way to do this with the debugging information, but I
don't want to slow down the process too much. Basically I'd like to be
able to start a thread and then wait until it exits and then ask how many
byte code instructions it processed? Any idea how one would do this?

Any other ideas on how to solve this sort of problem?

Thanks.

 

[Eric Sosman]

Kenneth P. Turvey wrote On 10/23/07 11:25,:

I'm thinking about working on a project with a friend of mine, but there
are some issues that I'm not sure how to handle. I'm going to break them
into a few different posts since the issues are unrelated to each other
and people may choose to ignore some of them that way.

I understand that Java doesn't allow one to find out how much CPU time was
used by a process. That's fine. This isn't really a good measure for
what we want anyway since it would be vastly different from CPU to CPU.
What we really want is a way to measure how much work was done by a given
CPU on a given job.

What I was thinking would be a good measure was a simple count of the java
byte code instructions executed on a given machine. Is there anyway to
determine how many byte code instructions have been executed by a given
thread since it started?

I'm sure there is a way to do this with the debugging information, but I
don't want to slow down the process too much. Basically I'd like to be
able to start a thread and then wait until it exits and then ask how many
byte code instructions it processed? Any idea how one would do this?

Any other ideas on how to solve this sort of problem?

I think you're out of luck, and for more than one
reason.

First, I can think of no way to discover which of
the machine's CPU's ran which parts of your Java code.
That's difficult to do even in assembly, let alone in
Java.

Second, bytecode count is unlikely to be a very
accurate "progress" measure. Bytecode instructions
like iconst_0 are very simple, take little time, and
make little "progress." Instructions like athrow do
considerably more work, take more time, and make more
"progress" (although perhaps in reverse ...).

Finally, ask yourself how many bytecodes a method
executes after it's been JITted ...

"Progress" is really an application-specific notion,
not a machine-provided commodity. The machine can only
run the program; it can't tell whether the program is
"doing something" or just sitting in a polling loop
waiting for something else. If you want to keep track
of "progress," I think you need to establish milestones
within the application itself and count them as they pass.

 

[Kenneth P. Turvey]

"Progress" is really an application-specific notion,
not a machine-provided commodity. The machine can only
run the program; it can't tell whether the program is
"doing something" or just sitting in a polling loop
waiting for something else. If you want to keep track
of "progress," I think you need to establish milestones
within the application itself and count them as they pass.

I guess I'm not really interested in progress as such.. more like expense.

I don't really care if the client code is spinning in a loop doing
nothing. If it is wasting my CPU time it counts. So what I'm really
trying to measure here is a sort of normalized CPU time. Ideally,
something like how many CPU seconds this command would spend on my
Inspiron 1420N would be a terrific measure.

I know that might be a bit too much to ask, so I'm looking for something I
can get. I understand your point though. The JIT compiler really would
completely mess this statistic up.

Any other ideas?

If the numbers came out +/- 30% that would probably be fine. I'm just
trying to get a rough measure.

 

[Roedy Green]

What I was thinking would be a good measure was a simple count of the java
byte code instructions executed on a given machine. Is there anyway to
determine how many byte code instructions have been executed by a given
thread since it started?

not in hotspot, because the byte codes are not used. They are
converted to native code. You can get a nanotimer tick. See
http://mindprod.com/jgloss/time.html

Of course it also counts time used by other tasks too.

 

[Lew]

If the numbers came out +/- 30% that would probably be fine. I'm just
trying to get a rough measure.

Simply switching the JVM from -client to -server can speed it up 50%, that is,
the process would take half the time.

I've seen this happen, measured on an IBM JVM in a large enterprise system,
running on a multi-CPU system.

The optimizer can really change things, not just the time code takes, but the
space through inlining.

You need to find a different metric. Aren't any of the profilers out there
useful to you?

 

[Kenneth P. Turvey]

Simply switching the JVM from -client to -server can speed it up 50%, that is,
the process would take half the time.

I've seen this happen, measured on an IBM JVM in a large enterprise system,
running on a multi-CPU system.

The optimizer can really change things, not just the time code takes, but the
space through inlining.

You need to find a different metric. Aren't any of the profilers out there
useful to you?

Hmm... what I'm looking for is a metric that would be useful when
averaged over 1000 runs on 1000 different computers. I'd like to be able
to compare two different programs and be able to say with some sense of
confidence that program A requires twice as many resources as program B
after this exercise (of course only referring to CPU, not other resources).

A profiler doesn't help because this will all be happening on remote
machines.

 

[Patricia Shanahan]

Hmm... what I'm looking for is a metric that would be useful when
averaged over 1000 runs on 1000 different computers. I'd like to be able
to compare two different programs and be able to say with some sense of
confidence that program A requires twice as many resources as program B
after this exercise (of course only referring to CPU, not other resources).

A profiler doesn't help because this will all be happening on remote
machines.

Have you looked at java.lang.management? I use it to collect data like
the total CPU time for a job.

Patricia

 

[Lew]

Hmm... what I'm looking for is a metric that would be useful when
averaged over 1000 runs on 1000 different computers. I'd like to be able
to compare two different programs and be able to say with some sense of
confidence that program A requires twice as many resources as program B
after this exercise (of course only referring to CPU, not other resources).

In the case I observed, the very same program running on the very same machine
required twice as much CPU in one configuration as in another. Would you
consider this program to be more compact on one configuration than the other,
even though it's the very same bytecode?

 

[Kenneth P. Turvey]

Have you looked at java.lang.management? I use it to collect data like
the total CPU time for a job.

That will probably work. I would have to average it over all the
machines, but it might be good enough. Byte codes would be better, but I
guess you can't always get exactly what you want.

If I pick 100 machines at random and run some code on them and time it and
then, a week later, pick another 100 machines at random and run the same
code on them, do you think I'll come up with close to the same number?

I'll have to think about the statistics involved in that.

Thanks.

 

[Lew]

That will probably work. I would have to average it over all the
machines, but it might be good enough. Byte codes would be better, but I
guess you can't always get exactly what you want.

But, bytecodes cannot be consistently counted, sometimes not even at all, as
others have pointed out on this thread. How would they be better?

 

[Roedy Green]

I know that might be a bit too much to ask, so I'm looking for something I
can get. I understand your point though. The JIT compiler really would
completely mess this statistic up.

One technique is to create a low-priority task that soaks up CPU
cycles. You calibrate it its time-wasting loop so that you can
calculate how much CPU time you were able to scavenge. The rest was
used.

Do you count the time spent in the OS doing i/o, or preparing i/o as
part of the time? It is not completely straightforward.