Performance benchmarks Java vs C(++) ?

[Mark Sizzler]

Are there any performance benchmarks between c++ and java code for standard algorithms?

For example the "sieve of the erastothenes"?

Mark

 

[Lew]

Are there any performance benchmarks between c++ [sic] and java [sic] code for standard algorithms?

For example the "sieve of the erastothenes"?

GIYF.

<http://butunclebob.com/ArticleS.UncleBob.SpeedOfJavaCppRuby>

There are a number of factors that influence microbenchmarks, such as
the incredible plethora of options in each environment that affect
speed, details of cache locality, whether you include JVM startup
time, and so forth. The experts who perform rigorous tests tend to
agree that dynamically optimizing JVMs achieve similar performance to C
++ programs, give or take, and sometimes faster. The advantage is
increasing in favor of languages like Java and C# because they can
optimize in ways that static optimizers cannot, and more recent
platforms take advantage of those techniques more and more.

Even when Java or C# are somewhat slower, it's not by much any more
(for most things), and as the cited article points out, the advantages
(e.g., maintainability, correctness, portability and not having to
manage memory explicitly (up to 30% of a C++ program according to the
experts)) outweigh most performance disadvantages. Assuming there are
disadvantages.

The Java vs. languageX wars are like the Editor Wars. They go on and
on and on and a lot of people are never going to let themselves move
off whatever position they start with.

In practice, Java and C# are close enough to C++ in performance, if
not faster in real life for real tasks, that their other advantages
should weigh more heavily in one's decision as to what platform to
use.

Also, it's really hard to design benchmarks that give results that
apply to the real world. Compare writing a parallel, concurrently-
executing sieve in C++ to writing one in Java. Remember that human
costs exceed computer run-time costs for almost all applications.

What information are you really after? Will this help you make a
decision, are you trying to start a flame war, do you have to make a
case to a boss or client, or what? What other factors matter to you?

 

[Daniel Pitts]

Are there any performance benchmarks between c++ and java code for standard algorithms?

For example the "sieve of the erastothenes"?

Mark

Possibly, but they are useless comparison for most people. Machine
architecture, compiler, memory speed, and many other factors could
affect the results of the benchmark significantly. So, unless you are
planning on using the exact algorithm on the exact platform, the
benchmark means nothing.

 

[Tom Anderson]

Never heard of it. I *have* heard of something called the Sieve of
Eratosthenes that identifies prime numbers, but since the supply of
primes is infinite the running time is O(1/0) and the constant factors
are unimportant. All programming languages take the same amount of time
to complete an infinite loop.

Except perl, which does it slower.

tom

 

[Roedy Green]

Are there any performance benchmarks between c++ and java code for standard algorithms?

For example the "sieve of the erastothenes"?

You can't compare languages; you have to compare specific compilers.

If you want fast start up and code that beats hand assembler, use Jet.

See http://mindprod.com/jgloss/jet.html

For fast Sun Java, you need long running app and the -server option.
Sun's java is not designed to start quickly.

--
Roedy Green Canadian Mind Products
http://mindprod.com
Your old road is
Rapidly agin'.
Please get out of the new one
If you can't lend your hand
For the times they are a-changin'.

 

[public boolean]

Even when Java or C# are somewhat slower, it's not by much any more
(for most things), and as the cited article points out, the advantages
(e.g., maintainability, correctness, portability and not having to
manage memory explicitly (up to 30% of a C++ program according to the
experts)) outweigh most performance disadvantages.

That memory management is one of the places where Java can show a speed
advantage.

During ten minutes of program run:
Allocation takes time proportional to the number of objects allocated.
Deallocation in C++ takes time proportional to the number of objects
that die during ten minutes.
Deallocation in Java takes time proportional to the number of objects
that survive the ten minutes, assuming one garbage collection at the end
of that period.

Generally, deallocation time is proportional to the number of objects
deleted in C++, and proportional to the number of objects that live
longer than some amount of time* in Java.

* Dependent on how frequently the GC ends up running, rather than fixed.

Since typical apps have large numbers of short-lived objects and smaller
numbers of long-lived objects, this gives Java the advantage in *speed*
of memory management, not just in *ease* of memory management.

Older garbage collector technologies carried performance penalties
relative to manual memory management, but modern generational collectors
actually have performance advantages over manual memory management.

Java on a modern enough JVM (i.e., one with JIT and a generational GC)
is faster, better, and cheaper to develop apps in than C++.

 

[Andrew Reilly]

Are there any performance benchmarks between c++ and java code for
standard algorithms?

For example the "sieve of the erastothenes"?

Mark

Not the sieve in this instance, but you'll find some interesting answers
at the computer language benchmark game:

http://shootout.alioth.debian.org/

I was pleased to see that this has recently (11 Nov 2008) been updated,
so the results are quite current.

Cheers,

 

[Isaac Gouy]

Not the sieve in this instance, but you'll find some interesting answers
at the computer language benchmark game:

http://shootout.alioth.debian.org/

I was pleased to see that this has recently (11 Nov 2008) been updated,
so the results are quite current.

1) That timestamp indicates when /any/ of the results were updated -
the 11 Nov 2008 update was re-measurement Scala programs after
upgrading to version 2.7.2.final

The "build & benchmark results" for each program begin with a
timestamp indicating when each program was last measured, for example

http://shootout.alioth.debian.org/u32/benchmark.php?test=nbody&lang=java&id=2#log


2) Although the naïve Sieve of Eratosthenes is no longer measured,
there are (not very interesting) measurements from ~ 6 months ago on
the old Pentium 4 machine:

http://shootout.alioth.debian.org/gp4/benchmark.php?test=nsieve&lang=all

 

[ldv]

Are there any performance benchmarks between c++ andjavacode for standard algorithms?

For example the "sieve of the erastothenes"?

Mark

A recent comparison for somewhat more sophisticated algorithms:

http://www.stefankrause.net/wp/?p=9

LDV

 

[Norbert Ziegler]

Are there any performance benchmarks between c++ and java code for standard algorithms?

For example the "sieve of the erastothenes"?

Mark

Search the web with your words "performance benchmark java c++" and get
e.g.
http://www.idiom.com/~zilla/Computer/javaCbenchmark.html
http://kano.net/javabench/
and many more.

As a rule of thumb one can assume a factor of 1.5 comparing compiled vs.
"interpreted bytecode".
In fact, it depends on the implementation and the concrete problem.
Imaging a chess program: using Objects (new..) is killing performance
here, but that's (chessprog) a very special task..

Last: spell the name correct: "Eratosthenes"
and you get http://www.shudo.net/jit/perf/

Regards, Norbert

 

[Labyrinth]

Search the web with your words "performance benchmark java c++" and get
e.g.
http://www.idiom.com/~zilla/Computer/javaCbenchmark.html
http://kano.net/javabench/
and many more.

As a rule of thumb one can assume a factor of 1.5 comparing compiled vs.
"interpreted bytecode".
In fact, it depends on the implementation and the concrete problem.
Imaging a chess program: using Objects (new..) is killing performance
here, but that's (chessprog) a very special task..

Not even mentioning a garbage collector and constand
object allocation and deallocation.

It is amazing Java performs as reasonable good as it does,
considering all these massive allocations, at least in some
applications, if not most.

Another thing is JVM as an intermediary between the
code and O/S. That is another layer. Plus deep levels
of abstraction inevitably cause references to references
to references, which at the end may half the performance.

 

[Lew]

Search the web with your words "performance benchmark java c++" and get
e.g. <http://www.idiom.com/~zilla/Computer/javaCbenchmark.html> <http://kano.net/javabench/>
and many more.

As a rule of thumb one can assume a factor of 1.5 comparing compiled vs.
"interpreted bytecode".
In fact, it depends on the implementation and the concrete problem.
Imaging a chess program: using Objects (new..) is killing performance
here, but that's (chessprog) a very special task..

Is it?

Allocation for objects in Java is blazingly fast, on the order of
10-12 machine instructions, much faster than most other languages.
That's because Java allocation involves mostly just incrementing a
pointer to heap.

Garbage collection does take time, less for short-lived object than
for Methuselahs, but people tend to overestimate the impact relative
to real-world programs in non-managed-memory environments. That's
because without managed memory, the program itself must manage
deallocation, which results in bugs and takes its own overhead. (I've
read that 30-40% of code in a non-managed-memory language supports
memory management.)

As for "interpreted bytecode" overhead, that depends on how much the
bytecode actually is interpreted. In practice, real-world Java
programs have their performance-intensive sections compiled to native
machine code, taking advantage of dynamic optimization not available
to statically-compiled languages.

I was on a project in the early 1990s that interpreted HP Business
BASIC on a '386 PC with performance equivalent to HP's own compiled
version of that language on their own mainframe. That interpreter
didn't even have an equivalent to Java's Hotspot compiler; it was just
efficient.

In some cases Java runs within 1-2% of the speed equivalent compiled C
or C++ code, and in others it runs even faster. Micro-benchmarks I've
run typically agree with Norbert's rule of thumb that the Java goes
runs about 30-50% slower for CPU-bound, single-threaded code, but
benchmarks are pretty blunt tools. In the real world the run-time
performance is so close as to be overshadowed by other considerations,
for most applications.

Regardless, one must avoid spreading urban legends about the overhead
of memory allocation and garbage collection, or "interpreted
bytecode", when discussing Java run-time performance.

 

[Norbert Ziegler]

Is it?

Yes, it is. I'm talking about creating millions (!) of objects: imaging
you invent a class "Move" to represent a move (what else...). You gain
performance by eliminating that class and use a primitive long instead.
But as mentioned before: it's very specific and in "real life" no problem.

Allocation for objects in Java is blazingly fast, on the order of
10-12 machine instructions, much faster than most other languages.
That's because Java allocation involves mostly just incrementing a
pointer to heap.

I know.

Garbage collection does take time, less for short-lived object than
for Methuselahs, but people tend to overestimate the impact relative
to real-world programs in non-managed-memory environments. That's
because without managed memory, the program itself must manage
deallocation, which results in bugs and takes its own overhead. (I've
read that 30-40% of code in a non-managed-memory language supports
memory management.)

No doubt about that. But nobody asked it, did one?

As for "interpreted bytecode" overhead, that depends on how much the
bytecode actually is interpreted. In practice, real-world Java
programs have their performance-intensive sections compiled to native
machine code, taking advantage of dynamic optimization not available
to statically-compiled languages.

Really, do they? That implies that every Java-Application that has not
compiled it's performance-intensive stuff is not a real-world Java-App..
Compiling java-Code should only the be the last line of defence.

I was on a project in the early 1990s that interpreted HP Business
BASIC on a '386 PC with performance equivalent to HP's own compiled
version of that language on their own mainframe. That interpreter
didn't even have an equivalent to Java's Hotspot compiler; it was just
efficient.

In some cases Java runs within 1-2% of the speed equivalent compiled C
or C++ code, and in others it runs even faster. Micro-benchmarks I've
run typically agree with Norbert's rule of thumb that the Java goes
runs about 30-50% slower for CPU-bound, single-threaded code, but
benchmarks are pretty blunt tools. In the real world the run-time
performance is so close as to be overshadowed by other considerations,
for most applications.

Regardless, one must avoid spreading urban legends about the overhead
of memory allocation and garbage collection, or "interpreted
bytecode", when discussing Java run-time performance.

True. The discussion "Is language A faster than lanuage B" is mostly not
worth to be discussed. There are dozens of other questions that are more
important.

So take the value 1.5 as an analogy to the value "42" (the answer to all
questions) and offer this value to CEOs or other guys that only want
ONE, exactly ONE value and no details. And if you want, change the value
to 0.9 or 1.2, as you like, depending on what result you want to get ;-)

 

[Lew]

Really, do they?

Really, they do. It's a little thing called the "Hotspot" compiler.

That implies that every Java-Application that has not
compiled it's [sic] performance-intensive stuff is not a real-world Java-App..
Compiling java-Code should only the be the last line of defence.

Nonsense. *Every* Java program has some of its bytecode translated to
mashine language at run-time, if it's using the Hotspot compiler,
especially if run with the "-server" option.

I wasn't talking about manual translation to machine code but what the
JVM does for free. That's why I used the term "real-world". In
practice, particularly in enterprise environments where GC
optimization and the "-server" option are widely used, there is
significant speedup due to compilation of bytecode to machine code.
With the latest JVMs (version 6+), there is dynamic decompilation back
to bytecode and recompilation to machine code as the performance
characteristics of the program vary over time.

None of this is a "last line of defense" but automatic and universal.

 

[Arne Vajhøj]

As a rule of thumb one can assume a factor of 1.5 comparing compiled vs.
"interpreted bytecode".

Not particular relevant for the question since C++ is AOT compiled
and Java is JIT compiled. Neither is interpreted byte code.

Arne

 

[Arne Vajhøj]

Yes, it is. I'm talking about creating millions (!) of objects

And ?

Lot of Java programs does that.

Allocating a million objects per second would take something
like 0.1% of the CPU power on a good PC.

Really, do they?
Yes.

That implies that every Java-Application that has not
compiled it's performance-intensive stuff is not a real-world Java-App..

It is approx. 10 years since SUN JVM added the possibility to JIT
compile.

It is approx. 6 years since SUN JVM removed the possibility not
to JIT compile.

I think you may want to buy some newer Java books.

Arne

 

[Roedy Green]

As a rule of thumb one can assume a factor of 1.5 comparing compiled vs.
"interpreted bytecode".

But nobody has used interpreted byte code in Java for years, except in
cell phones where RAM and batter power is limited. On the desktop it
is now hotspotted (Sun JVM) or native machine code (Jet).
--
Roedy Green Canadian Mind Products
http://mindprod.com
PM Steven Harper is fixated on the costs of implementing Kyoto, estimated as high as 1% of GDP.
However, he refuses to consider the costs of not implementing Kyoto which the
famous economist Nicholas Stern estimated at 5 to 20% of GDP

 

[Tom Anderson]

But nobody has used interpreted byte code in Java for years, except in
cell phones where RAM and batter power is limited. On the desktop it is
now hotspotted (Sun JVM) or native machine code (Jet).

My understanding is that in the Sun JVM, bytecode is still interpreted
before it gets hotspotted. Unlike the IBM JVM, for instance, where it's
always compiled (although with a quick, poor compiler at first, there
being a hotspot-like compiler to redo important bits later). I could be
out of date on this, but if i'm not, then we're all using interpreted code
every day.

tom

 

[Stefan Ram]

My understanding is that in the Sun JVM, bytecode is still interpreted
before it gets hotspotted.

Users might have some control with an XX option:

java -XX:CompileThreshold=1 ...

 

[Eric Sosman]

As a rule of thumb one can assume a factor of 1.5 comparing compiled vs.
"interpreted bytecode".

But nobody has used interpreted byte code in Java for years, except in
cell phones where RAM and batter power is limited. [...]

If batter power is limited, the infield crowds the plate
anticipating a bunt.