performance question

  • Thread starter Olivier Scalbert
  • Start date
O

Olivier Scalbert

Daniel said:
Olivier said:
Hello,

I would like to show to somebody how different languages look like by
doing a very simple case.
I was very surprised by the poor performance of the java version !

Here are the programs in different languages:

in C:

#include <stdio.h>

int main()
{
int i;

for(i = 0; i < 1000000; i++)
{
printf("abcdefghijk %d\n", i);
}

return 0;
}

time ./test1 > out.txt

real 0m0.710s
user 0m0.576s
sys 0m0.124s

in java:
public class Test
{
public static void main(String[] args)
{
for (int i = 0; i < 1000000; i++)
{
System.out.println("abcdefghijk " + i);
}
}
}

time java Test > out.txt

real 0m12.364s
user 0m4.180s
sys 0m7.676s

time java -server Test > out.txt

real 0m10.537s
user 0m3.120s
sys 0m6.544s


That is not good at all !
ols@tatooine:~/projects/ruby$ java -version
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing)

ols@tatooine:~/projects/ruby$ uname -a
Linux tatooine 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007
i686 GNU/Linux

In python:
i=0
while i < 1000000:
print "abcdefghijk", i
i=i+1

time python test.py > out.txt

real 0m2.292s
user 0m2.064s
sys 0m0.112s

In perl:
for ($count = 0; $count < 1000000; $count++)
{
print "abcdefghijk $count\n";
}
time perl test.pl > out.txt

real 0m1.243s
user 0m1.060s
sys 0m0.160s

In ruby:
counter = 0
while counter < 1000000
puts("abcdefghijk #{counter}")
counter+=1
end

time ruby test.rb > out.txt

real 0m4.731s
user 0m4.452s
sys 0m0.100s

As you can see the java program performance is far from the other one. I
was very surprised! Before the tests, I was sure that the winner will be
C followed by java, but it is not the case ...

Of course, I can improve the performance of the java version, by using a
StringBuffer, and print this buffer when it is bigger than a given size,
but it is not fair !

If you have any ideas, there are welcomed !

Thanks,

Olivier
I did my own benchmarks using these files:
bench.c:
#include <stdio.h>
#include <time.h>

void bench() {
long foo = 0;
clock_t start = clock();
for (long i = 1; i < 5000; ++i) {
for (long j = 1; j < i; ++j) {
if ((i % j) == 0) {
foo ++;
}
}
}
clock_t end = clock();
printf("%d %dms\n", foo,
(int) ((end - start) * 1000 / CLOCKS_PER_SEC));
}


int main() {
for (long i = 1; i < 10; ++i) {
printf("%d: ", i);
bench();
}
}

Bench.java:

public class Bench {
static final long CLOCKS_PER_SEC = 1000;
static void bench() {
int foo = 0;
long start = System.currentTimeMillis();
for (int i = 1; i < 5000; ++i) {
for (int j = 1; j < i; ++j) {
if ((i % j) == 0) {
foo ++;
}
}
}
long end = System.currentTimeMillis();
System.out.printf("%d %dms\n", foo,
(int) ((end - start) * 1000 / CLOCKS_PER_SEC));
}


public static void main(String[] args) {
for (int i = 1; i < 10; ++i) {
System.out.printf("%d: ", i);
bench();
}
}
}

Then ran these:
-bash-3.00$ java -version
java version "1.5.0_09"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
Java HotSpot(TM) Client VM (build 1.5.0_09-b03, mixed mode, sharing)
-bash-3.00$ javac Bench.java
-bash-3.00$ g++ --version
g++ (GCC) 3.3.3 (NetBSD nb3 20040520)
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is
NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
-bash-3.00$ java -server Bench
1: 38357 457ms
2: 38357 416ms
3: 38357 401ms
4: 38357 394ms
5: 38357 394ms
6: 38357 401ms
7: 38357 395ms
8: 38357 401ms
9: 38357 394ms
-bash-3.00$ java -client Bench
1: 38357 421ms
2: 38357 400ms
3: 38357 394ms
4: 38357 400ms
5: 38357 393ms
6: 38357 393ms
7: 38357 400ms
8: 38357 394ms
9: 38357 401ms
-bash-3.00$ ./bench
1: 38357 450ms
2: 38357 440ms
3: 38357 450ms
4: 38357 430ms
5: 38357 450ms
6: 38357 440ms
7: 38357 450ms
8: 38357 440ms
9: 38357 450ms


This looks to me like the c version is slower...

Thanks for the info !

ols@tatooine:~/projects/ruby$ java -server Bench
1: 38357 615ms
2: 38357 389ms
3: 38357 293ms
4: 38357 290ms
5: 38357 295ms
6: 38357 293ms
7: 38357 294ms
8: 38357 295ms
9: 38357 290ms
ols@tatooine:~/projects/ruby$ java -client Bench
1: 38357 649ms
2: 38357 353ms
3: 38357 290ms
4: 38357 289ms
5: 38357 291ms
6: 38357 291ms
7: 38357 286ms
8: 38357 290ms
9: 38357 299ms
ols@tatooine:~/projects/ruby$ ./bench
1: 38357 530ms
2: 38357 480ms
3: 38357 270ms
4: 38357 270ms
5: 38357 260ms
6: 38357 260ms
7: 38357 270ms
8: 38357 260ms
9: 38357 260ms


On my computer, c is just a little faster, but not a lot !
But I am not on BSD !
;-)

By the way, difficult to install java on NetBSD ?

Olivier
 
D

Daniel Pitts

Olivier said:
Daniel said:
Olivier said:
Hello,

I would like to show to somebody how different languages look like by
doing a very simple case.
I was very surprised by the poor performance of the java version !

Here are the programs in different languages:

in C:

#include <stdio.h>

int main()
{
int i;

for(i = 0; i < 1000000; i++)
{
printf("abcdefghijk %d\n", i);
}

return 0;
}

time ./test1 > out.txt

real 0m0.710s
user 0m0.576s
sys 0m0.124s

in java:
public class Test
{
public static void main(String[] args)
{
for (int i = 0; i < 1000000; i++)
{
System.out.println("abcdefghijk " + i);
}
}
}

time java Test > out.txt

real 0m12.364s
user 0m4.180s
sys 0m7.676s

time java -server Test > out.txt

real 0m10.537s
user 0m3.120s
sys 0m6.544s


That is not good at all !
ols@tatooine:~/projects/ruby$ java -version
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) Client VM (build 1.6.0_02-b05, mixed mode, sharing)

ols@tatooine:~/projects/ruby$ uname -a
Linux tatooine 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007
i686 GNU/Linux

In python:
i=0
while i < 1000000:
print "abcdefghijk", i
i=i+1

time python test.py > out.txt

real 0m2.292s
user 0m2.064s
sys 0m0.112s

In perl:
for ($count = 0; $count < 1000000; $count++)
{
print "abcdefghijk $count\n";
}
time perl test.pl > out.txt

real 0m1.243s
user 0m1.060s
sys 0m0.160s

In ruby:
counter = 0
while counter < 1000000
puts("abcdefghijk #{counter}")
counter+=1
end

time ruby test.rb > out.txt

real 0m4.731s
user 0m4.452s
sys 0m0.100s

As you can see the java program performance is far from the other one. I
was very surprised! Before the tests, I was sure that the winner will be
C followed by java, but it is not the case ...

Of course, I can improve the performance of the java version, by using a
StringBuffer, and print this buffer when it is bigger than a given size,
but it is not fair !

If you have any ideas, there are welcomed !

Thanks,

Olivier
I did my own benchmarks using these files:
bench.c:
#include <stdio.h>
#include <time.h>

void bench() {
long foo = 0;
clock_t start = clock();
for (long i = 1; i < 5000; ++i) {
for (long j = 1; j < i; ++j) {
if ((i % j) == 0) {
foo ++;
}
}
}
clock_t end = clock();
printf("%d %dms\n", foo,
(int) ((end - start) * 1000 / CLOCKS_PER_SEC));
}


int main() {
for (long i = 1; i < 10; ++i) {
printf("%d: ", i);
bench();
}
}

Bench.java:

public class Bench {
static final long CLOCKS_PER_SEC = 1000;
static void bench() {
int foo = 0;
long start = System.currentTimeMillis();
for (int i = 1; i < 5000; ++i) {
for (int j = 1; j < i; ++j) {
if ((i % j) == 0) {
foo ++;
}
}
}
long end = System.currentTimeMillis();
System.out.printf("%d %dms\n", foo,
(int) ((end - start) * 1000 / CLOCKS_PER_SEC));
}


public static void main(String[] args) {
for (int i = 1; i < 10; ++i) {
System.out.printf("%d: ", i);
bench();
}
}
}

Then ran these:
-bash-3.00$ java -version
java version "1.5.0_09"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
Java HotSpot(TM) Client VM (build 1.5.0_09-b03, mixed mode, sharing)
-bash-3.00$ javac Bench.java
-bash-3.00$ g++ --version
g++ (GCC) 3.3.3 (NetBSD nb3 20040520)
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is
NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
-bash-3.00$ java -server Bench
1: 38357 457ms
2: 38357 416ms
3: 38357 401ms
4: 38357 394ms
5: 38357 394ms
6: 38357 401ms
7: 38357 395ms
8: 38357 401ms
9: 38357 394ms
-bash-3.00$ java -client Bench
1: 38357 421ms
2: 38357 400ms
3: 38357 394ms
4: 38357 400ms
5: 38357 393ms
6: 38357 393ms
7: 38357 400ms
8: 38357 394ms
9: 38357 401ms
-bash-3.00$ ./bench
1: 38357 450ms
2: 38357 440ms
3: 38357 450ms
4: 38357 430ms
5: 38357 450ms
6: 38357 440ms
7: 38357 450ms
8: 38357 440ms
9: 38357 450ms


This looks to me like the c version is slower...

Thanks for the info !

ols@tatooine:~/projects/ruby$ java -server Bench
1: 38357 615ms
2: 38357 389ms
3: 38357 293ms
4: 38357 290ms
5: 38357 295ms
6: 38357 293ms
7: 38357 294ms
8: 38357 295ms
9: 38357 290ms
ols@tatooine:~/projects/ruby$ java -client Bench
1: 38357 649ms
2: 38357 353ms
3: 38357 290ms
4: 38357 289ms
5: 38357 291ms
6: 38357 291ms
7: 38357 286ms
8: 38357 290ms
9: 38357 299ms
ols@tatooine:~/projects/ruby$ ./bench
1: 38357 530ms
2: 38357 480ms
3: 38357 270ms
4: 38357 270ms
5: 38357 260ms
6: 38357 260ms
7: 38357 270ms
8: 38357 260ms
9: 38357 260ms


On my computer, c is just a little faster, but not a lot !
But I am not on BSD !
;-)

By the way, difficult to install java on NetBSD ?

Olivier

cd /usr/pkgsrc/lang/sun-jdk15
make install

:)
Although, now they have sun-jdk6, so I would use that instead.
 
S

Steve Wampler

Olivier said:
On my computer, c is just a little faster, but not a lot !

I've noticed that Java's performance relative to C is better
on my dual Opteron (CentOS4) than on my dual Athlon (CentOS4,
also). So hardware apparently plays a role.
 
P

Patricia Shanahan

Mark Thornton wrote:
....
One downside of techniques like those used in HotSpot is that it makes
writing useful micro benchmarks much harder. Some suggest that a useful
micro benchmark is now all but impossible. HotSpot isn't the only
culprit here; modern CPUs also have very complex performance
characteristics that similarly confound attempts at simple benchmarks.
....

The uselessness of micro-benchmarks is a lot older than Hotspot.

I became convinced while comparing two computers, both with vector
processors, on the double precision Fortran version of Livermore Loops,
http://www.netlib.org/benchmark/livermore.

Any one of the loops could have been extracted and treated as a micro
benchmark. By selecting a loop, I could have proved a three to one
performance ratio between the two computers. *In either direction*.

The compilers for the two computers vectorized different subsets of the
loops.

Patricia
 
T

Tim Smith

I've noticed that Java's performance relative to C is better
on my dual Opteron (CentOS4) than on my dual Athlon (CentOS4,
also). So hardware apparently plays a role.

I noticed that too. I came across this problem in a book of probability
puzzles: you have a drawer with a mix of red and black socks. You pull
out two at random. The probability they will both be red is 1/2. What
is the minimum number of socks that can be in the drawer? How about if
there are an even number of black socks?

After solving this, I wrote a little program to find all numbers of red
and black socks that could be in the drawer, for drawers with up to
999999 red socks.

Afterwards, I decided it would make an amusing little test of languages,
and tried it in C and Java (I had originally done it in Perl). Here's
the Java, turned into a benchmark:

======= JAVA VERSION =======
class Socks
{
public static void main( String[] args )
{
double red, total;
int found = 0;
long start = System.nanoTime();
for ( red = 2; red < 1000000; ++red )
{
double p = 2 * red * (red - 1);
total = 1 + Math.floor(Math.sqrt( p ));
if ( total * (total - 1) == p )
++found;
}
long elapsed = (System.nanoTime() - start)/1000000;
System.out.println( found + " found, elapsed: " + elapsed + " msec" );
}
}

======= C VERSION =======

#include <stdio.h>
#include <math.h>
#include <sys/time.h>

int main( void )
{
struct timeval start, stop;
double red, total;
double elapsed;
int found = 0;

gettimeofday( &start, 0 );
for ( red = 2; red < 1000000; ++red )
{
double p = 2 * red * (red - 1);
total = 1 + floor(sqrt(p));
if ( total * (total - 1) == p )
++found;
}
gettimeofday( &stop, 0 );
elapsed = stop.tv_sec + stop.tv_usec/1000000.0
- start.tv_sec - start.tv_usec/1000000.0;
printf( "%d found, elapsed time: %f msec\n", found, elapsed*1000 );
return 0;
}

On my PowerBook (G4 PPC), Java and C take pretty much the same time. On
my Linux box (2.4 GHz P4), C is twice as fast as Java. Same result on
my iMac (Core Duo)--C is twice as fast.

This is consistent with what I've seen on similar tests. Whenever I've
had some small compute-bound thing I've wanted to do, and tried it on
Java and C, I don't think I've seen Java win on any platform, but it
always seems to be close enough that I would not avoid it if there were
anything that made me otherwise prefer Java for that problem.

Also keep in mind that once you get out of compute-bound programs where
the bottleneck is your code, and start getting into programs where the
bottleneck is in libraries, things can change a lot. Java, for example,
seems to have a pretty good regular expression library, faster than some
I've seen for other languages, so even if Java were going to do my
computation quite a bit slower than C would, that could easily be made
up if I were doing a lot of regular expression matching.
 
L

Lew

Tim said:
On my PowerBook (G4 PPC), Java and C take pretty much the same time. On
my Linux box (2.4 GHz P4), C is twice as fast as Java. Same result on
my iMac (Core Duo)--C is twice as fast.

This is consistent with what I've seen on similar tests. Whenever I've
had some small compute-bound thing I've wanted to do, and tried it on
Java and C, I don't think I've seen Java win on any platform, but it
always seems to be close enough that I would not avoid it if there were
anything that made me otherwise prefer Java for that problem.

Also keep in mind that once you get out of compute-bound programs where
the bottleneck is your code, and start getting into programs where the
bottleneck is in libraries, things can change a lot. Java, for example,
seems to have a pretty good regular expression library, faster than some
I've seen for other languages, so even if Java were going to do my
computation quite a bit slower than C would, that could easily be made
up if I were doing a lot of regular expression matching.

Christian made what I consider the definitive point over in "The myth of
Java's slowness":
... its a question of what you want.
We often tend to advertise our language as the one for everything .. but
no language is.
The problem is sometimes just people bitching about other languages..
But we Java developers are happy with this, because:

"There are only two kinds of languages: the kind everybody bitches
about, and the kind nobody uses."
- B. Stroustrup

Oh isn't he wise? Listen to the Guru!

in response to Joshua Cranmer's equally definitive point about planes being
faster to fly a few hundred people across a continent but trains being faster
to transport tons of coal.

Although, it occurs to me that if one used a multi-airplane solution with
parallelized coal transport that it would be possible to transport coal more
swiftly with a fleet of transport aircraft than with a train. Naturally it
would be much, much more expensive than a freight train, but hey, speed is
all, right?
 
G

George Neuner

Jon said:
On the contrary, Olivier's test is simple and flawless. Can you make the
Java nearly as fast as the C? Feel free to not use unicode, or does Java
have poor support for everything else?

Well, if you like such simple and flawless tests, here's another one.
Of course, this one is (on my machine) over 700 times *faster* in
Java than in C. So, why does C have such poor performance - surely
it's been around enough that optimizers should be able to as well
as Java on such a simple test.

(By the way, the 700 is pretty arbitrary - I could just as easily
make it 700,000 times faster in Java!)

*This is why such 'simple, flawless' tests are so flawed*...

------------ C version and time ---
#include <stdio.h>

int
main(int ac, char **av) {
char *s = malloc(1024*1024);
int i = 0;
for (i = 0; i < 1024*1023; ++i) {
s = 'a';
}
s[++i] = '\0';

int k = 0;
int j = 0;
int n = 0;
for (n = 0; n < 100; ++n) {
for (j = 0; j < 1024*1024*1024; ++j) {
k = strlen(s);
}
}
printf("s has %ul (%u) characters\n", strlen(s), k);
}
->gcc -O4 t1.c -o t1
->time t1
s has 1047552l (1047552) characters
t1 113.74s user 0.25s system 99% cpu 1:54.18 total
->
------------------------------------------------------------

--------------------------------- Java version and time ---
class T1 {

public static void main(String[] args) {
StringBuilder s = new StringBuilder(1024*1024);
for (int i = 0; i < 1024*1023; ++i) {
s.append('a');
}
int k = 0;
for (int n = 0; n < 100; ++n) {
for (int i = 0; i < (1024*1024*1024); ++i) {
k = s.length();
}
}
System.out.println("s has "+s.length()+" ("+k+") characters.");
}
}
----------------------------------------------------------------



Again not comparing like programs. Java knows the length of the
string at each point whereas C has to recompute it every time you ask
for it.

Each approach has drawbacks: Java imposes an arbitrary limit on string
length and C has a performance issue when it needs to determine the
length.

George
 
L

Lew

George said:
Again not comparing like programs. Java knows the length of the
string at each point whereas C has to recompute it every time you ask
for it.

Each approach has drawbacks: Java imposes an arbitrary limit on string
length and C has a performance issue when it needs to determine the
length.

Language Wars are like Editor Wars. George, your comment is nicely balanced,
as a few others' have been, pointing out that different languages have
different strengths.

This hardly refutes Steve Wampler's point, indeed, it validates it. He was
pointing out that one can easily construct a benchmark that exploits the
difference(s) between languages to prove whatever point, either side, one
wishes to make. A good knowledge of the different tradeoffs between languages
lets one construct a benchmark that exploits a strength in one language (Java
in his case) that is a weakness in another (C), with respect to what the
benchmark measures.

He was also subtly making the point that the OP was "again[,] not comparing
like programs".

I like Java not for its speed, which is plenty fast enough in the real world,
but for its features as a language and the richness of the API, and not
incidentally, for the abundance of job opportunities. In particular, its
fundamental architecture for portability, network awareness (anyone remember
"The network is the computer"?), inherent multi-threaded power, dynamic nature
(sure, reflection is dangerous, but shoo! it's powerful!) and yet, perhaps
paradoxically, its straightforwardness make for an extremely useful
development platform.
 
J

Jon Harrop

Daniel said:
-bash-3.00$ java -server Bench
1: 38357 457ms
2: 38357 416ms
3: 38357 401ms
4: 38357 394ms
5: 38357 394ms
6: 38357 401ms
7: 38357 395ms
8: 38357 401ms
9: 38357 394ms
-bash-3.00$ java -client Bench
1: 38357 421ms
2: 38357 400ms
3: 38357 394ms
4: 38357 400ms
5: 38357 393ms
6: 38357 393ms
7: 38357 400ms
8: 38357 394ms
9: 38357 401ms
-bash-3.00$ ./bench
1: 38357 450ms
2: 38357 440ms
3: 38357 450ms
4: 38357 430ms
5: 38357 450ms
6: 38357 440ms
7: 38357 450ms
8: 38357 440ms
9: 38357 450ms

This looks to me like the c version is slower...

Looks like there is no significant difference to me. On my Athlon64 X2, Java
is ~13% slower than C.

Note that the IO benchmark was 10x slower in Java.
 
J

Jon Harrop

Steve said:
*This is why such 'simple, flawless' tests are so flawed*...

The only thing flawed are your measurements for C.
------------ C version and time ---
#include <stdio.h>

int
main(int ac, char **av) {
char *s = malloc(1024*1024);
int i = 0;
for (i = 0; i < 1024*1023; ++i) {
s = 'a';
}
s[++i] = '\0';

int k = 0;
int j = 0;
int n = 0;
for (n = 0; n < 100; ++n) {
for (j = 0; j < 1024*1024*1024; ++j) {
k = strlen(s);
}
}
printf("s has %ul (%u) characters\n", strlen(s), k);
}
->gcc -O4 t1.c -o t1
->time t1
s has 1047552l (1047552) characters
t1 113.74s user 0.25s system 99% cpu 1:54.18 total


$ time ./t1
s has 1047552l (1047552) characters

real 0m0.005s
user 0m0.004s
sys 0m0.004s

Perhaps you would like to try again.
--------------------------------- Java version and time ---
class T1 {

public static void main(String[] args) {
StringBuilder s = new StringBuilder(1024*1024);
for (int i = 0; i < 1024*1023; ++i) {
s.append('a');
}
int k = 0;
for (int n = 0; n < 100; ++n) {
for (int i = 0; i < (1024*1024*1024); ++i) {
k = s.length();
}
}
System.out.println("s has "+s.length()+" ("+k+") characters.");
}
}
----------------------------------------------------------------

Note that, even if I could reproduce your results, I can trivially optimize
your C code but nobody has been able to optimize Olivier's Java to run
anything like as fast as C on any system.
 
J

Jon Harrop

Steve said:
Sigh. The test may be simple and flawless, but my cutting and pasting
wasn't. Here's the missing part:
---------------------------------------------------------
->javac T1.java
->time java -server T1
s has 1047552 (1047552) characters.
java -server T1 0.12s user 0.04s system 101% cpu 0.158 total
->
----------------------------------------------------------------

Here, your C can run this benchmark 20 times before your Java has even
started.
 
J

Jon Harrop

Mark said:
If you set the output encoding to ISO-8859-1, then the overhead is
usually insignificant. However people who mishandle text containing
characters outside US ASCII, in my opinion, deserve a special place in
hell. It is really unpleasant trying to recover usable data that has
been mangled in this way.

That depends entirely upon what you're doing. If you're happy with ASCII and
you want your program to run 10x faster then you should be able to make
this change at will. Lot's of people will want "10x faster". I know I
would.
Of course you can.

No, you can't.
You set buffers of any size you want in both Java and
C and select flushing policies too in both. If you want a fair
comparison, just make sure that you use the same conditions in both cases.

How do you do it in OCaml or F#, for example?
Until you equalise the buffering policies, you aren't comparing like
with like.

Nobody wants to compare "like with like" in that sense (replicating
internals). We are interested in comparing two things of great practical
value:

1. Obvious implementation vs obvious implementation, e.g. most concise C vs
most concise Java.

2. Optimized vs optimized.

From what we've seen here, Java is 15x slower in the first case and still
several times slower in the second.
If your benchmark is too simple to allow scope for dynamic optimisation
it is likely to be a poor representation of real use.

That is just putting spin on the fact that Java is unsuitable for
short-lived programs. A huge number of very important applications fall
into that category, e.g. interactive technical computing environments using
JIT compilation.
If your
application is computationally intensive, the results of previous runs
can be a poor guide to optimising the current run. Therefore on real
work the benefit of retaining previous HotSpot data is not as great as a
simplistic test might suggest.

I see no evidence that Hotspot improves the performance of longer-running
programs. I see a lot of people stating that, but no hard numbers.
One downside of techniques like those used in HotSpot is that it makes
writing useful micro benchmarks much harder. Some suggest that a useful
micro benchmark is now all but impossible. HotSpot isn't the only
culprit here; modern CPUs also have very complex performance
characteristics that similarly confound attempts at simple benchmarks.

That is a retreat into "this benchmark fails to reach the asymptotic limit
of infinite complexity and, therefore, shows Java in an unfairly-poor
light".
 
J

Jon Harrop

Joshua said:
Why cripple Java, or any other programming language for that matter, by
having it merely print out one million lines of code? Use sed, awk, or
bash for that matter.

If the Java were crippled, you would be able to optimize it. Can you?
Java's performance gets better in the long run thanks to Hotspot
optimization;

There is absolutely no evidence of that whatsoever.
you're metering a program with an abnormally short lifespan that's
not "realistic" at all.

On the contrary, that is very realistic for a wide range of important
applications, e.g. any interactive coding.
The problem with most benchmarks is that almost all of them are poorly
designed. The best benchmark is one that takes full power of the
language and "standard" libraries to mimic real-world applications. Most
benchmarks are far from real-world examples. How many programs actually
print out 1,000,000 lines of output in their entire lifespan?

A huge number of programs do millions of lines of IO.

I was analyzing our sales vs web stats yesterday and that entails millions
of lines of IO. Not only is interpreted OCaml both interactive and concise
compared to Java, it is also several times faster at this. I don't use Java
precisely because it is woefully inadequate in this respect.
I believe that there is a published benchmark measuring the FPS of a
Quake implementation in both Java and C; IIRC, the Java actually does
better than the C much of the time.

Quake 2 is a decade out of date and no longer even vaguely CPU intensive.
That is the _only_ reason Java will do as well as C on that benchmark.
Don't complain about performance until you have benchmarks of that sort.

I have plenty of benchmarks of that sort. Look how you must butcher the Java
code to make this ray tracer 60% slower than the fastest languages:

http://www.ffconsultancy.com/languages/ray_tracer/

In the cases where Java can attain competitive performance (e.g. with 2x the
performance of OCaml), you must apply so many optimizations by hand that
the result would be totally unmaintainable for any real program.
 
J

Jon Harrop

Mark said:
Oddly you mention allocation, which is often extremely fast in Java
relative to C, especially in multithreaded systems. Despite a
superficial resemblance to C++, Java's performance characteristics are
significantly different (and that does not mean slow). My work is
computationally intensive, yet has not been hampered at all by using Java.

Here is an allocation-bound benchmark: allocate an array of "n" pairs of
floats.

In OCaml:

Array.init (int_of_string Sys.argv.(1))
(fun i -> (float i, float i +. 1.))

In Java:

public class Bench {
static final long CLOCKS_PER_SEC = 1000;

public final class Pair {
private Object fst;
private Object snd;
public Pair(Object fst, Object snd) {this.fst=fst; this.snd=snd;}
public Object getFirst() { return fst; }
public Object getSecond() { return snd; }
}

public void run(int n) {
Pair[] a = new Pair[n];
for (int i=0; i<n; ++i)
a = new Pair((double)i, (double)i + 1);
}

public static void main(String[] args) {
long start = System.currentTimeMillis();
int n = Integer.parseInt(args[0]);
(new Bench()).run(n);
long end = System.currentTimeMillis();
System.out.printf("%dms\n",
(int) ((end - start) * 1000 / CLOCKS_PER_SEC));
}
}

For n = 2x10^6. OCaml:

$ time ./bench_ocaml 2000000

real 0m1.674s
user 0m1.500s
sys 0m0.172s
$ time java Bench 2000000
3750ms

real 0m3.930s
user 0m4.688s
sys 0m0.592s

Not only does Java require several times as much code to get this trivial
task done, it is >2x slower at doing it.
Advice on Java performance is more easily obtained if your posts are
phrased so as not to look like yet another troll out to trash Java.

Sure. Unless I'm pandering to existing Java users by building products for
them, there's no way I'm going to choose to use Java for real work because
it suffers from so many of these design flaws.

Incidentally, .NET chose many of the same trade-offs (e.g. parallel GC) but
is typically much faster than Java. So there is still quite a bit of scope
for optimization of the JVM implementation even within the design
constraints.
 
M

Mark Thornton

Jon said:
I have plenty of benchmarks of that sort. Look how you must butcher the Java
code to make this ray tracer 60% slower than the fastest languages:

http://www.ffconsultancy.com/languages/ray_tracer/

I suspect that using a JVM that does escape analysis to allocate objects
on the stack where possible would change the performance of the
ray_tracer quite significantly. There is at least one JVM available
which does this.

Mark Thornton
 
M

Mark Thornton

Jon said:
I see no evidence that Hotspot improves the performance of longer-running
programs. I see a lot of people stating that, but no hard numbers.

You haven't looked very hard, and you have declared that you aren't
interested in the answer anyway.
 
M

Mark Thornton

Jon said:
Mark said:
Oddly you mention allocation, which is often extremely fast in Java
relative to C, especially in multithreaded systems. Despite a
superficial resemblance to C++, Java's performance characteristics are
significantly different (and that does not mean slow). My work is
computationally intensive, yet has not been hampered at all by using Java.

Here is an allocation-bound benchmark: allocate an array of "n" pairs of
floats.

In OCaml:

Array.init (int_of_string Sys.argv.(1))
(fun i -> (float i, float i +. 1.))

In Java:

public class Bench {
static final long CLOCKS_PER_SEC = 1000;

public final class Pair {
private Object fst;
private Object snd;
public Pair(Object fst, Object snd) {this.fst=fst; this.snd=snd;}
public Object getFirst() { return fst; }
public Object getSecond() { return snd; }
}

public void run(int n) {
Pair[] a = new Pair[n];
for (int i=0; i<n; ++i)
a = new Pair((double)i, (double)i + 1);
}

public static void main(String[] args) {
long start = System.currentTimeMillis();
int n = Integer.parseInt(args[0]);
(new Bench()).run(n);
long end = System.currentTimeMillis();
System.out.printf("%dms\n",
(int) ((end - start) * 1000 / CLOCKS_PER_SEC));
}
}

For n = 2x10^6. OCaml:

$ time ./bench_ocaml 2000000

real 0m1.674s
user 0m1.500s
sys 0m0.172s
$ time java Bench 2000000
3750ms

real 0m3.930s
user 0m4.688s
sys 0m0.592s

Not only does Java require several times as much code to get this trivial
task done, it is >2x slower at doing it.


Do you know what you are measuring in the Java case? Try rerunning with

time java -Xms32m Bench 2000000

Java's allocation IS fast. However the cost of garbage collection
depends mostly on the number and size of live objects. I find gc very
valuable, but none of your examples benefit from it (was that your
deliberate choice).

Mark Thornton
 
S

Steve Wampler

Jon said:
The only thing flawed are your measurements for C. ....
$ time ./t1
s has 1047552l (1047552) characters

real 0m0.005s
user 0m0.004s
sys 0m0.004s

Perhaps you would like to try again.

I wrote:

Of course, this one is (on my machine) over 700 times *faster* in
Java than in C.

There was a reason for the parenthetical remark. Benchmarks are
dependent on a *lot* of factors beyond the code they contain, and
rarely simply measure one *language* against another. On my machine, with
gcc 3.4.6 and Sun's JDK 1.6, the times I get are the times I
posted. Had I used gcc4, for example, the optimizations would be different.
However, perhaps I could also find a Java system that optimizes that
code differently as well (JET, perhaps?) "Simple, flawless" 'benchmarks'
are particularly susceptible to environmental differences and should
never be used to make blanket statements about languages.

I've made *no* claim about the relative performance of Java and C, but
agree with others that there are plenty of applications where Java is
certainly "fast enough". And I find its ease of use *compared to C and
C++* superior for the tasks I have to write. I have no doubt that others
would disagree *even for those same tasks* and am sure that, for at
least some of those tasks, I could find an easier-to-use language as
well. So what?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,756
Messages
2,569,533
Members
45,007
Latest member
OrderFitnessKetoCapsules

Latest Threads

Top