Another C++ performance article

[Cholo Lennon]

May be this post is a bit off topic:

Do you think this article is fair with C++?

http://www.codeproject.com/Articles/304935/Operation-Performance-Evaluation

 

[Jorgen Grahn]

May be this post is a bit off topic:

Do you think this article is fair with C++?

http://www.codeproject.com/Articles/304935/Operation-Performance-Evaluation

"Investigating the cost of an operation in cycles within a real
world, i.e., no peak, performance measurement of C#, C++, Java,
Fortran and JavaScript"

It's on topic, meaningless, and will be discussed here ad nauseum,
just like all other benchmarks have in the past.

/Jorgen

 

[MikeWhy]

"Investigating the cost of an operation in cycles within a real
world, i.e., no peak, performance measurement of C#, C++, Java,
Fortran and JavaScript"

It's on topic, meaningless, and will be discussed here ad nauseum,
just like all other benchmarks have in the past.

I'll start... The reason to examine computation overhead is an overriding
concern about system performance. I care much more about total throughput
and latency rather than cycle counts of individual arithmetic operations.
The benchmark is simplistic, uninteresting, and unenlightening by failing to
exercise and account for memory bandwidth, cache misses, and instruction
order optimizations available on modern processors.

Real world computation intensive applications will run in multiple threads,
not all involved in computation. Hyperthreading might or might not prove
meaningful in the analysis. A truly good optimizer can leverage the
consecutive locations of parallel arrays of values, to minimize stalling on
cache misses. Throughput will almost certainly differ with more than the 3
values involved, and again if the related values are stored as cache line
aligned structs rather than separate arrays.

More to the point, and not at all incidentally, the benchmark test is
trivially and ideally suited for implementation in CUDA or other gpGPU. It
falls into the category of being embarrassingly parallel. Even a clumsy
implementation effectively measures memory bandwidth rather than computation
cost.

With this as context, the better metric of efficiency is percentage
occupancy of available memory bandwidth, not the cycle cost of individual
arithmetic operations. A benchmark that would interest me is one that
relates memory bandwith to throughput of various implementations in CUDA,
arrays, packed structs, and CPU (not GPU) threading schemes.

 

[Cholo Lennon]

The benchmark is simplistic, uninteresting, and
unenlightening by failing to exercise and account for memory bandwidth,
cache misses, and instruction order optimizations available on modern
processors.

I thought the same at first glance. That's why I wanted to share it here
in order to confirm my initial opinion: Another test where an author
fails to set up a valid benchmark/experiment. The benchmark is very
simple and general. The funny side is the article has received almost
five stars from the readers.

 

[Bo Persson]

May be this post is a bit off topic:

Do you think this article is fair with C++?

http://www.codeproject.com/Articles/304935/Operation-Performance-Evaluation

Any C++ benchmark starting like this

int main()
{
int* A = new int[SIZE];
int* B = new int[SIZE];
int* C = new int[SIZE];

is obviously not serious. LOL!Bo Persson

 

[Juha Nieminen]

Do you think this article is fair with C++?

IMO no benchmark between more than one programming language is fair for
any of those languages (except for the winner in that particular benchmark,
of course).

Different programming languages tend to require different approaches
when talking about efficient code. One of the most common mistakes in
a multi-language benchmark is to try to match the code as exactly as
possible in all the languages, using as similar constructs as possible
in each one of them. (It's a rather common misconception that "implement
the same task in all the languages" means "match every single construct
as closely as possible in each language".) This will invariably favor
some of the languages over the others, where such constructs will
invariably not be the most optimal way to implement the task.

Also, more often than not, the creator of the multi-language benchmark
will be very proficient in one or two of the languages, and significantly
less experienced with the rest. This naturally gives an edge to those
languages he's more experienced with.

If each of the implementations is made by a very competent programmer
in said language, the benchmark may become a bit fairer. Even then, there
may be different opinions even among experts what is the "best" way to
implement something. Also, quite often, if this gives a more favorable
result to a disliked language, the argument will usually shift towards
the implementation in the popular language being "simpler" and "much
easier to implement" and "doesn't require you to know so much about the
intricacies of the language" and so on.