Anti-benchmarking clauses

[Philippe]

It was interesting to read some synthesis benchmarking results on
comp.lang.vhdl last week. I feel it's high time that EDA vendors drop
the anti-benchmarking clauses from their license agreements:

http://www.sigasi.com/content/your-milage-may-vary-lot

To summarize my points:

1. There is no clear definition of what a "benchmark result" is, you
don't know when you are breaching the contract.
2. Public benchmarking hinders competition and innovation.
3. Customers deserve to be treated with more respect.

I'd love to hear your thoughts, either on this newsgroup, or on my
webpage.


kind regards

 

[Brian Davis]

Philippe posted:

1. There is no clear definition of what a "benchmark result" is,
you don't know when you are breaching the contract.

I agree that those clauses are a bit much.

FYI, about a decade ago, there was some vendor
clarification regarding this issue on comp.arch.fpga
in the context of posting _single_ vendor Brand S
result variations vs. tool settings/coding style:

In 2001, I had posted [1] a summary of LUT counts vs.
tool settings on a thread about big counters:

And tweaking counter size/target frequency, gives:

Synplify Synplify
CNT_MSB Frequency LUT count
__________________________________________
55 77 57
55 78 110

31 95 33
31 96 46
31 122 46
31 123 83



Uh oh! The person who posted Synplify's results for that
56-bit counter problem is in line for an ass-whooping!

I replied [4] in part:

Personally, I wouldn't consider answering a question on how
to make a counter synthesize better to be a "benchmark"...

Then Ken M. of Brand S, formerly known as Brand S, posted [5]:

We don't consider it to be a "benchmark" either.

Brian Davis


[1] post from thread "High level synthesis will never work well"
http://groups.google.com/group/comp.arch.fpga/msg/01e9d02b8e85983d

[2] Thread "Synplicity/Leonardo License Agreement Information"
http://groups.google.com/group/comp.arch.fpga/browse_frm/thread/a201fc7f4a639215

[3] post from thread "Synplicity/Leonardo License Agreement
Information"
http://groups.google.com/group/comp.arch.fpga/msg/6f9694e180581202

[4] post from thread "Synplicity/Leonardo License Agreement
Information"
http://groups.google.com/group/comp.arch.fpga/msg/172b3fbcb1ffe4a5

[5] post from thread "Synplicity/Leonardo License Agreement
Information"
http://groups.google.com/group/comp.arch.fpga/msg/52f4c2a915b383d3

 

[Philippe]

Hi Brian,

Thanks for pointing to this conversation from 2001. It seems that EDA
vendors are not interested in enforcing this clause for small
benchmark results.
Still it gives them a stick to hit you with if you would ever publish
something serious. That leaves al the benchmarking work for the brave
and those with little to loose.

 

[Kolja Sulimma]

It was interesting to read some synthesis benchmarking results on
comp.lang.vhdl last week. I feel it's high time that EDA vendors drop
the anti-benchmarking clauses from their license agreements:

Just make sure to purchase software in a way where that license
agreement ist not included in a contract.
A sales contract is finalized when goods and money have been
exchanged. Afterwards no clauses can be
added to the contract by one side alone.
At least in Germany it is well established by court that this holds
for software sales, and other countries have similar contract law.

So as long as the clause is not included in a click through contract
during a download purchase, or is presented
to you before purchase in another way, the clause does not become part
of the contract.
Clauses presented during installation are irrelevant.

Also note: Your contract is with the reseller, not with the vendor of
the software. So information on the vendor homepage
is irrelavant.

For private users (not companies) the clause anyway is likely to
violate EU law, because it is a surprising clause.

Kolja

 

[geobsd]

why do you complain about you restricted softwares while the hardwer
is much more closed ?

 

[Paul Colin Gloster]

Philippe <[email protected]> sent on March 8th, 2011:

|---------------------------------------------------------------------|
|"It was interesting to read some synthesis benchmarking results on |
|comp.lang.vhdl last week. I feel it's high time that EDA vendors drop|
|the anti-benchmarking clauses from their license agreements: |
| |
|[..]" |
|---------------------------------------------------------------------|

Who could feel confident about products which are not subjected to as
much scrutiny as benchmarks from the Standard Performance Evaluation
Corporation?