Benchmark C vs C++

Discussion in 'C Programming' started by jacob navia, Oct 4, 2009.

  1. jacob navia

    jacob navia Guest

    Consider this benchmark

    --------------------------------------------------------C++
    #include <iostream>
    #include <list>
    using namespace std;

    // Simple example uses type int
    #define MAXIT 50000000
    int main(void)
    {
    list<int> L;
    int i;

    for (i=0; i<MAXIT;i++)
    L.push_back(i); // Insert a new element at the end
    list<int>::iterator it;
    int sum=0;
    for (it=L.begin(); it != L.end(); ++it)
    sum += *it;
    printf("%d\n",sum);
    return 0;
    }

    --------------------------------------------------------C
    /* compile with list.c and containererror.c */
    #include "containers.h"
    #include <stdio.h>
    int main(void)
    {
    #define MAX_IT 50000000

    List *l = newList(sizeof(int));
    list_element *le;
    int i,sum=0;
    for (i=0; i<MAX_IT; i++) {
    l->lpVtbl->Add(l,&i); // Insert at the end
    }
    for (le = GetFirst(l); le != NULL; le = GetNext(le)) {
    sum += *(int *)le->Data;
    }
    printf("%d\n",sum);
    return 0;
    }

    ---------------------------------------------------------

    I think it would be interesting to see the results in as much
    machines as possible.

    Thanks
    jacob navia, Oct 4, 2009
    #1
    1. Advertising

  2. jacob navia

    Ike Naar Guest

    In article <hab5re$16u$>, jacob navia <> wrote:
    >Consider this benchmark
    >
    >--------------------------------------------------------C++
    >#include <iostream>
    >#include <list>
    >using namespace std;
    >
    >// Simple example uses type int
    >#define MAXIT 50000000
    >int main(void)
    >{
    > list<int> L;
    > int i;
    >
    > for (i=0; i<MAXIT;i++)
    > L.push_back(i); // Insert a new element at the end
    > list<int>::iterator it;
    > int sum=0;
    > for (it=L.begin(); it != L.end(); ++it)
    > sum += *it;
    > printf("%d\n",sum);
    > return 0;
    >}
    >
    >--------------------------------------------------------C
    >/* compile with list.c and containererror.c */
    >#include "containers.h"
    >#include <stdio.h>
    >int main(void)
    >{
    >#define MAX_IT 50000000
    >
    > List *l = newList(sizeof(int));
    > list_element *le;
    > int i,sum=0;
    > for (i=0; i<MAX_IT; i++) {
    > l->lpVtbl->Add(l,&i); // Insert at the end
    > }
    > for (le = GetFirst(l); le != NULL; le = GetNext(le)) {
    > sum += *(int *)le->Data;
    > }
    > printf("%d\n",sum);
    > return 0;
    >}
    >
    >---------------------------------------------------------
    >
    >I think it would be interesting to see the results in as much
    >machines as possible.


    On a SUNW,UltraSPARC-IIIi, using the Sun C++ 5.9 and Sun C 5.9 compilers,
    the C++ version runs in 8.1 seconds CPU and needs 622 MB RAM,
    the C version (with list.c and containererror.c) runs in 16.4 seconds CPU
    and needs 1065 MB RAM.
    Ike Naar, Oct 4, 2009
    #2
    1. Advertising

  3. jacob navia

    jacob navia Guest

    For a PC with intel 8 core CPU+12GB RAM
    I see:

    cl -Ox -EHsc l1.cpp
    timethis l1.cpp
    TimeThis : Command Line : l1
    TimeThis : Start Time : Mon Oct 05 00:38:01 2009

    1283106752

    TimeThis : Command Line : l1
    TimeThis : Start Time : Mon Oct 05 00:38:01 2009
    TimeThis : End Time : Mon Oct 05 00:38:05 2009
    TimeThis : Elapsed Time : 00:00:04.167

    For the C version I see:
    timethis listtest

    TimeThis : Command Line : listtest
    TimeThis : Start Time : Mon Oct 05 00:36:46 2009

    1283106752

    TimeThis : Command Line : listtest
    TimeThis : Start Time : Mon Oct 05 00:36:46 2009
    TimeThis : End Time : Mon Oct 05 00:36:50 2009
    TimeThis : Elapsed Time : 00:00:04.049

    That version is compiled with lcc-win (lcc -O)
    jacob navia, Oct 4, 2009
    #3
  4. On Sun, 04 Oct 2009 23:59:19 +0200, jacob navia <> wrote:
    > Consider this benchmark
    >
    > --------------------------------------------------------C++
    > #include <iostream>
    > #include <list>
    > using namespace std;
    >
    > // Simple example uses type int
    > #define MAXIT 50000000
    > int main(void)
    > {
    > list<int> L;
    > int i;
    >
    > for (i=0; i<MAXIT;i++)
    > L.push_back(i); // Insert a new element at the end
    > list<int>::iterator it;
    > int sum=0;
    > for (it=L.begin(); it != L.end(); ++it)
    > sum += *it;
    > printf("%d\n",sum);
    > return 0;
    > }
    >
    > --------------------------------------------------------C
    > /* compile with list.c and containererror.c */
    > #include "containers.h"
    > #include <stdio.h>
    > int main(void)
    > {
    > #define MAX_IT 50000000
    >
    > List *l = newList(sizeof(int));
    > list_element *le;
    > int i,sum=0;
    > for (i=0; i<MAX_IT; i++) {
    > l->lpVtbl->Add(l,&i); // Insert at the end
    > }
    > for (le = GetFirst(l); le != NULL; le = GetNext(le)) {
    > sum += *(int *)le->Data;
    > }
    > printf("%d\n",sum);
    > return 0;
    > }
    >
    > ---------------------------------------------------------
    >
    > I think it would be interesting to see the results in as much
    > machines as possible.


    Hi Jacob,

    Can you please post a link where we can fetch the container files too?
    I've seen the related thread in c.l.c a few days ago, but the posts have
    expired from my newsreader cache and are no longer easy to find.

    A download link with a ZIP or plain tar(1) archive would be very nice.
    Giorgos Keramidas, Oct 5, 2009
    #4
  5. jacob navia

    jacob navia Guest

    > Hi Jacob,
    >
    > Can you please post a link where we can fetch the container files too?
    > I've seen the related thread in c.l.c a few days ago, but the posts have
    > expired from my newsreader cache and are no longer easy to find.
    >
    > A download link with a ZIP or plain tar(1) archive would be very nice.
    >


    I posted the files again 4-5 hours ago.

    They should still be there.

    jacob
    jacob navia, Oct 5, 2009
    #5
  6. jacob navia

    jacob navia Guest

    Ike Naar a écrit :
    > In article <hab5re$16u$>, jacob navia <> wrote:
    >> Consider this benchmark
    >>
    >> --------------------------------------------------------C++
    >> #include <iostream>
    >> #include <list>
    >> using namespace std;
    >>
    >> // Simple example uses type int
    >> #define MAXIT 50000000
    >> int main(void)
    >> {
    >> list<int> L;
    >> int i;
    >>
    >> for (i=0; i<MAXIT;i++)
    >> L.push_back(i); // Insert a new element at the end
    >> list<int>::iterator it;
    >> int sum=0;
    >> for (it=L.begin(); it != L.end(); ++it)
    >> sum += *it;
    >> printf("%d\n",sum);
    >> return 0;
    >> }
    >>
    >> --------------------------------------------------------C
    >> /* compile with list.c and containererror.c */
    >> #include "containers.h"
    >> #include <stdio.h>
    >> int main(void)
    >> {
    >> #define MAX_IT 50000000
    >>
    >> List *l = newList(sizeof(int));
    >> list_element *le;
    >> int i,sum=0;
    >> for (i=0; i<MAX_IT; i++) {
    >> l->lpVtbl->Add(l,&i); // Insert at the end
    >> }
    >> for (le = GetFirst(l); le != NULL; le = GetNext(le)) {
    >> sum += *(int *)le->Data;
    >> }
    >> printf("%d\n",sum);
    >> return 0;
    >> }
    >>
    >> ---------------------------------------------------------
    >>
    >> I think it would be interesting to see the results in as much
    >> machines as possible.

    >
    > On a SUNW,UltraSPARC-IIIi, using the Sun C++ 5.9 and Sun C 5.9 compilers,
    > the C++ version runs in 8.1 seconds CPU and needs 622 MB RAM,
    > the C version (with list.c and containererror.c) runs in 16.4 seconds CPU
    > and needs 1065 MB RAM.
    >


    The benchmark makes 50 million calls to malloc.

    If malloc is well done, this is the same as C++. If not, it will be quite
    bad performance.

    The performance of the C version hasn't been optimized at all.
    jacob navia, Oct 5, 2009
    #6
  7. jacob navia <> writes:

    > Consider this benchmark


    C++:
    > #include <list>


    C:
    > #include "containers.h"


    Your list is singly-linked is it not? The STL's list is a
    doubly-linked list and is therefore more flexible. Comparing them for
    speed alone is maybe a little biased.

    --
    Ben.
    Ben Bacarisse, Oct 5, 2009
    #7
  8. On Mon, 05 Oct 2009 01:43:02 +0200, jacob navia <> wrote:
    >> Hi Jacob,
    >> Can you please post a link where we can fetch the container files
    >> too? I've seen the related thread in c.l.c a few days ago, but the
    >> posts have expired from my newsreader cache and are no longer easy to
    >> find.
    >>
    >> A download link with a ZIP or plain tar(1) archive would be very nice.

    >
    > I posted the files again 4-5 hours ago.
    > They should still be there.


    Ok, I tracked these down and I just finished running 30 iterations of
    each test program on my laptop:

    : Machine: Thinkpad X61s
    : OS: FreeBSD 9.0-CURRENT [svn rev /head@197735]
    : Arch: i386
    : -- dmesg output --
    : CPU: Intel(R) Core(TM)2 Duo CPU L7700 @ 1.80GHz (1795.51-MHz
    : 686-class CPU)
    : Origin = "GenuineIntel" Id = 0x6fb Stepping = 11
    : Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
    : Features2=0xe3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
    : AMD Features=0x20100000<NX,LM>
    : AMD Features2=0x1<LAHF>
    : TSC: P-state invariant
    : real memory = 3221225472 (3072 MB)
    : avail memory = 3119816704 (2975 MB)

    Comparing the real/user/sys times of each program with the `ministat'
    utility shows that there is a marked difference in wall clock times and
    userland/user-level times:

    : Real Times
    : ==========
    :
    : x real-cpp
    : + real-stdc
    : +------------------------------------------------------------------------------+
    : | ++ x |
    : | ++ xx |
    : | ++ xx |
    : | ++ xx |
    : | ++ xx |
    : | ++ xx |
    : | ++ xx |
    : | ++ xx |
    : | +++ xx |
    : | +++ xx |
    : | +++ xxx |
    : | ++++ + xxx x x|
    : ||_MA_| |___MA____| |
    : +------------------------------------------------------------------------------+
    : N Min Max Median Avg Stddev
    : x 30 12.116 13.127 12.17 12.2203 0.20618943
    : + 30 9.833 10.526 9.907 9.9256667 0.11781205
    : Difference at 95.0% confidence
    : -2.29463 +/- 0.0867998
    : -18.7772% +/- 0.710292%
    : (Student's t, pooled s = 0.167919)


    : User Times
    : ==========
    :
    : x user-cpp
    : + user-stdc
    : +------------------------------------------------------------------------------+
    : | + |
    : | + |
    : | ++ |
    : | ++ x |
    : | +++ xx |
    : | +++ xxx |
    : | +++++ x xxxx |
    : | +++++ x x xxxxxx |
    : |+ ++++++ + x xx xxxxxxx x|
    : | |__MA__| |____A___| |
    : +------------------------------------------------------------------------------+
    : N Min Max Median Avg Stddev
    : x 30 10.241 11.347 10.579 10.569833 0.1915676
    : + 30 8.044 8.98 8.35 8.3697333 0.14194388
    : Difference at 95.0% confidence
    : -2.2001 +/- 0.0871474
    : -20.8149% +/- 0.824491%
    : (Student's t, pooled s = 0.168592)

    But a much smaller difference in system/kernel times:

    : System Times
    : ============
    :
    : x sys-cpp
    : + sys-stdc
    : +------------------------------------------------------------------------------+
    : | + ++ + |
    : | + x + ++ x ++ + xx x |
    : |x +* x + +* x xx**+* **++** + xx* x +x xxx x + xx|
    : | |_|_________MA_____M_A____|_____________| |
    : +------------------------------------------------------------------------------+
    : N Min Max Median Avg Stddev
    : x 30 1.279 1.786 1.484 1.4974333 0.12388724
    : + 30 1.309 1.735 1.438 1.4421 0.084948808
    : Difference at 95.0% confidence
    : -0.0553333 +/- 0.0549054
    : -3.69521% +/- 3.66663%
    : (Student's t, pooled s = 0.106218)
    Giorgos Keramidas, Oct 5, 2009
    #8
  9. jacob navia

    gwowen Guest

    On Oct 5, 12:44 am, jacob navia <> wrote:
    >
    > The benchmark makes 50 million calls to malloc.
    >
    > If malloc is well done, this is the same as C++. If not, it will be quite
    > bad performance.


    What happens in the case of memory exhaustion?
    In the C++ case, push_back() throws std::bad_alloc.
    For similar semantics, do you need to check a return value from
    l->lpVtbl->Add() ?
    gwowen, Oct 6, 2009
    #9
  10. jacob navia

    jacob navia Guest

    gwowen a écrit :
    > On Oct 5, 12:44 am, jacob navia <> wrote:
    >> The benchmark makes 50 million calls to malloc.
    >>
    >> If malloc is well done, this is the same as C++. If not, it will be quite
    >> bad performance.

    >
    > What happens in the case of memory exhaustion?
    > In the C++ case, push_back() throws std::bad_alloc.
    > For similar semantics, do you need to check a return value from
    > l->lpVtbl->Add() ?


    There are two mechanisms.
    First, a callback will be called with the name of the function that failed, and
    a numeric code indicating which error happened.

    Second, (if the callback returns) a NULL value will be returned.
    jacob navia, Oct 6, 2009
    #10
  11. jacob navia

    Ike Naar Guest

    In article <habc0c$790$>, jacob navia <> wrote:
    >Ike Naar a écrit :
    >> On a SUNW,UltraSPARC-IIIi, using the Sun C++ 5.9 and Sun C 5.9 compilers,
    >> the C++ version runs in 8.1 seconds CPU and needs 622 MB RAM,
    >> the C version (with list.c and containererror.c) runs in 16.4 seconds CPU
    >> and needs 1065 MB RAM.

    >
    >The benchmark makes 50 million calls to malloc.
    >If malloc is well done, this is the same as C++. If not, it will be quite
    >bad performance.


    In the above implementation, the C++ Standard Library uses malloc
    under the hood.
    Ike Naar, Oct 6, 2009
    #11
  12. jacob navia

    jacob navia Guest

    Ike Naar a écrit :
    > In article <habc0c$790$>, jacob navia <> wrote:
    >> Ike Naar a écrit :
    >>> On a SUNW,UltraSPARC-IIIi, using the Sun C++ 5.9 and Sun C 5.9 compilers,
    >>> the C++ version runs in 8.1 seconds CPU and needs 622 MB RAM,
    >>> the C version (with list.c and containererror.c) runs in 16.4 seconds CPU
    >>> and needs 1065 MB RAM.

    >> The benchmark makes 50 million calls to malloc.
    >> If malloc is well done, this is the same as C++. If not, it will be quite
    >> bad performance.

    >
    > In the above implementation, the C++ Standard Library uses malloc
    > under the hood.


    Sure, but not 50 million times. I am implementing a simple heap manager
    (that will be used by all containers) to avoid calling malloc 50
    million times.

    It was very instructive to see where the weak points are. For instance
    I tried to use a little memory as possible, but since I am allocating
    50 million small blocks of 8 bytes, overhead for malloc goes up to 100%!

    It is better to allocate bigger blocks and not try to optimize memory as
    this version does.

    Thanks for your interest

    jacob
    jacob navia, Oct 6, 2009
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ke zhang

    ASP.NET performance benchmark

    ke zhang, Nov 6, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    3,760
    John Timney \(Microsoft MVP\)
    Nov 7, 2003
  2. Assaf Harel
    Replies:
    0
    Views:
    405
    Assaf Harel
    Jan 28, 2004
  3. Roedy Green

    Benchmark

    Roedy Green, Jun 10, 2004, in forum: Java
    Replies:
    0
    Views:
    535
    Roedy Green
    Jun 10, 2004
  4. Jean-Francois Theoret

    JMS Benchmark - OpenJMS VS JBossMQ?

    Jean-Francois Theoret, Jul 6, 2004, in forum: Java
    Replies:
    0
    Views:
    1,857
    Jean-Francois Theoret
    Jul 6, 2004
  5. Razvan
    Replies:
    1
    Views:
    12,598
    Will Hartung
    Aug 19, 2004
Loading...

Share This Page