Re: C-extension 2 times slower than exe

Discussion in 'Python' started by Philip Semanchuk, Jun 23, 2009.

  1. On Jun 23, 2009, at 9:51 AM, Rolf Wester wrote:

    > Hi,
    >
    > I have a C++ program that I would like to steer using Python. I made
    > the
    > wrapper using swig and linked the code (without the main function)
    > into
    > a shared object. My Python script loads the extension and calls a
    > function of the C-extension, the rest runs entirely within the
    > C-extension. For comparison I compiled the code including the main
    > function with the same compilation options and linked all into an exe.
    > The main function of the exe calls the same function as my Python
    > script
    > does. Surprisingly the code in the Python C-extension runs twice as
    > long
    > as the same code in the exe. Does anyone know what could be the reason
    > for this behaviour?


    If the runtime of the C/C++ code is short, the time spent initializing
    the Python interpreter might have a big impact on the runtime of the
    Python version.
     
    Philip Semanchuk, Jun 23, 2009
    #1
    1. Advertising

  2. Rolf Wester wrote:
    > The runtime is about 2.5 sec and 5.0 sec respectively. I not only use
    > the time command to measure the time consumed but I also measure the
    > time within the C-code using clock() and get similar result. So the
    > Python startup time is not the reason for the runtime difference I
    > found. Thank you anyway.


    Without a profiler it's hard to say where the time is being consumed.
    You might want to use ctypes and open your c++ code as a DLL without the
    python extension wrapper.
     
    Michael Torrie, Jun 23, 2009
    #2
    1. Advertising

  3. Philip Semanchuk

    Carl Banks Guest

    On Jun 23, 7:20 am, Rolf Wester <> wrote:
    > Philip Semanchuk wrote:
    >
    > > On Jun 23, 2009, at 9:51 AM, Rolf Wester wrote:

    >
    > >> Hi,

    >
    > >> I have a C++ program that I would like to steer using Python. I made the
    > >> wrapper using swig and linked the code (without the main function) into
    > >> a shared object. My Python script loads the extension and calls a
    > >> function of the C-extension, the rest runs entirely within the
    > >> C-extension. For comparison I compiled the code including the main
    > >> function with the same compilation options and linked all into an exe.
    > >> The main function of the exe calls the same function as my Python script
    > >> does. Surprisingly the code in the Python C-extension runs twice as long
    > >> as the same code in the exe. Does anyone know what could be the reason
    > >> for this behaviour?

    >
    > > If the runtime of the C/C++ code is short, the time spent initializing
    > > the Python interpreter might have a big impact on the runtime of the
    > > Python version.

    >
    > The runtime is about 2.5 sec and 5.0 sec respectively. I not only use
    > the time command to measure the time consumed but I also measure the
    > time within the C-code using clock() and get similar result. So the
    > Python startup time is not the reason for the runtime difference I
    > found. Thank you anyway.


    We can't really help you much unless you give us more details about
    what you did (how did you built it, how did you time it, and how are
    you calling the C extension from Python). All we can do is throw out
    possible ideas, and I will toss out a few, but if that's not it you'll
    have to post details.

    Are you building the extension with the same optimization flags as the
    compiler?

    Are you calling the C code repeatedly from inside a Python loop?
     
    Carl Banks, Jun 24, 2009
    #3
  4. Philip Semanchuk

    MRAB Guest

    Nick Craig-Wood wrote:
    > Rolf Wester <> wrote:
    >> Hello,
    >>
    >> thank you all very much for your replies.
    >>
    >> I tried to simplify things and make the two versions as comparable as
    >> possible. I put the C++ part of the program into a shared object
    >> libff.so. For the exe the main function is linked against this shared
    >> object. For the python stuff I made an interface consiting of only one
    >> function call_solver with the same code that has the main function used
    >> for the exe. Then I created a wrapper for this interface using swig and
    >> linked interface.o, ff_warp.o and libff.so into _ff.so. The Python code
    >> just imports _ff and calls call_solver wich creates an object of the
    >> class Solver and calls its member solve (the main function of the exe
    >> does the same).
    >>
    >> I included some code for timing into the C++-code.
    >>
    >> #include <time.h>
    >>
    >> //beginning of solve
    >> clock_t t0 = clock();
    >> ...
    >> clock_t t1 = clock();
    >> //end of solve
    >> cout << "time used = " << (t1-t0)/CLOCKS_PER_SEC << endl;
    >>
    >> I'm using gcc4.5 (latest snapshot) and Python2.6 under Suse 10.3. The
    >> sources are compiled using the flags -fPIC -O3.
    >>
    >> Timing:
    >>
    >> 1) time python ff.py
    >> time used = 3.74
    >> real 0m3.234s
    >> user 0m3.712s
    >> sys 0m0.192s

    >
    > Those times look odd because the user time is > than the real time.
    >
    > User time is number of CPU seconds used. Real time is wallclock time.
    >
    > That must mean
    > a) your program is threading
    > b) there is something up with timing on your computer
    >
    > Looks odd but exactly what it means I don't know!
    >
    >> 2) time ff
    >> time used = 2.19
    >> real 0m3.170s
    >> user 0m2.088s
    >> sys 0m0.168s

    >

    Perhaps multithreading on dual cores?
     
    MRAB, Jun 25, 2009
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. John Rivers

    VS.NET is 10 times slower than VB6

    John Rivers, Aug 23, 2005, in forum: ASP .Net
    Replies:
    91
    Views:
    3,177
    =?Utf-8?B?TWFuaWthbmRhbg==?=
    Nov 2, 2005
  2. croeltgen
    Replies:
    1
    Views:
    512
    Andrew Thompson
    Oct 25, 2004
  3. Replies:
    15
    Views:
    526
    Roy Harvey
    Oct 26, 2006
  4. Jack Steven
    Replies:
    2
    Views:
    457
    Chris Rebert
    Mar 9, 2009
  5. sanket
    Replies:
    7
    Views:
    1,055
    Tsung
    Nov 3, 2011
Loading...

Share This Page