python extension, -pthreads and speed

Discussion in 'Python' started by Joseph Heled, Apr 23, 2004.

  1. Joseph Heled

    Joseph Heled Guest

    Hi,

    My python module is built using the recommended distutils.core, which
    uses the -pthread flag. To my amazement this slows down the (-O3) code
    by a factor of two (!2)

    My gcc documentation says pthread is a PowerPC flag, but I guess this
    is wrong.

    Would my code fail if I drop this flag (Assuming I don't use threads
    at all)?

    Why would there be such a speed penalty?

    Anyone can shed some light on that?

    Thanks, Joseph

    >python -V

    Python 2.3

    >gcc -v

    Reading specs from /usr/local/lib/gcc-lib/i686-pc-linux-gnu/3.3/specs
    Configured with: ../gcc-3.3/configure : (reconfigured)
    .../gcc-3.3/configure --enable-languages=c,c++
    Thread model: posix
    gcc version 3.3
     
    Joseph Heled, Apr 23, 2004
    #1
    1. Advertising

  2. On Fri, 23 Apr 2004 09:36:08 +0200, Joseph Heled wrote:

    > Hi,
    >
    > My python module is built using the recommended distutils.core, which
    > uses the -pthread flag. To my amazement this slows down the (-O3) code
    > by a factor of two (!2)
    >
    > My gcc documentation says pthread is a PowerPC flag, but I guess this is
    > wrong.
    >
    > Would my code fail if I drop this flag (Assuming I don't use threads at
    > all)?
    >
    > Why would there be such a speed penalty?
    >
    > Anyone can shed some light on that?
    >
    > Thanks, Joseph
    >
    >>python -V

    > Python 2.3
    >
    >>gcc -v

    > Reading specs from /usr/local/lib/gcc-lib/i686-pc-linux-gnu/3.3/specs
    > Configured with: ../gcc-3.3/configure : (reconfigured)
    > ../gcc-3.3/configure --enable-languages=c,c++ Thread model: posix gcc
    > version 3.3


    My understanding of your question is : my C extension module works slower
    when used through Python. If my guess is right, this isa possible hint
    on the subject.

    If your extension use IOs (fgets, sprintf...) or others standard library calls,
    they may be affected by the fact that Python is build with multithreading
    enabled, on so all those API start using mutexes to be thread safe, thus the
    extract cost. This simple C example exhibit the problem :

    #include <stdio.h>

    int main(int argc, char *argv[])
    {
    int i;
    FILE *in;

    in = fopen("/dev/null", "r");

    for (i=0; i< 10000000; i++) {
    fgetc(in);
    }

    close(in);
    }

    $ gcc z.c -O3 -o z
    $ time ./z

    real 0m12.355s
    user 0m5.810s
    sys 0m6.550s

    $ gcc z.c -O3 -o z -pthread
    $ time ./z

    real 0m16.055s
    user 0m10.610s
    sys 0m5.440s

    Cheers,
    Pedro
     
    Pedro Rodriguez, Apr 23, 2004
    #2
    1. Advertising

  3. Joseph Heled

    Rolf Magnus Guest

    Pedro Rodriguez wrote:

    > If your extension use IOs (fgets, sprintf...) or others standard
    > library calls, they may be affected by the fact that Python is build
    > with multithreading enabled, on so all those API start using mutexes
    > to be thread safe, thus the extract cost.


    Another candidate for such slowdowns is malloc.
     
    Rolf Magnus, Apr 25, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jim

    java and pthreads

    Jim, Sep 14, 2005, in forum: Java
    Replies:
    0
    Views:
    2,478
  2. Savagesmc
    Replies:
    3
    Views:
    460
    Savagesmc
    Nov 11, 2003
  3. Kevin Dahlhausen

    Pthreads and Extension Modules on Linux

    Kevin Dahlhausen, Nov 15, 2004, in forum: Python
    Replies:
    0
    Views:
    318
    Kevin Dahlhausen
    Nov 15, 2004
  4. Samuel M. Smith
    Replies:
    0
    Views:
    350
    Samuel M. Smith
    Nov 3, 2005
  5. blop

    pthreads and gdb?

    blop, Feb 23, 2006, in forum: C Programming
    Replies:
    5
    Views:
    486
    Randy Howard
    Feb 23, 2006
Loading...

Share This Page