basic thread question

Discussion in 'Python' started by Derek Martin, Aug 18, 2009.

  1. Derek Martin

    Derek Martin Guest

    I have some simple threaded code... If I run this
    with an arg of 1 (start one thread), it pegs one cpu, as I would
    expect. If I run it with an arg of 2 (start 2 threads), it uses both
    CPUs, but utilization of both is less than 50%. Can anyone explain
    why?

    I do not pretend it's impeccable code, and I'm not looking for a
    critiqe of the code per se, excepting the case where what I've written
    is actually *wrong*. I hacked this together in a couple of minutes,
    with the intent of pegging my CPUs. Performance with two threads is
    actually *worse* than with one, which is highly unintuitive. I can
    accomplish my goal very easily with bash, but I still want to
    understand what's going on here...

    The OS is Linux 2.6.24, on a Ubuntu base. Here's the code:

    Thanks

    -=-=-=-=-

    #!/usr/bin/python

    import thread, sys, time

    def busy(thread):
    x=0
    while True:
    x+=1

    if __name__ == '__main__':
    try:
    cpus = int(sys.argv[1])
    except ValueError:
    cpus = 1
    print "cpus = %d, argv[1] = %s\n" % (cpus, sys.argv[1])
    i=0
    thread_list = []
    while i < cpus:
    x = thread.start_new_thread(busy, (i,))
    thread_list.append(x)
    i+=1
    while True:
    pass

    --
    Derek D. Martin
    http://www.pizzashack.org/
    GPG Key ID: 0x81CFE75D


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.5 (GNU/Linux)

    iD8DBQFKiwqndjdlQoHP510RAtMjAJwMI77kgLeJ6BwPCzpa0X0S+ufNDACfXI5j
    JwDqKbMirN5bknUDvZ8yWYQ=
    =KbeP
    -----END PGP SIGNATURE-----
    Derek Martin, Aug 18, 2009
    #1
    1. Advertising

  2. Derek Martin

    birdsong Guest

    On Aug 18, 1:10 pm, Derek Martin <> wrote:
    > I have some simple threaded code...  If I run this
    > with an arg of 1 (start one thread), it pegs one cpu, as I would
    > expect.  If I run it with an arg of 2 (start 2 threads), it uses both
    > CPUs, but utilization of both is less than 50%.  Can anyone explain
    > why?  
    >
    > I do not pretend it's impeccable code, and I'm not looking for a
    > critiqe of the code per se, excepting the case where what I've written
    > is actually *wrong*. I hacked this together in a couple of minutes,
    > with the intent of pegging my CPUs.  Performance with two threads is
    > actually *worse* than with one, which is highly unintuitive.  I can
    > accomplish my goal very easily with bash, but I still want to
    > understand what's going on here...
    >
    > The OS is Linux 2.6.24, on a Ubuntu base.  Here's the code:
    >
    > Thanks
    >
    > -=-=-=-=-
    >
    > #!/usr/bin/python
    >
    > import thread, sys, time
    >
    > def busy(thread):
    >     x=0
    >     while True:
    >         x+=1
    >
    > if __name__ == '__main__':
    >     try:
    >         cpus = int(sys.argv[1])
    >     except ValueError:
    >         cpus = 1
    >     print "cpus = %d, argv[1] = %s\n" % (cpus, sys.argv[1])
    >     i=0
    >     thread_list = []
    >     while i < cpus:
    >         x = thread.start_new_thread(busy, (i,))
    >         thread_list.append(x)
    >         i+=1
    >     while True:
    >         pass
    >
    > --
    > Derek D. Martinhttp://www.pizzashack.org/
    > GPG Key ID: 0x81CFE75D
    >
    >  application_pgp-signature_part
    > < 1KViewDownload


    watch this and all your findings will be explained: http://blip.tv/file/2232410

    this talk marked a pivotal moment in my understanding of python
    threads and signal handling in threaded programs.
    birdsong, Aug 18, 2009
    #2
    1. Advertising

  3. Derek Martin

    sturlamolden Guest

    On 18 Aug, 22:10, Derek Martin <> wrote:

    > I have some simple threaded code...  If I run this
    > with an arg of 1 (start one thread), it pegs one cpu, as I would
    > expect.  If I run it with an arg of 2 (start 2 threads), it uses both
    > CPUs, but utilization of both is less than 50%.  Can anyone explain
    > why?  


    Access to the Python interpreter is serialized by the global
    interpreter lock (GIL). You created two threads the OS could schedule,
    but they competed for access to the Python interpreter. If you want to
    utilize more than one CPU, you have to release the GIL or use multiple
    processes instead (os.fork since you are using Linux).

    This is how the GIL can be released:

    * Many functions in Python's standard library, particularly all
    blocking i/o functions, release the GIL. This covers the by far most
    common use of threads.

    * In C or C++ extensions, use the macros Py_BEGIN_ALLOW_THREADS and
    Py_END_ALLOW_THREADS.

    * With ctypes, functions called from a cdll release the GIL, whereas
    functions called from a pydll do not.

    * In f2py, declaring a Fortran function threadsafe in a .pyf file or
    cf2py comment releases the GIL.

    * In Cython or Pyrex extensions, use a "with nogil:" block to execute
    code without holding the GIL.


    Sturla Molden
    sturlamolden, Aug 24, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. john
    Replies:
    4
    Views:
    565
    Lee Fesperman
    Jun 3, 2005
  2. Engineer
    Replies:
    6
    Views:
    605
    Jeremy Bowers
    May 1, 2005
  3. Replies:
    0
    Views:
    417
  4. Jan Kaliszewski

    Re: basic thread question

    Jan Kaliszewski, Aug 18, 2009, in forum: Python
    Replies:
    10
    Views:
    501
    sturlamolden
    Aug 25, 2009
  5. Derek Martin

    Re: basic thread question

    Derek Martin, Aug 18, 2009, in forum: Python
    Replies:
    2
    Views:
    276
    Sean DiZazzo
    Aug 20, 2009
Loading...

Share This Page