Re: Please help with Threading

Discussion in 'Python' started by Fábio Santos, May 20, 2013.

  1. My use case was a tight loop processing an image pixel by pixel, or
    crunching a CSV file. If it only uses local variables (and probably hold a
    lock before releasing the GIL) it should be safe, no?

    My idea is that it's a little bad to have to write C or use multiprocessing
    just to do simultaneous calculations. I think an application using a
    reactor loop such as twisted would actually benefit from this. Sure, it
    will be slower than a C implementation of the same loop, but isn't fast
    prototyping a very important feature of the Python language?
    On 20 May 2013 08:45, "Cameron Simpson" <> wrote:

    > On 20May2013 07:25, Fábio Santos <> wrote:
    > | On 18 May 2013 20:33, "Dennis Lee Bieber" <> wrote:
    > | > Python threads work fine if the threads either rely on
    > intelligent
    > | > DLLs for number crunching (instead of doing nested Python loops to
    > | > process a numeric array you pass it to something like NumPy which
    > | > releases the GIL while crunching a copy of the array) or they do lots
    > of
    > | > I/O and have to wait for I/O devices (while one thread is waiting for
    > | > the write/read operation to complete, another thread can do some number
    > | > crunching).
    > |
    > | Has nobody thought of a context manager to allow a part of your code to
    > | free up the GIL? I think the GIL is not inherently bad, but if it posesa
    > | problem at times, there should be a way to get it out of your... Way.
    > The GIL makes individual python operations thread safe by never
    > running two at once. This makes the implementation of the operations
    > simpler, faster and safer. It is probably totally infeasible to
    > write meaningful python code inside your suggested context
    > manager that didn't rely on the GIL; if the GIL were not held the
    > code would be unsafe.
    > It is easy for a C extension to release the GIL, and then to do
    > meaningful work until it needs to return to python land. Most C
    > extensions will do that around non-trivial sections, and anything
    > that may stall in the OS.
    > So your use case for the context manager doesn't fit well.
    > --
    > Cameron Simpson <>
    > Gentle suggestions being those which are written on rocks of less than
    > 5lbs.
    > - Tracy Nelson in comp.lang.c
    Fábio Santos, May 20, 2013
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. KK
    Big Brian
    Oct 14, 2003
  2. Replies:
    Mark Space
    Dec 29, 2007
  3. silkenpy
    Dennis Lee Bieber
    Feb 15, 2008
  4. Steven Woody
    Steven Woody
    Jan 9, 2009
  5. Steven Woody
    Steven Woody
    Jan 9, 2009

Share This Page