Re: Please help with Threading

Discussion in 'Python' started by Fábio Santos, May 20, 2013.

  1. My use case was a tight loop processing an image pixel by pixel, or
    crunching a CSV file. If it only uses local variables (and probably hold a
    lock before releasing the GIL) it should be safe, no?

    My idea is that it's a little bad to have to write C or use multiprocessing
    just to do simultaneous calculations. I think an application using a
    reactor loop such as twisted would actually benefit from this. Sure, it
    will be slower than a C implementation of the same loop, but isn't fast
    prototyping a very important feature of the Python language?
    On 20 May 2013 08:45, "Cameron Simpson" <> wrote:

    > On 20May2013 07:25, Fábio Santos <> wrote:
    > | On 18 May 2013 20:33, "Dennis Lee Bieber" <> wrote:
    > | > Python threads work fine if the threads either rely on
    > intelligent
    > | > DLLs for number crunching (instead of doing nested Python loops to
    > | > process a numeric array you pass it to something like NumPy which
    > | > releases the GIL while crunching a copy of the array) or they do lots
    > of
    > | > I/O and have to wait for I/O devices (while one thread is waiting for
    > | > the write/read operation to complete, another thread can do some number
    > | > crunching).
    > |
    > | Has nobody thought of a context manager to allow a part of your code to
    > | free up the GIL? I think the GIL is not inherently bad, but if it posesa
    > | problem at times, there should be a way to get it out of your... Way.
    >
    > The GIL makes individual python operations thread safe by never
    > running two at once. This makes the implementation of the operations
    > simpler, faster and safer. It is probably totally infeasible to
    > write meaningful python code inside your suggested context
    > manager that didn't rely on the GIL; if the GIL were not held the
    > code would be unsafe.
    >
    > It is easy for a C extension to release the GIL, and then to do
    > meaningful work until it needs to return to python land. Most C
    > extensions will do that around non-trivial sections, and anything
    > that may stall in the OS.
    >
    > So your use case for the context manager doesn't fit well.
    > --
    > Cameron Simpson <>
    >
    > Gentle suggestions being those which are written on rocks of less than
    > 5lbs.
    > - Tracy Nelson in comp.lang.c
    >
    Fábio Santos, May 20, 2013
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. KK
    Replies:
    2
    Views:
    552
    Big Brian
    Oct 14, 2003
  2. Replies:
    9
    Views:
    1,036
    Mark Space
    Dec 29, 2007
  3. silkenpy
    Replies:
    1
    Views:
    353
    Dennis Lee Bieber
    Feb 15, 2008
  4. Steven Woody
    Replies:
    0
    Views:
    406
    Steven Woody
    Jan 9, 2009
  5. Steven Woody
    Replies:
    0
    Views:
    446
    Steven Woody
    Jan 9, 2009
Loading...

Share This Page