Re: to py or not to py ?

Discussion in 'Python' started by Carl J. Van Arsdall, Jun 28, 2006.

  1. Robert Kern wrote:
    >> Carl , what are the problems that could arise with threading ??
    >>

    Because of the GIL only one thread can actually run at a time. So if
    you are going for speed (say you have an SMP box) and your two requests
    require significant computation, you'd want each processor to take on a
    little bit of that work to get things done faster. Now say you want
    these two to be computed simultaneously and you are using python, you
    won't get that, one thread will process and then the other will process
    as the GIL is passed around between the python threads. Now if your
    bottleneck is IO then you are in good shape for python threads (Your
    machine only has one network port and you don't have a lot of
    computation overhead, so your single processor can serve things up
    faster than your network card can take it).

    So its not problems that arise, its more like inefficiency. Anyhow, it
    all depends on what you are trying to do and the needs of your
    application.

    -carl

    --

    Carl J. Van Arsdall

    Build and Release
    MontaVista Software
    Carl J. Van Arsdall, Jun 28, 2006
    #1
    1. Advertising

  2. Carl J. Van Arsdall

    Tom Plunket Guest

    Carl J. Van Arsdall wrote:

    > Because of the GIL only one thread can actually run at a time.


    I've recently been wondering about this, since in the work I do, a lot
    of time is spent doing disk I/O. So if I want the UI to remain
    responsive, I could spawn an IO thread to handle requests, and do a
    pretty simple "just whack new requests onto the queue" without locks
    since I'm guaranteed to not have the IO thread read at the same time
    as the requestor thread?

    ....what exactly constitutes an atomic operation in Python, anyway?

    e.g.

    class IoThread:
    # ...

    # called from the other thread...
    def RequestFile(self, name):
    self.fileQueue.append(name)

    # called during the IO thread
    def GetNextFile(self);
    next = self.fileQueue[0]
    self.fileQueue.pop(0)
    return next

    ?
    -tom!
    Tom Plunket, Jun 28, 2006
    #2
    1. Advertising

  3. Carl J. Van Arsdall

    Steve Holden Guest

    Tom Plunket wrote:
    > Carl J. Van Arsdall wrote:
    >
    >
    >>Because of the GIL only one thread can actually run at a time.

    >
    >
    > I've recently been wondering about this, since in the work I do, a lot
    > of time is spent doing disk I/O. So if I want the UI to remain
    > responsive, I could spawn an IO thread to handle requests, and do a
    > pretty simple "just whack new requests onto the queue" without locks
    > since I'm guaranteed to not have the IO thread read at the same time
    > as the requestor thread?
    >
    > ...what exactly constitutes an atomic operation in Python, anyway?
    >

    Standard (and excellent) advice to those new to Python: use threading,
    not thread, and have threads communicate using Queue.Queue(s).

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC/Ltd http://www.holdenweb.com
    Love me, love my blog http://holdenweb.blogspot.com
    Recent Ramblings http://del.icio.us/steve.holden
    Steve Holden, Jun 29, 2006
    #3
  4. Tom Plunket wrote:
    > Carl J. Van Arsdall wrote:
    >
    >
    >> Because of the GIL only one thread can actually run at a time.
    >>

    >
    > I've recently been wondering about this, since in the work I do, a lot
    > of time is spent doing disk I/O. So if I want the UI to remain
    > responsive, I could spawn an IO thread to handle requests, and do a
    > pretty simple "just whack new requests onto the queue" without locks
    > since I'm guaranteed to not have the IO thread read at the same time
    > as the requestor thread?
    >
    > ...what exactly constitutes an atomic operation in Python, anyway?
    >
    >


    Well, although only one thread can run at a time due to the GIL you
    can't accurately predict when the GIL is going to be released and
    therefore you don't know when another thread is going to pick up and
    start going (GIL is released on every so many byte instructs - correct
    me if i'm wrong, certain operations that have a bit to do with IO, and
    modules you wrote yourself where you manually release the GIL using
    macros provided in the C API). If you have your own data structure that
    is shared among threads you can use the threading modules
    synchronization constructs to get the job done, using locks, conditions,
    and events. Queue.Queue is also a good way to go about communicating
    with threads.



    > e.g.
    >
    > class IoThread:
    > # ...
    >
    > # called from the other thread...
    > def RequestFile(self, name):
    > self.fileQueue.append(name)
    >
    > # called during the IO thread
    > def GetNextFile(self);
    > next = self.fileQueue[0]
    > self.fileQueue.pop(0)
    > return next
    >
    > ?
    > -tom!
    >



    --

    Carl J. Van Arsdall

    Build and Release
    MontaVista Software
    Carl J. Van Arsdall, Jun 29, 2006
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Roxanne
    Replies:
    0
    Views:
    1,231
    Roxanne
    Jul 4, 2003
  2. Andy Elmhorst
    Replies:
    2
    Views:
    482
    Bassel Tabbara [MSFT]
    Jul 8, 2003
  3. Kristian Domke
    Replies:
    11
    Views:
    505
    George Sakkis
    Jan 23, 2008
  4. Stephanie Stowe
    Replies:
    2
    Views:
    217
    Peter X
    Apr 7, 2004
  5. Ruby Freak

    To be not, or not to be not?

    Ruby Freak, Sep 23, 2008, in forum: Ruby
    Replies:
    2
    Views:
    137
    Thomas B.
    Sep 23, 2008
Loading...

Share This Page