Re: FW: [Tutor] Multi-Threading and KeyboardInterrupt

Discussion in 'Python' started by Dennis Lee Bieber, Jun 12, 2009.

  1. On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
    <> declaimed the following in
    gmane.comp.python.general:

    > I sent this to the Tutor mailing list and did not receive a response.
    > Perhaps one of you might be able to offer some sagely wisdom or pointed
    > remarks?
    >
    > Please reply off-list and thanks in advance. Code examples are below in
    > plain text.
    >

    Sorry -- you post to a public forum, expect to get the response on a
    public forum...

    > > My program runs interactively by allowing the user to directly
    > > interact with the python prompt. This program has a runAll() method
    > > that runs a series of subprocesses with a cap on how many instances
    > > are running at a time. My intent is to allow the user to use Ctrl-C to
    > > break these subprocesses. Note that while not reflected in the demo


    Are they subprocesses or threads? Your sample code seems to be using
    threads.

    When using threads, there is no assurance that any thread other than
    the main program will receive a keyboard interrupt. If true
    subprocesses, they may see a keyboard interrupt IF stdin is connected to
    that subprocess AND the subprocess is the current active process when
    the keyboard interrupt was detected.

    > def runAll():
    > workers = [ Thread(target = runSingle, args = )
    > for i in xrange(MAX_SUBPROCS + 1) ]
    > try:
    > for w in workers:
    > w.start()
    > except KeyboardInterrupt:
    > ## I want this to be shown on a KeyboardInterrupt
    > print '********* stopped midway ********'


    You are unlikely to see that... After you start the defined worker
    /threads/ (which doesn't take very long -- all threads will be started,
    but some may immediately block on the semaphore) this block will exit
    and you will be at...

    > for w in workers:
    > w.join()
    >

    ... a .join() call, which is the most likely position at which the
    keyboard interrupt will be processed, killing the main program thread
    and probably generating some errors as dangling active threads are
    forceably killed.
    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
    Dennis Lee Bieber, Jun 12, 2009
    #1
    1. Advertising

  2. On Jun 12, 3:35 pm, Dennis Lee Bieber <> wrote:
    > On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
    > <> declaimed the following in
    > gmane.comp.python.general:
    >
    > > I sent this to the Tutor mailing list and did not receive a response.
    > > Perhaps one of you might be able to offer some sagely wisdom or pointed
    > > remarks?

    >
    > > Please reply off-list and thanks in advance. Code examples are below in
    > > plain text.

    >
    >         Sorry -- you post to a public forum, expect to get the response on a
    > public forum...


    Bbit off topic, but if you are a proponent of public forums, how come
    your post on Google Groups shows:

    """Note: The author of this message requested that it not be archived.
    This message will be removed from Groups in 6 days (Jun 19, 3:35
    pm)."""

    I always find it a tad annoying that people have this, as you loose
    posts from the record of a discussion and if that post carries useful
    information, it is lost.

    Am also curious as to what mechanism for posting you use that allows
    you to set an expiration time on the post anyway. Are you using old
    usenet news reader or something?

    Graham
    Graham Dumpleton, Jun 12, 2009
    #2
    1. Advertising

  3. On Thu, 11 Jun 2009 23:29:10 -0700 (PDT), Graham Dumpleton
    <> declaimed the following in
    gmane.comp.python.general:

    > On Jun 12, 3:35 pm, Dennis Lee Bieber <> wrote:
    > > On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
    > > <> declaimed the following in
    > > gmane.comp.python.general:
    > >
    > > > I sent this to the Tutor mailing list and did not receive a response.
    > > > Perhaps one of you might be able to offer some sagely wisdom or pointed
    > > > remarks?

    > >
    > > > Please reply off-list and thanks in advance. Code examples are below in
    > > > plain text.

    > >
    > >         Sorry -- you post to a public forum, expect to get the response on a
    > > public forum...

    >
    > Bbit off topic, but if you are a proponent of public forums, how come
    > your post on Google Groups shows:
    >
    > """Note: The author of this message requested that it not be archived.
    > This message will be removed from Groups in 6 days (Jun 19, 3:35
    > pm)."""
    >
    > I always find it a tad annoying that people have this, as you loose
    > posts from the record of a discussion and if that post carries useful
    > information, it is lost.
    >
    > Am also curious as to what mechanism for posting you use that allows
    > you to set an expiration time on the post anyway. Are you using old
    > usenet news reader or something?
    >


    While I've not upgraded it recently, I wouldn't call it "old"...

    And yes, what else would I use to access three different NNTP
    servers -- downloading messages in the background... Posting all replies
    in a batch prior to my shutting down for the night...

    A web browser where I have to give it all my attention for each
    message? Peruse all the unfiltered junk that shows up? And have to guess
    which topics have new messages? With practically no editing control, and
    no capability to hold a draft reply for later...

    As for the use of X-Noarchive headers... Back before Google, most
    NNTP servers had automatic expiration of messages (on my ISP at the
    time, the binary groups expired after 24 hours!, most messages expired
    after one or two weeks). I never used X-Noarchive -- in truth, I didn't
    even know about it.

    THEN... Google started doing Google Groups with no automatic
    expiration. That is when the subject of X-Noarchive headers came up in
    some of the groups I'd been following. I and a fair number of others
    immediately took advantage of it.
    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
    Dennis Lee Bieber, Jun 13, 2009
    #3
  4. Re: [Tutor] Multi-Threading and KeyboardInterrupt

    On Thu, 11 Jun 2009 22:35:15 -0700
    Dennis Lee Bieber <> wrote:

    > On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
    > <> declaimed the following in
    > gmane.comp.python.general:
    >
    > > I sent this to the Tutor mailing list and did not receive a response.
    > > Perhaps one of you might be able to offer some sagely wisdom or pointed
    > > remarks?
    > >
    > > Please reply off-list and thanks in advance. Code examples are below in
    > > plain text.
    > >

    > Sorry -- you post to a public forum, expect to get the response on a
    > public forum...
    >
    > > > My program runs interactively by allowing the user to directly
    > > > interact with the python prompt. This program has a runAll() method
    > > > that runs a series of subprocesses with a cap on how many instances
    > > > are running at a time. My intent is to allow the user to use Ctrl-C to
    > > > break these subprocesses. Note that while not reflected in the demo

    >
    > Are they subprocesses or threads? Your sample code seems to be using
    > threads.
    >
    > When using threads, there is no assurance that any thread other than
    > the main program will receive a keyboard interrupt.


    In fact, no thread other than the main will get interrupt.


    > > def runAll():
    > > workers = [ Thread(target = runSingle, args = )
    > > for i in xrange(MAX_SUBPROCS + 1) ]
    > > try:
    > > for w in workers:
    > > w.start()
    > > except KeyboardInterrupt:
    > > ## I want this to be shown on a KeyboardInterrupt
    > > print '********* stopped midway ********'

    >
    > You are unlikely to see that... After you start the defined worker
    > /threads/ (which doesn't take very long -- all threads will be started,
    > but some may immediately block on the semaphore) this block will exit
    > and you will be at...
    >
    > > for w in workers:
    > > w.join()

    >
    > ... a .join() call, which is the most likely position at which the
    > keyboard interrupt will be processed, killing the main program thread
    > and probably generating some errors as dangling active threads are
    > forceably killed.


    There was quite interesting explaination of what happens when you send
    ^C with threads, posted on concurrency-sig list recently:

    http://blip.tv/file/2232410
    http://www.dabeaz.com/python/GIL.pdf

    Can be quite shocking, but my experience w/ threads only confirms that.


    --
    Mike Kazantsev // fraggod.net

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.11 (GNU/Linux)

    iEYEARECAAYFAkozOF0ACgkQASbOZpzyXnGzZQCfftmsS/fraJ/XYe9r915gmU4I
    negAmwdiSIH6gtYfbvZ/R5DDxuUF5O6d
    =oNGI
    -----END PGP SIGNATURE-----
    Mike Kazantsev, Jun 13, 2009
    #4
  5. Dennis Lee Bieber

    koranthala Guest

    Re: Multi-Threading and KeyboardInterrupt

    On Jun 13, 10:25 am, Mike Kazantsev <> wrote:
    > On Thu, 11 Jun 2009 22:35:15 -0700
    > Dennis Lee Bieber <> wrote:
    >
    >
    >
    > > On Thu, 11 Jun 2009 08:44:24 -0500, "Strax-Haber, Matthew (LARC-D320)"
    > > <> declaimed the following in
    > > gmane.comp.python.general:

    >
    > > > I sent this to the Tutor mailing list and did not receive a response.
    > > > Perhaps one of you might be able to offer some sagely wisdom or pointed
    > > > remarks?

    >
    > > > Please reply off-list and thanks in advance. Code examples are below in
    > > > plain text.

    >
    > >    Sorry -- you post to a public forum, expect to get the response on a
    > > public forum...

    >
    > > > > My program runs interactively by allowing the user to directly
    > > > > interact with the python prompt. This program has a runAll() method
    > > > > that runs a series of subprocesses with a cap on how many instances
    > > > > are running at a time. My intent is to allow the user to use Ctrl-C to
    > > > > break these subprocesses. Note that while not reflected in the demo

    >
    > >    Are they subprocesses or threads? Your sample code seems to be using
    > > threads.

    >
    > >    When using threads, there is no assurance that any thread other than
    > > the main program will receive a keyboard interrupt.

    >
    > In fact, no thread other than the main will get interrupt.
    >
    >
    >
    > > > def runAll():
    > > >     workers = [ Thread(target = runSingle, args = )
    > > >             for i in xrange(MAX_SUBPROCS + 1) ]
    > > >     try:
    > > >         for w in workers:
    > > >             w.start()
    > > >     except KeyboardInterrupt:
    > > >         ## I want this to be shown on a KeyboardInterrupt
    > > >         print '********* stopped midway ********'

    >
    > >    You are unlikely to see that... After you start the defined worker
    > > /threads/ (which doesn't take very long -- all threads will be started,
    > > but some may immediately block on the semaphore) this block will exit
    > > and you will be at...

    >
    > > >     for w in workers:
    > > >         w.join()

    >
    > >    ... a .join() call, which is the most likely position at which the
    > > keyboard interrupt will be processed, killing the main program thread
    > > and probably generating some errors as dangling active threads are
    > > forceably killed.

    >
    > There was quite interesting explaination of what happens when you send
    > ^C with threads, posted on concurrency-sig list recently:
    >
    >  http://blip.tv/file/2232410
    >  http://www.dabeaz.com/python/GIL.pdf
    >
    > Can be quite shocking, but my experience w/ threads only confirms that.
    >
    > --
    > Mike Kazantsev // fraggod.net
    >
    >  signature.asc
    > < 1KViewDownload


    Thank you very much for the link Mike.
    Are there other videos/audio like this? I am learning more from these
    videos than by experience alone.
    I did find one - http://www.awaretek.com/python/ - are there other
    links?
    koranthala, Jun 13, 2009
    #5
  6. Re: Multi-Threading and KeyboardInterrupt

    On Sat, 13 Jun 2009 04:42:16 -0700 (PDT)
    koranthala <> wrote:

    > Are there other videos/audio like this? I am learning more from these
    > videos than by experience alone.


    Indeed, it is a very interesting presentation, but I'm afraid I've
    stumbled upon it just as you did, but on concurrency-sig mailing list.

    It's a relatively new list (now hosted on mail.python.org), not
    specifically dedicated to podcasts or, for that matter, any
    implementation details. I haven't seen any other material like this
    there.

    > I did find one - http://www.awaretek.com/python/ - are there other
    > links?


    Thanks for sharing this link, although I prefer such information in
    written form - it's easier/faster to work with and much more accessible.

    --
    Mike Kazantsev // fraggod.net

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.11 (GNU/Linux)

    iEYEARECAAYFAkoznS4ACgkQASbOZpzyXnE/7wCgrRLpSZy7tASinsNabZMyPzdt
    pQQAoKMeS7I9dmbvYAqg9JOwHjjSgxmY
    =d89p
    -----END PGP SIGNATURE-----
    Mike Kazantsev, Jun 13, 2009
    #6
  7. Dennis Lee Bieber

    Aahz Guest

    Off-topic: Usenet archiving history (was Re: FW: [Tutor] Multi-Threading and KeyboardInterrupt)

    In article <>,
    Dennis Lee Bieber <> wrote:
    >
    > As for the use of X-Noarchive headers... Back before Google, most
    >NNTP servers had automatic expiration of messages (on my ISP at the
    >time, the binary groups expired after 24 hours!, most messages expired
    >after one or two weeks). I never used X-Noarchive -- in truth, I didn't
    >even know about it.
    >
    > THEN... Google started doing Google Groups with no automatic
    >expiration. That is when the subject of X-Noarchive headers came up in
    >some of the groups I'd been following. I and a fair number of others
    >immediately took advantage of it.


    Actually, it's not really fair to blame Google for this, they were only
    following the design of DejaNews, which they purchased in 2001. Google
    did create some additional value by finding old Usenet archives that
    predated DejaNews. (Google Groups, which I normally refer to as Gooja in
    deference to the history, debuted the archive on Dec 10, 2001.)
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Many customs in this life persist because they ease friction and promote
    productivity as a result of universal agreement, and whether they are
    precisely the optimal choices is much less important." --Henry Spencer
    Aahz, Jun 14, 2009
    #7
  8. Re: Off-topic: Usenet archiving history (was Re: FW: [Tutor]Multi-Threading and KeyboardInterrupt)

    On 13 Jun 2009 21:22:58 -0700, (Aahz) declaimed the
    following in gmane.comp.python.general:

    > Actually, it's not really fair to blame Google for this, they were only
    > following the design of DejaNews, which they purchased in 2001. Google
    > did create some additional value by finding old Usenet archives that
    > predated DejaNews. (Google Groups, which I normally refer to as Gooja in
    > deference to the history, debuted the archive on Dec 10, 2001.)


    Mea culpa -- I had a feeling there was something just before
    Google... But they sort of made it really visible <G>

    Either way -- it was still a change from "expiration at some
    date"... Though since (Netcom/Mindspring)Earthlink seems to have
    subcontracted NNTP service to Giganews (or some such) it wouldn't
    surprise me to learn that service also keeps a mammoth archive...
    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
    Dennis Lee Bieber, Jun 14, 2009
    #8
  9. Dennis Lee Bieber

    David Bolen Guest

    Re: Off-topic: Usenet archiving history

    Dennis Lee Bieber <> writes:

    > Either way -- it was still a change from "expiration at some
    > date"... Though since (Netcom/Mindspring)Earthlink seems to have
    > subcontracted NNTP service to Giganews (or some such) it wouldn't
    > surprise me to learn that service also keeps a mammoth archive...


    I'm not sure it's really a change, or if it is, it certainly isn't a
    change from how things were originally. "Expiration at some date" was
    never any sort of global policy for Usenet - just an aspect of a
    individual news server. Some servers held messages for long periods,
    particularly for the big seven groups - it's true that alt.* and in
    particular the binaries, might expire quickly. I know I certainly ran
    some servers that didn't bother expiring - or had expiration times in
    years - of the big seven. My experience post-dates the great
    renaming, so I can't speak to before that, but don't think behavior
    was very different.

    Individual messages could include an Expires: header if they wished,
    but even that was just a suggestion. Any actual expiration was due to
    local configuration on each news server, which while it could take
    Expires: headers into account, was just as often driven by local
    storage availability or the whims of the local news admin :)

    I think Deja News was providing web access to their archive from the
    mid-90s on (so quite a while before Google even existed) so certainly
    by that point everyone had access to a rather complete archive even if
    messages had expired on their local server. I think Deja was also the
    first to introduce X-No-Archive. But other archives certainly existed
    pre-Deja, which I'm sure is, in large part, how Google was able to
    locate and incorporate the older messages into their system after
    their acquisition of the Deja archive.

    -- David
    David Bolen, Jun 14, 2009
    #9
  10. Dennis Lee Bieber

    David Bolen Guest

    Re: Off-topic: Usenet archiving history

    Ben Finney <> writes:

    > David Bolen <> writes:
    >
    >> Individual messages could include an Expires: header if they wished,

    >
    > Since we're already well off-topic: NNTP, HTTP, and email, and probably
    > other protocols as well, all deal with messages. They are all consistent
    > in defining a message [0] as having *exactly one* header.


    Heh, I'm not sure it's quite as consistent as you may think,
    particularly with older RFCs, which are relevant in this discussion
    since we're talking about historical artifacts.

    For example, while more recent mail RFCs like 2822 may specifically
    talk about header fields as the "header" (singular) of the message,
    the older RFC 822 instead refers to a "headers" (plural) section.

    > Every time you call a field from the header “a headerâ€, or refer to
    > the plural “headers of a messageâ€, the IETF kills a kitten. You
    > don't want to hurt a kitten, do you?


    Heaven forbid - though I'd think I could hold my own with the IETF.
    My reference to "header" was in lieu of "header line", something that
    the Usenet RFCs (1036, and the older 850) do extensively themselves.

    But I'll be more careful in the future - need to ensure kitten safety!

    -- David
    David Bolen, Jun 14, 2009
    #10
  11. Dennis Lee Bieber

    OdarR Guest

    Re: Multi-Threading and KeyboardInterrupt

    On 13 juin, 07:25, Mike Kazantsev <> wrote:
    > There was quite interesting explaination of what happens when you send
    > ^C with threads, posted on concurrency-sig list recently:
    >
    >  http://blip.tv/file/2232410
    >  http://www.dabeaz.com/python/GIL.pdf
    >
    > Can be quite shocking, but my experience w/ threads only confirms that.


    Hi there,
    please read this package page (in 2.6), this is very interesting.
    http://docs.python.org/library/multiprocessing.html

    I tested it : it works. Multi-core cpu's are happy :)

    me too,
    Olivier
    OdarR, Jun 15, 2009
    #11
  12. Re: Multi-Threading and KeyboardInterrupt

    On Mon, 15 Jun 2009 05:37:14 -0700 (PDT)
    OdarR <> wrote:

    > On 13 juin, 07:25, Mike Kazantsev <> wrote:
    > > There was quite interesting explaination of what happens when you send
    > > ^C with threads, posted on concurrency-sig list recently:
    > >
    > >  http://blip.tv/file/2232410
    > >  http://www.dabeaz.com/python/GIL.pdf
    > >
    > > Can be quite shocking, but my experience w/ threads only confirms that.

    >
    > Hi there,
    > please read this package page (in 2.6), this is very interesting.
    > http://docs.python.org/library/multiprocessing.html
    >
    > I tested it : it works. Multi-core cpu's are happy :)


    I'd certainly prefer using processes because they indeed work
    flawlessly in that respect, but threads are way simplier and much more
    integrated into the language, so I can avoid re-imlementing tons of
    shared stuff, IPC and locking by using threads which bassically run in
    the same context.
    That might mean 90% of code for trivial but parallel task.

    Alas, they don't work flawlessly in all cases, but there is still
    million and one use for them.

    --
    Mike Kazantsev // fraggod.net

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.11 (GNU/Linux)

    iEYEARECAAYFAko2boMACgkQASbOZpzyXnFRLACfea6yd/iUNWMdHvKHHD201yzo
    hQ0AoOQIJaLofS8twruq3HJnlykeCgVI
    =lKpq
    -----END PGP SIGNATURE-----
    Mike Kazantsev, Jun 15, 2009
    #12
  13. Dennis Lee Bieber

    Matt Guest

    Re: Multi-Threading and KeyboardInterrupt

    I'm going to use the multipocessing library from here forward so I can
    take advantage of multiple cores and clusters. Either one should work
    for my use, since in my non-demonstration code each thread spends most
    of it's time waiting for a separate non-Python subprocess (created
    with subprocess.Popen) to finish anyway. (I guess Python would see
    this as IO-blocking) Therefore, if you can fix my toy example with
    threading, that's fine.

    DB.py, followed by a KeyboardInterrupt yields the output in a.out. I
    want roughly the output in desired.out.

    What do I need to do to modify this code to get my desired output and
    corresponding functionality? It would be a shame if this wasn't
    possible in any pure-Python way.

    ~Matt

    On Jun 15, 2009, at 11:53 AM, Mike Kazantsev wrote:

    > On Mon, 15 Jun 2009 05:37:14 -0700 (PDT)
    > OdarR <> wrote:
    >
    >> On 13 juin, 07:25, Mike Kazantsev <> wrote:
    >>> There was quite interesting explaination of what happens when you
    >>> send
    >>> ^C with threads, posted on concurrency-sig list recently:
    >>>
    >>> http://blip.tv/file/2232410
    >>> http://www.dabeaz.com/python/GIL.pdf
    >>>
    >>> Can be quite shocking, but my experience w/ threads only confirms
    >>> that.

    >>
    >> Hi there,
    >> please read this package page (in 2.6), this is very interesting.
    >> http://docs.python.org/library/multiprocessing.html
    >>
    >> I tested it : it works. Multi-core cpu's are happy :)

    >
    > I'd certainly prefer using processes because they indeed work
    > flawlessly in that respect, but threads are way simplier and much more
    > integrated into the language, so I can avoid re-imlementing tons of
    > shared stuff, IPC and locking by using threads which bassically run in
    > the same context.
    > That might mean 90% of code for trivial but parallel task.
    >
    > Alas, they don't work flawlessly in all cases, but there is still
    > million and one use for them.
    >
    > --
    > Mike Kazantsev // fraggod.net
    > --
    > http://mail.python.org/mailman/listinfo/python-list

    I'm going to use the multipocessing library from here forward so I can
    take advantage of multiple cores and clusters. Either one should work
    for my use, since in my non-demonstration code each thread spends most
    of it's time waiting for a separate non-Python subprocess (created
    with subprocess.Popen) to finish anyway. (I guess Python would see
    this as IO-blocking) DB.py, followed by a KeyboardInterrupt yields the
    output in a.out. I want roughly the output in desired.out.

    What do I need to do to modify this code to get my desired output and
    corresponding functionality? It would be a shame if this wasn't
    possible in any pure-Python way.

    ~Matt

    On Jun 15, 2009, at 11:53 AM, Mike Kazantsev wrote:

    > On Mon, 15 Jun 2009 05:37:14 -0700 (PDT)
    > OdarR <> wrote:
    >
    >> On 13 juin, 07:25, Mike Kazantsev <> wrote:
    >>> There was quite interesting explaination of what happens when you
    >>> send
    >>> ^C with threads, posted on concurrency-sig list recently:
    >>>
    >>> http://blip.tv/file/2232410
    >>> http://www.dabeaz.com/python/GIL.pdf
    >>>
    >>> Can be quite shocking, but my experience w/ threads only confirms
    >>> that.

    >>
    >> Hi there,
    >> please read this package page (in 2.6), this is very interesting.
    >> http://docs.python.org/library/multiprocessing.html
    >>
    >> I tested it : it works. Multi-core cpu's are happy :)

    >
    > I'd certainly prefer using processes because they indeed work
    > flawlessly in that respect, but threads are way simplier and much more
    > integrated into the language, so I can avoid re-imlementing tons of
    > shared stuff, IPC and locking by using threads which bassically run in
    > the same context.
    > That might mean 90% of code for trivial but parallel task.
    >
    > Alas, they don't work flawlessly in all cases, but there is still
    > million and one use for them.
    >
    > --
    > Mike Kazantsev // fraggod.net
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    Matt, Jun 15, 2009
    #13
  14. Re: Multi-Threading and KeyboardInterrupt

    On Mon, 15 Jun 2009 15:43:13 -0400
    Matt <> wrote:

    > I'm going to use the multipocessing library from here forward so I can
    > take advantage of multiple cores and clusters. Either one should work
    > for my use, since in my non-demonstration code each thread spends most
    > of it's time waiting for a separate non-Python subprocess (created
    > with subprocess.Popen) to finish anyway. (I guess Python would see
    > this as IO-blocking) Therefore, if you can fix my toy example with
    > threading, that's fine.
    >
    > DB.py, followed by a KeyboardInterrupt yields the output in a.out. I
    > want roughly the output in desired.out.
    >
    > What do I need to do to modify this code to get my desired output and
    > corresponding functionality? It would be a shame if this wasn't
    > possible in any pure-Python way.


    I don't know how complex task you have, but solving trivial IO blocks
    with threads or subprocesses look either like ugly hack or an overkill
    to me.

    Why not just use I/O without blocking?
    It's not 80s or 90s anymore, where you had to create subprocess to
    handle every non-synchronous task, and since the main burden will be
    pushed into non-py subprocesses already, why not implement controller
    as a nice, clean and simple single-threaded event loop?

    Consider this recipe:
    http://code.activestate.com/recipes/576759/

    And if the task before you is complex indeed, involving more than just
    two to five child processes with a simple "while True: ..." loop,
    consider using twisted framework - it'll allow you to do incredible
    stuff with any number of sockets with just few lines of code in a
    clean, abstracted way.
    Latter would also mean that you can always replace os pipes with network
    sockets just by changing transport name, distributing your app to any
    number of machines.

    --
    Mike Kazantsev // fraggod.net

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.11 (GNU/Linux)

    iEYEARECAAYFAko3KbUACgkQASbOZpzyXnGj7ACg0J+pQdYyWvK2AYzjkj5G1qTN
    4uEAn34gjpS8QHJroq6psj1dMi40M0kq
    =+8oo
    -----END PGP SIGNATURE-----
    Mike Kazantsev, Jun 16, 2009
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ivan Nestlerode

    KeyboardInterrupt and threading

    Ivan Nestlerode, Jan 2, 2004, in forum: Python
    Replies:
    8
    Views:
    513
    Anand Pillai
    Jan 8, 2004
  2. Replies:
    38
    Views:
    1,256
    Dennis Lee Bieber
    Feb 15, 2005
  3. PantherSE
    Replies:
    0
    Views:
    329
    PantherSE
    May 16, 2005
  4. akineko
    Replies:
    3
    Views:
    2,597
    Jesse Noller
    Jan 29, 2009
  5. Strax-Haber, Matthew (LARC-D320)

    FW: [Tutor] Multi-Threading and KeyboardInterrupt

    Strax-Haber, Matthew (LARC-D320), Jun 11, 2009, in forum: Python
    Replies:
    0
    Views:
    956
    Strax-Haber, Matthew (LARC-D320)
    Jun 11, 2009
Loading...

Share This Page