Re: Question about thread

Discussion in 'Python' started by Jp Calderone, Nov 19, 2004.

  1. Jp Calderone

    Jp Calderone Guest

    On Fri, 19 Nov 2004 22:50:17 +0800, Valkyrie <> wrote:
    >To be more precise, what I want to do is to have a threaded program to handle
    > some jobs concurrently (while those jobs are fetching information from the
    > Internet, so threading could speed up the process if the network is slow)
    >
    > start
    > | (blahblahblah...)
    > v
    > +-----+-----+-----+-----+
    > | | | | |
    > --+-- --+-- --+-- --+-- --+--
    > | | | | | | | | | |
    > | A | | B | | C | | D | | E |
    > | | | | | | | | | |
    > --+-- --+-- --+-- --+-- --+--
    > | | | | |
    > +-----+-----+-----+-----+
    > | (blahblahblah...)
    > v
    > finish!


    If your goal is efficient network concurrency, threads are a second-rate solution. Asynchronous IO is the winner: http://www.twistedmatrix.com/

    Here's an example (untested as usual):

    from twisted.web.client import downloadPage
    from twisted.internet import defer, reactor

    # Map URLs to files to which to save them
    URLs = {'www.google.com', 'google',
    # ...
    'wigu.com', 'wigu',
    }

    # Initiate the download and save of each page
    downloads = []
    for url in URLs:
    downloads.append(downloadPage(url, open(URLs, 'w'))) # Wait for all of the... # Start the reactor reactor.run() Jp
    Jp Calderone, Nov 19, 2004
    #1
    1. Advertising

  2. Jp Calderone

    Valkyrie Guest

    Thanks for your suggestion :) But I would like to have a more standard solution,
    I still don't know how to do so im mind...


    Jp Calderone wrote:
    > On Fri, 19 Nov 2004 22:50:17 +0800, Valkyrie <> wrote:
    >
    >>To be more precise, what I want to do is to have a threaded program to handle
    >>some jobs concurrently (while those jobs are fetching information from the
    >>Internet, so threading could speed up the process if the network is slow)
    >>
    >> start
    >> | (blahblahblah...)
    >> v
    >> +-----+-----+-----+-----+
    >> | | | | |
    >>--+-- --+-- --+-- --+-- --+--
    >>| | | | | | | | | |
    >>| A | | B | | C | | D | | E |
    >>| | | | | | | | | |
    >>--+-- --+-- --+-- --+-- --+--
    >> | | | | |
    >> +-----+-----+-----+-----+
    >> | (blahblahblah...)
    >> v
    >> finish!

    >
    >
    > If your goal is efficient network concurrency, threads are a second-rate solution. Asynchronous IO is the winner: http://www.twistedmatrix.com/
    >
    > Here's an example (untested as usual):
    >
    > from twisted.web.client import downloadPage
    > from twisted.internet import defer, reactor
    >
    > # Map URLs to files to which to save them
    > URLs = {'www.google.com', 'google',
    > # ...
    > 'wigu.com', 'wigu',
    > }
    >
    > # Initiate the download and save of each page
    > downloads = []
    > for url in URLs:
    > downloads.append(downloadPage(url, open(URLs, 'w'))) > > # Wait for all of...reactor > reactor.run() > > Jp[/color]
    Valkyrie, Nov 19, 2004
    #2
    1. Advertising

  3. Look at the example in the asyncore module in the standard Python
    library.

    <http://www.python.org/doc/current/lib/module-asyncore.html>


    /Jean Brouwers
    ProphICy Semiconductor, Inc.


    In article <1100878186.26229@eng-ser6>, Valkyrie <>
    wrote:

    > Thanks for your suggestion :) But I would like to have a more standard
    > solution,
    > I still don't know how to do so im mind...
    >
    >
    > Jp Calderone wrote:
    > > On Fri, 19 Nov 2004 22:50:17 +0800, Valkyrie <> wrote:
    > >
    > >>To be more precise, what I want to do is to have a threaded program to
    > >>handle
    > >>some jobs concurrently (while those jobs are fetching information from the
    > >>Internet, so threading could speed up the process if the network is slow)
    > >>
    > >> start
    > >> | (blahblahblah...)
    > >> v
    > >> +-----+-----+-----+-----+
    > >> | | | | |
    > >>--+-- --+-- --+-- --+-- --+--
    > >>| | | | | | | | | |
    > >>| A | | B | | C | | D | | E |
    > >>| | | | | | | | | |
    > >>--+-- --+-- --+-- --+-- --+--
    > >> | | | | |
    > >> +-----+-----+-----+-----+
    > >> | (blahblahblah...)
    > >> v
    > >> finish!

    > >
    > >
    > > If your goal is efficient network concurrency, threads are a second-rate
    > > solution. Asynchronous IO is the winner: http://www.twistedmatrix.com/
    > >
    > > Here's an example (untested as usual):
    > >
    > > from twisted.web.client import downloadPage
    > > from twisted.internet import defer, reactor
    > >
    > > # Map URLs to files to which to save them
    > > URLs = {'www.google.com', 'google',
    > > # ...
    > > 'wigu.com', 'wigu',
    > > }
    > >
    > > # Initiate the download and save of each page
    > > downloads = []
    > > for url in URLs:
    > > downloads.append(downloadPage(url, open(URLs, 'w'))) > > > > # Wait for al... reactor.run() > > > > Jp[/color][/color]
    Jean Brouwers, Nov 19, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Charles A. Lackman

    Terminating a thread from the main thread

    Charles A. Lackman, Dec 9, 2004, in forum: ASP .Net
    Replies:
    3
    Views:
    1,539
    Herfried K. Wagner [MVP]
    Dec 9, 2004
  2. pawo
    Replies:
    0
    Views:
    485
  3. Stephen Miller
    Replies:
    3
    Views:
    3,962
    Stephen Miller
    Jul 2, 2004
  4. Johanna
    Replies:
    0
    Views:
    601
    Johanna
    Oct 13, 2004
  5. vincent wehren
    Replies:
    0
    Views:
    430
    vincent wehren
    Dec 11, 2003
Loading...

Share This Page