Asynchronous urllib (urllib+asyncore)?

Discussion in 'Python' started by Jonathan Gardner, Feb 26, 2008.

  1. So, I ran into a problem that I would like to write as little code as
    possible to solve.

    The problem is that I would like to send out a bunch of HTTP requests
    simultaneously, using asynchronous techniques, and then do stuff with
    the results in parallel. Think of something like Google's map-reduce.

    It sounds like this would be something someone else has done before. I
    searched google, but I don't think there is any discussion anywhere of
    anyone doing something like this.

    I am sure Twisted folks can code this up in 2 lines or less, or it
    probably already exists there by default. But I need this to exist in
    a non-asynchronous environment. I can't let Twisted take over
    everything.

    So, has someone done something like this? Is it something where I can
    bring the urllib and asyncore together? Or is it much more involved?
     
    Jonathan Gardner, Feb 26, 2008
    #1
    1. Advertising

  2. Jonathan Gardner

    Terry Jones Guest

    Re: European python developers

    Hi Jerry

    > I am hoping one or two members of this list might help me locate in Europe


    Do you mean you want to re-locate here or that you're trying to locate
    (i.e., find) people already here?

    > My personal first choice is Spain


    I'm in Barcelona (but not looking for work!). There are a couple of
    companies around that I know of who use Python and quite a lot of
    individual programmers.

    > The point is that I can't really afford to run around and experiment, so I
    > was hoping for some helpful comments or suggestions on how to research


    I'm not sure what you're wanting to research. See first comment above.

    Terry
     
    Terry Jones, Feb 27, 2008
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael Welsh

    asyncore asynchat

    Michael Welsh, Jul 15, 2003, in forum: Python
    Replies:
    0
    Views:
    538
    Michael Welsh
    Jul 15, 2003
  2. Garth
    Replies:
    0
    Views:
    397
    Garth
    Jul 24, 2003
  3. Chris
    Replies:
    3
    Views:
    1,341
    Ng Pheng Siong
    Aug 28, 2003
  4. Stéphane Ninin

    asyncore question

    Stéphane Ninin, Sep 12, 2003, in forum: Python
    Replies:
    6
    Views:
    520
    A.M. Kuchling
    Sep 29, 2003
  5. Chris McDonald
    Replies:
    0
    Views:
    327
    Chris McDonald
    Nov 1, 2010
Loading...

Share This Page