gevent + urllib2 blocking

Discussion in 'Python' started by 月忧茗, Feb 22, 2013.

  1. Hi,

    One problem, thanks for help.


    import gevent.monkey
    gevent.monkey.match_all()

    from lxml import etree
    # I using xpath parse the html


    def _get(p):
    url = BUILD_URL(p)
    html = urllib2.urlopen(url)

    # RUN AT HERE AND BLOCKING

    # ver1
    tree = etree.parse(html, parse)

    # ver2
    # tree = etree.fromstring(html.read(), parse)

    ....


    workers = [gevent.spawn(_get, i) for i in xrange(1, 100)]
    gevent.joinall(workers)



    When using ver1, etree.parse(html, parse)
    I noticed there are only two greenlets runs at "RUN AT HERE AND BLOCKING"

    others are not running!


    But, when I using ver2, this code will running normally as expected.


    Why this ?
    月忧茗, Feb 22, 2013
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Josef Cihal
    Replies:
    0
    Views:
    747
    Josef Cihal
    Sep 5, 2005
  2. self.python
    Replies:
    4
    Views:
    809
    Damjan
    Jul 7, 2012
  3. subin

    concurrent.futures vs gevent

    subin, Jul 29, 2012, in forum: Python
    Replies:
    0
    Views:
    139
    subin
    Jul 29, 2012
  4. Tony Shao
    Replies:
    0
    Views:
    175
    Tony Shao
    Dec 29, 2012
  5. Roy Smith

    Weird problem with UDP and gevent

    Roy Smith, Oct 18, 2013, in forum: Python
    Replies:
    3
    Views:
    117
    Grant Edwards
    Oct 19, 2013
Loading...

Share This Page