Waiting for receiving data

Discussion in 'Python' started by Anjanesh Lekshminarayanan, Nov 23, 2009.

  1. fp = urllib.urlopen(url)
    data = fp.read()

    Retrieving XML data via an XML service API.
    Very often network gets stuck in between. No errors / exceptions.

    CTRL+C

    File "get-xml.py", line 32, in <module>
    fp = urllib.urlopen(url)
    File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
    return opener.open(url)
    File "/usr/lib/python2.6/urllib.py", line 206, in open
    return getattr(self, name)(url)
    File "/usr/lib/python2.6/urllib.py", line 348, in open_http
    errcode, errmsg, headers = h.getreply()
    File "/usr/lib/python2.6/httplib.py", line 1048, in getreply
    response = self._conn.getresponse()
    File "/usr/lib/python2.6/httplib.py", line 974, in getresponse
    response.begin()
    File "/usr/lib/python2.6/httplib.py", line 391, in begin
    version, status, reason = self._read_status()
    File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
    line = self.fp.readline()
    File "/usr/lib/python2.6/socket.py", line 397, in readline
    data = recv(1)
    KeyboardInterrupt

    Is there I can do to try something else if its taking too long to
    retrieve from the network ? Like kill previous attempt and retry ?

    Thanks
    Anjanesh Lekshmnarayanan
    Anjanesh Lekshminarayanan, Nov 23, 2009
    #1
    1. Advertising

  2. Anjanesh Lekshminarayanan wrote:
    > fp = urllib.urlopen(url)
    > data = fp.read()
    >
    > Retrieving XML data via an XML service API.
    > Very often network gets stuck in between. No errors / exceptions.
    >
    > CTRL+C
    >
    > File "get-xml.py", line 32, in <module>
    > fp = urllib.urlopen(url)
    > File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
    > return opener.open(url)
    > File "/usr/lib/python2.6/urllib.py", line 206, in open
    > return getattr(self, name)(url)
    > File "/usr/lib/python2.6/urllib.py", line 348, in open_http
    > errcode, errmsg, headers = h.getreply()
    > File "/usr/lib/python2.6/httplib.py", line 1048, in getreply
    > response = self._conn.getresponse()
    > File "/usr/lib/python2.6/httplib.py", line 974, in getresponse
    > response.begin()
    > File "/usr/lib/python2.6/httplib.py", line 391, in begin
    > version, status, reason = self._read_status()
    > File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
    > line = self.fp.readline()
    > File "/usr/lib/python2.6/socket.py", line 397, in readline
    > data = recv(1)
    > KeyboardInterrupt
    >
    > Is there I can do to try something else if its taking too long to
    > retrieve from the network ? Like kill previous attempt and retry ?
    >
    > Thanks
    > Anjanesh Lekshmnarayanan




    import socket
    from urllib2 import urlopen

    # A one-hundredths of a second (0.01) timeout before socket throws
    # an exception to demonstrate catching the timeout.
    # Obviously, this you will set this greater than 0.01 in real life.
    socket.setdefaulttimeout(0.01)

    # example xml feed
    xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

    try:
    data = urlopen(xml_source)
    except urllib2.URLError, e:
    print 'URLError = ' + str(e.reason)
    Gerald Walker, Nov 24, 2009
    #2
    1. Advertising


  3. >
    > import socket
    > from urllib2 import urlopen
    >
    > # A one-hundredths of a second (0.01) timeout before socket throws
    > # an exception to demonstrate catching the timeout.
    > # Obviously, this you will set this greater than 0.01 in real life.
    > socket.setdefaulttimeout(0.01)
    >
    > # example xml feed
    > xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"
    >
    > try:
    > data = urlopen(xml_source)
    > except urllib2.URLError, e:
    > print 'URLError = ' + str(e.reason)


    Also, if you are using multiple threads to retrieve the xml source(s)
    and any thread blocks due to network problems, the thread can go way by
    itself after the default timeout expires.
    Gerald Walker, Nov 24, 2009
    #3

  4. >
    > Also, if you are using multiple threads to retrieve the xml source(s)
    > and any thread blocks due to network problems, the thread can go way by
    > itself after the default timeout expires.
    >
    >


    Typo, edited for clarity:
    That is: "..the thread can go *away* by itself after the default timeout
    expires." You don't need to explicitly kill it.
    Gerald Walker, Nov 24, 2009
    #4

  5. >
    > import socket
    > from urllib2 import urlopen
    >
    > # A one-hundredths of a second (0.01) timeout before socket throws
    > # an exception to demonstrate catching the timeout.
    > # Obviously, this you will set this greater than 0.01 in real life.
    > socket.setdefaulttimeout(0.01)
    >
    > # example xml feed
    > xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"
    >
    > try:
    > data = urlopen(xml_source)
    > except urllib2.URLError, e:
    > print 'URLError = ' + str(e.reason)


    Oops, the above doesn't run. This version works:

    import socket
    import urllib2

    # A one-hundredths of a second (0.01) timeout before socket throws
    # an exception to demonstrate catching the timeout.
    # Obviously, this you will set this greater than 0.01 in real life.
    socket.setdefaulttimeout(0.01)

    # example xml feed
    xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

    try:
    data = urllib2.urlopen(xml_source)
    except urllib2.URLError, e:
    print 'URLError = ' + str(e.reason)
    Gerald Walker, Nov 24, 2009
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    489
  2. fredd00
    Replies:
    0
    Views:
    289
    fredd00
    May 31, 2007
  3. fredd00
    Replies:
    0
    Views:
    331
    fredd00
    May 31, 2007
  4. fredd00
    Replies:
    0
    Views:
    331
    fredd00
    Jun 1, 2007
  5. pintu
    Replies:
    0
    Views:
    199
    pintu
    Dec 14, 2006
Loading...

Share This Page