urllib2 FTP Weirdness

Discussion in 'Python' started by Nick Cash, Jan 23, 2013.

  1. Nick Cash

    Nick Cash Guest

    Python 2.7.3 on linux

    This has me fairly stumped. It looks like
    urllib2.urlopen("ftp://some.ftp.site/path").read()
    will either immediately return '' or hang indefinitely. But
    response = urllib2.urlopen("ftp://some.ftp.site/path")
    response.read()
    works fine and returns what is expected. This is only an issue with urllib2, vanilla urllib doesn't do it.

    The site I first noticed it on is private, but I can reproduce it with "ftp://ftp2.census.gov/".

    I've tested the equivalent code on Python 3.2.3 and get the same results, except that one time I got a socket error (may have been a spurious network blip, though).


    I'm at a loss as to how that could even work differently. My only guess is that by not having a reference to the addinfourl response object, somethingimportant is getting garbage collected or closed... that seems like a stretch, though. Is this a urllib2 bug, or am I crazy?

    -Nick Cash
    Nick Cash, Jan 23, 2013
    #1
    1. Advertising

  2. Nick Cash wrote:

    > Python 2.7.3 on linux
    >
    > This has me fairly stumped. It looks like
    > urllib2.urlopen("ftp://some.ftp.site/path").read()
    > will either immediately return '' or hang indefinitely. But
    > response = urllib2.urlopen("ftp://some.ftp.site/path")
    > response.read()
    > works fine and returns what is expected. This is only an issue with
    > urllib2, vanilla urllib doesn't do it.
    >
    > The site I first noticed it on is private, but I can reproduce it with
    > "ftp://ftp2.census.gov/".


    Then why not give that in your example, to make running your code
    easier? :)

    I cannot reproduce the problem:


    py> import urllib2
    py> x = urllib2.urlopen("ftp://ftp2.census.gov/").read()
    py> len(x)
    5550


    Works fine for me using Python 2.7.2 on Linux. I cannot see how the two
    snippets you give could possibly be different. If you are using a proxy,
    what happens if you bypass it?

    If you can reproduce this at will, with and without proxy, with multiple
    sites, then I suppose it is conceivable that it could be some sort of bug.
    But I wouldn't bet on it.



    --
    Steven
    Steven D'Aprano, Jan 24, 2013
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. steve
    Replies:
    4
    Views:
    522
    Brian van den Broek
    Mar 13, 2005
  2. Josef Cihal
    Replies:
    0
    Views:
    738
    Josef Cihal
    Sep 5, 2005
  3. Devraj
    Replies:
    2
    Views:
    482
    Devraj
    Oct 29, 2007
  4. D. Buck
    Replies:
    2
    Views:
    477
    D. Buck
    Jun 29, 2004
  5. Chris Angelico

    Re: urllib2 FTP Weirdness

    Chris Angelico, Jan 23, 2013, in forum: Python
    Replies:
    5
    Views:
    147
    Cameron Simpson
    Feb 7, 2013
Loading...

Share This Page