urllib2 FTP Weirdness

N

Nick Cash

Python 2.7.3 on linux

This has me fairly stumped. It looks like
urllib2.urlopen("ftp://some.ftp.site/path").read()
will either immediately return '' or hang indefinitely. But
response = urllib2.urlopen("ftp://some.ftp.site/path")
response.read()
works fine and returns what is expected. This is only an issue with urllib2, vanilla urllib doesn't do it.

The site I first noticed it on is private, but I can reproduce it with "ftp://ftp2.census.gov/".

I've tested the equivalent code on Python 3.2.3 and get the same results, except that one time I got a socket error (may have been a spurious network blip, though).


I'm at a loss as to how that could even work differently. My only guess is that by not having a reference to the addinfourl response object, somethingimportant is getting garbage collected or closed... that seems like a stretch, though. Is this a urllib2 bug, or am I crazy?

-Nick Cash
 
S

Steven D'Aprano

Nick said:
Python 2.7.3 on linux

This has me fairly stumped. It looks like
urllib2.urlopen("ftp://some.ftp.site/path").read()
will either immediately return '' or hang indefinitely. But
response = urllib2.urlopen("ftp://some.ftp.site/path")
response.read()
works fine and returns what is expected. This is only an issue with
urllib2, vanilla urllib doesn't do it.

The site I first noticed it on is private, but I can reproduce it with
"ftp://ftp2.census.gov/".

Then why not give that in your example, to make running your code
easier? :)

I cannot reproduce the problem:


py> import urllib2
py> x = urllib2.urlopen("ftp://ftp2.census.gov/").read()
py> len(x)
5550


Works fine for me using Python 2.7.2 on Linux. I cannot see how the two
snippets you give could possibly be different. If you are using a proxy,
what happens if you bypass it?

If you can reproduce this at will, with and without proxy, with multiple
sites, then I suppose it is conceivable that it could be some sort of bug.
But I wouldn't bet on it.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,479
Members
44,900
Latest member
Nell636132

Latest Threads

Top