httplib.InvalidURL: nonnumeric port: For characters in the proxy password in URL

[Phoe6]

Hi,
The following piece of code works properly when my proxy password
contains characters[a-zA-Z0-9] etc. But when my proxy password
contained something like '|\/|' , the httplib incorrectly indentified
it as separator. How do I resolve this issue.

<code>
# Proxy Address
PROXY_IP = "1.1.9.8:80"

# Trying with linear way

proxy_user = 'user_name'
proxy_password='|\/|something'

# Setup the Proxy with urllib2

proxy_url = 'http://' + proxy_user + ':' + proxy_password + '@' +
PROXY_IP
proxy_support = urllib2.ProxyHandler({"http"roxy_url})
opener = urllib2.build_opener(proxy_support,urllib2.HTTPHandler)
urllib2.install_opener(opener)

</code>

I get the Error:

Traceback (most recent call last):
File "C:\Python24\forge\ngwallp\ngwall.py", line 35, in ?
data = urllib2.urlopen(site)
File "C:\Python24\lib\urllib2.py", line 130, in urlopen
return _opener.open(url, data)
File "C:\Python24\lib\urllib2.py", line 358, in open
response = self._open(req, data)
File "C:\Python24\lib\urllib2.py", line 376, in _open
'_open', req)
File "C:\Python24\lib\urllib2.py", line 337, in _call_chain
result = func(*args)
File "C:\Python24\lib\urllib2.py", line 1021, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "C:\Python24\lib\urllib2.py", line 980, in do_open
h = http_class(host) # will parse hostort
File "C:\Python24\lib\httplib.py", line 586, in __init__
self._set_hostport(host, port)
File "C:\Python24\lib\httplib.py", line 598, in _set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '|\'


Thanks,
Senthil

 

[Fredrik Lundh]

Hi,
The following piece of code works properly when my proxy password
contains characters[a-zA-Z0-9] etc. But when my proxy password
contained something like '|\/|' , the httplib incorrectly indentified
it as separator.

if you want to prevent the URI parser from treating something as part of
the URI, you need to quote it:

pwd = urllib.quote(pwd, "")

</F>

 

[Phoe6]

Hi,
The following piece of code works properly when my proxy password
contains characters[a-zA-Z0-9] etc. But when my proxy password
contained something like '|\/|' , the httplib incorrectly indentified
it as separator.

if you want to prevent the URI parser from treating something as part of
the URI, you need to quote it:

pwd = urllib.quote(pwd, "")

Tried this and did not work out. It ended up confusing urllib and
urllib2 objects.
I dont want to import urllib, with the urllib2 itself wanted to work
around this piece of code.

<code>
proxy_user = r'senthil_or'
proxy_password='|\/|pr0c'
#proxy_password = r'|\/|pr0c'

# Setup the Proxy with urllib2

proxy_url = r'http://' + proxy_user + ':' + proxy_password + '@' +
PROXY_IP

proxy_support = urllib2.ProxyHandler({"http"roxy_url})
opener = urllib2.build_opener(proxy_support,urllib2.HTTPHandler)
urllib2.install_opener(opener)
</code>

Thanks,
Senthil

 

[Fredrik Lundh]

Tried this and did not work out. It ended up confusing urllib and
urllib2 objects.

oh, please. urllib.quote is a *function*; there's no way that calling that function
from code written for urllib2 will affect anything.

but you can access urllib.quote from the urllib2 namespace too. it's actually the
very same function:
<function quote at 0x00F430F0>

(urllib2 imports lots of stuff from urllib)

I dont want to import urllib, with the urllib2 itself wanted to work
around this piece of code.

not wanting to solve a problem is a pretty lousy problem-solving strategy, really.

</F>

 

[Phoe6]

Fredrik Lundh wrote:

Hi Fredrik,
I apologize if I offended you or have shown any
impatience. I shall try again:

oh, please. urllib.quote is a *function*; there's no way that calling that function
from code written for urllib2 will affect anything.

but you can access urllib.quote from the urllib2 namespace too. it's actually the
very same function:

<function quote at 0x00F430F0>

(urllib2 imports lots of stuff from urllib)

I dont know, it does not for me.
Traceback (most recent call last):

not wanting to solve a problem is a pretty lousy problem-solving strategy, really.

Sorry. I shall take your advice and try to get to bottom of this
issue.
- To the piece of code, I import urllib
- use urllib.quote() to covert the proxy url to a quoted one.
and try again, I get an error:

<error>
Traceback (most recent call last):
File "C:\Python24\forge\ngwallp\ngwall.py", line 38, in ?
data = urllib2.urlopen(site)
File "C:\Python24\lib\urllib2.py", line 130, in urlopen
http://lava.nationalgeographic.com/cgi-bin/pod/wallpaper.cgi?day=13&month=11&year=06
return _opener.open(url, data)
File "C:\Python24\lib\urllib2.py", line 358, in open
response = self._open(req, data)
File "C:\Python24\lib\urllib2.py", line 376, in _open
'_open', req)
File "C:\Python24\lib\urllib2.py", line 337, in _call_chain
result = func(*args)
File "C:\Python24\lib\urllib2.py", line 573, in <lambda>
lambda r, proxy=url, type=type, meth=self.proxy_open: \
File "C:\Python24\lib\urllib2.py", line 580, in proxy_open
if '@' in host:
TypeError: iterable argument required
</error>

This is where, I gave up and wrote the previous email as not to mess up
urllib with urllib2.
The piece of code, was working properly till I changed my proxy
password to something containng '|\/|'.
- Before shooting a mail to the grooups, i kindda encoded password to
valid url characters ( %7C%5C/%7C) and tried. But this did not work.
So, I dont think this is urllib.quote() issue. I am looking for a way
as how to use the ProxyHandler with authentication in different way
than I have used below.

proxy_url_add = r'http://' + proxy_user + ':' + proxy_password + '@' +
PROXY_IP
proxy_url = urllib.quote(proxy_url_add)

proxy_support = urllib2.ProxyHandler({"http"roxy_url})
opener = urllib2.build_opener(proxy_support,urllib2.HTTPHandler)
urllib2.install_opener(opener)

Thanks for your response.
Regards,
Senthil

 

[Fredrik Lundh]

- use urllib.quote() to covert the proxy url to a quoted one.

you should use quote to convert the *password* to quoted form, not use it on
the entire URL.

</F>

 

[Phoe6]

you should use quote to convert the *password* to quoted form, not use it on
the entire URL.

Am sorry Fred. The same problem:

File "C:\Python24\lib\httplib.py", line 598, in _set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '|\'

I dont think its an issue to be resolved with quote().

Thanks,
Senthil

 

[Fredrik Lundh]

Am sorry Fred. The same problem:

File "C:\Python24\lib\httplib.py", line 598, in _set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '|\'

I dont think its an issue to be resolved with quote().

can you post the code you're using to build the URI ?

</F>

 

[Phoe6]

can you post the code you're using to build the URI ?

Okay. This piece of code fetches a page from a particular site. As I am
behind a firewall, I have to communicate through a proxy.

# Set the Proxy Address
PROXY_IP = "10.1.9.4:80"

# Trying with linear way

proxy_user = 'user_name'
proxy_password_orig='|\/|pc'
proxy_password = urllib.quote(proxy_password_orig)

#proxy_password = r'|\/|pr0c'

# Setup the Proxy with urllib2

proxy_url = 'http://' + proxy_user + ':' + proxy_password + '@' +
PROXY_IP

proxy_support = urllib2.ProxyHandler({"http"roxy_url})
opener = urllib2.build_opener(proxy_support,urllib2.HTTPHandler)
urllib2.install_opener(opener)

# Read the ngsite
tdate = str(datetime.date.today())
sepdate = re.compile('\d\d(\d\d)-(\d+)-(\d+)')

year = sepdate.match(tdate).group(1)
month = sepdate.match(tdate).group(2)
day = sepdate.match(tdate).group(3)

site =
r"http://lava.nationalgeographic.com/cgi-bin/pod/wallpaper.cgi?day=" +
day + "&month=" + month + "&year=" + year
print site
data = urllib2.urlopen(site)

 

[Fredrik Lundh]

proxy_password = urllib.quote(proxy_password_orig)

make that:

proxy_password = urllib.quote(proxy_password_orig, "")

</F>

 

[Sion Arrowsmith]

<function quote at 0x00F430F0>

Traceback (most recent call last):
'2.4.1 (#2, May 5 2005, 11:32:06) \n[GCC 3.3.5 (Debian 1:3.3.5-12)]'

 

[Phoe6]

make that:

proxy_password = urllib.quote(proxy_password_orig, "")

Oh yeah. Wonderful!!
That worked. Thanks a lot Fredrik!. (<^> prostrations)

Am sorry for the trouble and my mistake in not getting it properly the
first time.

Regards,
Senthil