ClientCookie problem - Difference between 'post' on the local networkand the internet

Discussion in 'Python' started by Max M, Oct 22, 2004.

  1. Max M

    Max M Guest

    I am using ClientCookie for login on to servers and browsing them as
    authenticated users.

    I kept getting "HTTP Error 400: Bad Request" errors when submitting my
    forms. So I boiled it down to a simple example.

    When I try to use ClientCookie.urlopen() on my private network with ip
    numbers like "http://localhost:8081/test_site/logged_in", it works fine.

    If I try to call the same site through the Internet, with a url like:
    http://www.mxm.dk/test_site/logged_in I get the 400 message. But only
    when using "post". 'get' works fine :-s

    On the Internet I go out through a gateway, so my external IP is
    different than my internal.

    I assume that it causes an url mismatch somewhere. With my local ip
    being set in some header by urlopener, and the server expecting the
    remote ip.

    But I have no idea as to how I can avoid it? Any takers?


    #########################################

    import ClientCookie, urllib

    req = ClientCookie.Request
    urlopen = ClientCookie.urlopen

    def login(came_from, login_form):
    data = {
    'came_from' : came_from,
    '__ac_name' : name,
    '__ac_password' : password,
    '__ac_persistent' : '1',
    'submit' : 'Log ind',
    }
    encoded_data = urllib.urlencode(data)

    request = req(login_form, encoded_data)
    response = urlopen(request)
    content = response.read()
    return content

    # This works
    came_from = 'http://localhost:8081/test_site/request_tester'
    login_form = "http://localhost:8081/test_site/logged_in"
    print login(came_from, login_form)

    # This results in "HTTP Error 400: Bad Request"
    came_from = 'http://www.mxm.dk/test_site/request_tester'
    login_form = "http://www.mxm.dk/test_site/logged_in"
    print login(came_from, login_form)



    ############################################
    # traceback from the last call to login()

    Traceback (most recent call last):
    File "E:\zopes\test_site\PRODUCTS\MXMPRO~1\z.py", line 35, in ?
    print login(came_from, login_form)
    File "E:\zopes\test_site\PRODUCTS\MXMPRO~1\z.py", line 22, in login
    response = urlopen(request)
    File
    "G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_urllib2_support.py",
    line 715, in urlopen
    return _opener.open(url, data)
    File
    "G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_Opener.py",
    line 125, in open
    response = meth(req, response)
    File
    "G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_urllib2_support.py",
    line 387, in http_response
    response = self.parent.error(
    File
    "G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_Opener.py",
    line 189, in error
    return apply(self._call_chain, args)
    File "C:\pytyons\Python23\lib\urllib2.py", line 306, in _call_chain
    result = func(*args)
    File "C:\pytyons\Python23\lib\urllib2.py", line 412, in
    http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
    urllib2.HTTPError: HTTP Error 400: Bad Request
    --

    hilsen/regards Max M, Denmark

    http://www.mxm.dk/
    IT's Mad Science
     
    Max M, Oct 22, 2004
    #1
    1. Advertising

  2. Max M

    Fuzzyman Guest

    Re: ClientCookie problem - Difference between 'post' on the local network and the internet

    On Fri, 22 Oct 2004 13:39:05 +0200, Max M <> wrote:

    >I am using ClientCookie for login on to servers and browsing them as
    >authenticated users.
    >
    >I kept getting "HTTP Error 400: Bad Request" errors when submitting my
    >forms. So I boiled it down to a simple example.
    >
    >When I try to use ClientCookie.urlopen() on my private network with ip
    >numbers like "http://localhost:8081/test_site/logged_in", it works fine.
    >
    >If I try to call the same site through the Internet, with a url like:
    >http://www.mxm.dk/test_site/logged_in I get the 400 message. But only
    >when using "post". 'get' works fine :-s
    >


    Are you using Version 1.0.0a ?
    There is a bug (well actually a bug in urllib2 - but it amoutns to the
    same thing) that is fixed in CVS. Either use CVS or go back to 0.4
    whatever which is the latest stable version.

    Regards,

    Fuzzy

    >On the Internet I go out through a gateway, so my external IP is
    >different than my internal.
    >
    >I assume that it causes an url mismatch somewhere. With my local ip
    >being set in some header by urlopener, and the server expecting the
    >remote ip.
    >
    >But I have no idea as to how I can avoid it? Any takers?
    >
    >
    >#########################################
    >
    >import ClientCookie, urllib
    >
    >req = ClientCookie.Request
    >urlopen = ClientCookie.urlopen
    >
    >def login(came_from, login_form):
    > data = {
    > 'came_from' : came_from,
    > '__ac_name' : name,
    > '__ac_password' : password,
    > '__ac_persistent' : '1',
    > 'submit' : 'Log ind',
    > }
    > encoded_data = urllib.urlencode(data)
    >
    > request = req(login_form, encoded_data)
    > response = urlopen(request)
    > content = response.read()
    > return content
    >
    ># This works
    >came_from = 'http://localhost:8081/test_site/request_tester'
    >login_form = "http://localhost:8081/test_site/logged_in"
    >print login(came_from, login_form)
    >
    ># This results in "HTTP Error 400: Bad Request"
    >came_from = 'http://www.mxm.dk/test_site/request_tester'
    >login_form = "http://www.mxm.dk/test_site/logged_in"
    >print login(came_from, login_form)
    >
    >
    >
    >############################################
    ># traceback from the last call to login()
    >
    >Traceback (most recent call last):
    > File "E:\zopes\test_site\PRODUCTS\MXMPRO~1\z.py", line 35, in ?
    > print login(came_from, login_form)
    > File "E:\zopes\test_site\PRODUCTS\MXMPRO~1\z.py", line 22, in login
    > response = urlopen(request)
    > File
    >"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_urllib2_support.py",
    >line 715, in urlopen
    > return _opener.open(url, data)
    > File
    >"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_Opener.py",
    >line 125, in open
    > response = meth(req, response)
    > File
    >"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_urllib2_support.py",
    >line 387, in http_response
    > response = self.parent.error(
    > File
    >"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_Opener.py",
    >line 189, in error
    > return apply(self._call_chain, args)
    > File "C:\pytyons\Python23\lib\urllib2.py", line 306, in _call_chain
    > result = func(*args)
    > File "C:\pytyons\Python23\lib\urllib2.py", line 412, in
    >http_error_default
    > raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
    >urllib2.HTTPError: HTTP Error 400: Bad Request
     
    Fuzzyman, Oct 22, 2004
    #2
    1. Advertising

  3. Max M

    Max M Guest

    Re: ClientCookie problem - Difference between 'post' on the localnetwork and the internet

    Fuzzyman wrote:
    > On Fri, 22 Oct 2004 13:39:05 +0200, Max M <> wrote:


    > Are you using Version 1.0.0a ?


    Yes

    > There is a bug (well actually a bug in urllib2 - but it amoutns to the
    > same thing) that is fixed in CVS. Either use CVS or go back to 0.4
    > whatever which is the latest stable version.


    argh! Well that was a day of my life wasted :-( Well at least *this* day
    was saved. Thank you.

    It's one of the damnest problems with programming.

    Either you write a module like ClientCookie yourself. Then you can find
    bugs rather easily. But it takes so long to work out the special cases.

    Or you can use an existing library. But if that has a bug, or is poorly
    documented, you need to understand the library pretty well to bug fix
    it. Leaving you esssentially in the hands of the author.

    I find that moments like these are among the most frustrating in
    programming.

    Unfurtunately there is no easy way out, but some of the best stuff I
    have seen to solve these kind of documentation problems, is the doctest
    based code in Zope 3.

    This approach to testing makes their code *really* easy to understand.



    --

    hilsen/regards Max M, Denmark

    http://www.mxm.dk/
    IT's Mad Science
     
    Max M, Oct 23, 2004
    #3
  4. Max M

    Max M Guest

    Re: ClientCookie problem - Difference between 'post' on the localnetwork and the internet

    Fuzzyman wrote:

    > Are you using Version 1.0.0a ?
    > There is a bug (well actually a bug in urllib2 - but it amoutns to the
    > same thing) that is fixed in CVS. Either use CVS or go back to 0.4
    > whatever which is the latest stable version.



    I cannot seem to find a cvs repository for ClientCookie or wwwsearch
    anywhere. Or do you mean that I should get urllib2 from cvs?


    --

    hilsen/regards Max M, Denmark

    http://www.mxm.dk/
    IT's Mad Science
     
    Max M, Oct 23, 2004
    #4
  5. Re: ClientCookie problem - Difference between 'post' on the local network and the internet

    Max M <> wrote in message news:<>...
    > Fuzzyman wrote:
    >
    > > Are you using Version 1.0.0a ?
    > > There is a bug (well actually a bug in urllib2 - but it amoutns to the
    > > same thing) that is fixed in CVS. Either use CVS or go back to 0.4
    > > whatever which is the latest stable version.

    >
    >
    > I cannot seem to find a cvs repository for ClientCookie or wwwsearch
    > anywhere. Or do you mean that I should get urllib2 from cvs?


    You're right.. I can't find it either. At least it's not on
    sourceforge anyway..
    John J Lee is the author of the module and he usually hangs around on
    the wwwsearch mailing list (and sometimes here). When I reported the
    bug to him a while ago he said he has fixed it in CVS. I just went
    back to using 0.9 which I'm not sure is now easily available....

    If you want I can mail that one to you.
    Mail me on fuzzyman AT voidspace DOT org DOT uk and I'll see if I can
    dig it out.

    Regards,

    Fuzzy

    http://www.voidspace.org.uk/atlantibots/pythonutils.html
     
    Michael Foord, Oct 23, 2004
    #5
  6. Max M

    John J Lee Guest

    Re: ClientCookie problem - Difference between 'post' on the localnetwork and the internet

    On Fri, 22 Oct 2004, Max M wrote:

    > I kept getting "HTTP Error 400: Bad Request" errors when submitting my
    > forms. So I boiled it down to a simple example.

    [...]
    > #########################################
    >
    > import ClientCookie, urllib

    [...]
    > urllib2.HTTPError: HTTP Error 400: Bad Request


    I'm not certain this is the problem fuzzyman refers to.

    Thanks for sample, but I get NameError instead, so your script needs
    fixing before I can easily investigate:

    Traceback (most recent call last):
    File "/home/john/bin/tst.py", line 59, in ?
    print login(came_from, login_form)
    File "/home/john/bin/tst.py", line 42, in login
    data = {
    NameError: global name 'name' is not defined


    John
     
    John J Lee, Oct 26, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. jakk
    Replies:
    4
    Views:
    12,614
  2. Mark Carter

    ClientCookie bug

    Mark Carter, Aug 14, 2003, in forum: Python
    Replies:
    28
    Views:
    836
    John J. Lee
    Aug 27, 2003
  3. Grant Edwards

    Why is ClientCookie/urllib2 using https?

    Grant Edwards, Sep 24, 2003, in forum: Python
    Replies:
    0
    Views:
    354
    Grant Edwards
    Sep 24, 2003
  4. PerlFAQ Server
    Replies:
    0
    Views:
    393
    PerlFAQ Server
    Jan 6, 2011
  5. PerlFAQ Server
    Replies:
    0
    Views:
    319
    PerlFAQ Server
    Apr 15, 2011
Loading...

Share This Page