SimpleHTTPRequestHandler used with HTTP/1.1 hangs after the secondresource on a page.

Discussion in 'Python' started by Piotr Dobrogost, Apr 12, 2013.

  1. Hi!

    I'd like to bring your attention to the question titled "Use HTTP/1.1 with SimpleHTTPRequestHandler" at http://stackoverflow.com/q/15839718/95735 which reads; "When I use HTTP/1.1 with SimpleHTTPRequestHandler, loading a pagethat pulls in other resources will hang after the second resource." and there's a tiny test showing the problem.
    It's hard to believe that SimpleHTTPServer doesn't handle such a simple task, does it?
    If I checked correctly code is stuck in handle_one_request() method at line370 of server.py (http://hg.python.org/cpython/file/d9893d13c628/Lib/http/server.py#l370) but I can't see what's wrong.
    Any ideas?


    Best regards,
    Piotr Dobrogost
    Piotr Dobrogost, Apr 12, 2013
    #1
    1. Advertising

  2. On 4/12/2013 4:46 PM, Piotr Dobrogost wrote:
    > Hi!
    >
    > I'd like to bring your attention to the question titled "Use HTTP/1.1
    > with SimpleHTTPRequestHandler" at
    > http://stackoverflow.com/q/15839718/95735 which reads; "When I use


    I find the doc slightly confusing. The SO code uses BaseHTTPServer. The
    doc says "Usually, this module isn’t used directly," On the other hand,
    SimpleHTTPServer only defines a request handler and not a server itself.

    > HTTP/1.1 with SimpleHTTPRequestHandler, loading a page that pulls in
    > other resources will hang after the second resource." and there's a


    What does 'hang' mean? Does the page get displayed? If so,
    server.serve_forever is supposes to 'hang'.

    > tiny test showing the problem. It's hard to believe that
    > SimpleHTTPServer doesn't handle such a simple task, does it? If I
    > checked correctly code is stuck in handle_one_request() method at
    > line 370 of server.py
    > (http://hg.python.org/cpython/file/d9893d13c628/Lib/http/server.py#l370)
    > but I can't see what's wrong. Any ideas?


    The SO post says

    class MyRequestHandler(SimpleHTTPRequestHandler):
    #protocol_version = "HTTP/1.0" # works
    protocol_version = "HTTP/1.1" # hangs

    so this should be some sort of clue. The code says

    569 # The version of the HTTP protocol we support.
    570 # Set this to HTTP/1.1 to enable automatic keepalive
    571 protocol_version = "HTTP/1.0"

    The only substantive code I found using protocol_version (in the CPython
    default branch you linked to) is

    300 if version_number >= (1, 1) and self.protocol_version >= "HTTP/1.1":
    301 self.close_connection = 0

    331 elif (conntype.lower() == 'keep-alive' and
    332 self.protocol_version >= "HTTP/1.1"):
    333 self.close_connection = 0
    334 # Examine the headers and look for an Expect directive
    335 expect = self.headers.get('Expect', "")
    336 if (expect.lower() == "100-continue" and
    337 self.protocol_version >= "HTTP/1.1" and
    338 self.request_version >= "HTTP/1.1"):
    339 if not self.handle_expect_100():
    340 return False

    I would re-test with the recently released 3.3.1 (and/or 2.7.4) to make
    sure nothing has changed
    Terry Jan Reedy, Apr 12, 2013
    #2
    1. Advertising

  3. Re: SimpleHTTPRequestHandler used with HTTP/1.1 hangs after thesecond resource on a page.

    On Saturday, April 13, 2013 12:21:33 AM UTC+2, Terry Jan Reedy wrote:
    > I find the doc slightly confusing. The SO code uses BaseHTTPServer. The
    > doc says "Usually, this module isn’t used directly," On the other hand,
    > SimpleHTTPServer only defines a request handler and not a server itself.


    Thanks for taking time to reply.

    Well, I've seen presentations where BaseHTTPServer is a recommended way of running simple, ad-hoc web server. In addition the documentation of http.server module in Python 3.3 (which contains implementation of the same HTTPServer class as found in BaseHTTPServer module in Python 2.x) does not contain the statement you cited which I take to mean it can safely be used directly.

    As Python 3.3.1 exhibits the same behavior let's focus on the following Python 3 version of the code from the original question:

    from http.server import SimpleHTTPRequestHandler
    from http.server import HTTPServer

    class MyRequestHandler(SimpleHTTPRequestHandler):
    #protocol_version = "HTTP/1.0" # works
    protocol_version = "HTTP/1.1" # hangs

    server = HTTPServer(("localhost", 7080), MyRequestHandler)
    server.serve_forever()

    > What does 'hang' mean? Does the page get displayed? If so,
    > server.serve_forever is supposes to 'hang'.


    I increased the number of resources to 7 to be able to detect possible patterns. The page and some of the resources are being displayed almost instantly. However browser waits for the requests to get remaining resources to finish and so those resources naturally are not being displayed. When browserwaits the code is stuck at the line I showed in my original post. What's interesting is that after some time the request to get the second resource (in case there are only 2 on the page) finishes. However the time spent waiting for the second resource is ridiculously long compared to time spend waiting for the first resource. For example in one of my tests Firebug showed waiting time for the first resource to be 4ms whereas the waiting time for the second resource to be 1m 55s.

    > The SO post says
    >
    > class MyRequestHandler(SimpleHTTPRequestHandler):
    > #protocol_version = "HTTP/1.0" # works
    > protocol_version = "HTTP/1.1" # hangs
    >
    > so this should be some sort of clue. The code says
    >
    > 569 # The version of the HTTP protocol we support.
    > 570 # Set this to HTTP/1.1 to enable automatic keepalive
    > 571 protocol_version = "HTTP/1.0"


    Sure, in version 1.1 of http the connection is not closed by default unless'Connection: Close' header was sent in request. Firefox sends all requestswith 'Connection: keep-alive' header.

    I don't really know how to debug this...
    Piotr Dobrogost, Apr 13, 2013
    #3
  4. Re: SimpleHTTPRequestHandler used with HTTP/1.1 hangs after thesecond resource on a page.

    On Saturday, April 13, 2013 12:21:33 AM UTC+2, Terry Jan Reedy wrote:
    > I find the doc slightly confusing. The SO code uses BaseHTTPServer. The
    > doc says "Usually, this module isn’t used directly," On the other hand,
    > SimpleHTTPServer only defines a request handler and not a server itself.


    Thanks for taking time to reply.

    Well, I've seen presentations where BaseHTTPServer is a recommended way of running simple, ad-hoc web server. In addition the documentation of http.server module in Python 3.3 (which contains implementation of the same HTTPServer class as found in BaseHTTPServer module in Python 2.x) does not contain the statement you cited which I take to mean it can safely be used directly.

    As Python 3.3.1 exhibits the same behavior let's focus on the following Python 3 version of the code from the original question:

    from http.server import SimpleHTTPRequestHandler
    from http.server import HTTPServer

    class MyRequestHandler(SimpleHTTPRequestHandler):
    #protocol_version = "HTTP/1.0" # works
    protocol_version = "HTTP/1.1" # hangs

    server = HTTPServer(("localhost", 7080), MyRequestHandler)
    server.serve_forever()

    > What does 'hang' mean? Does the page get displayed? If so,
    > server.serve_forever is supposes to 'hang'.


    I increased the number of resources to 7 to be able to detect possible patterns. The page and some of the resources are being displayed almost instantly. However browser waits for the requests to get remaining resources to finish and so those resources naturally are not being displayed. When browserwaits the code is stuck at the line I showed in my original post. What's interesting is that after some time the request to get the second resource (in case there are only 2 on the page) finishes. However the time spent waiting for the second resource is ridiculously long compared to time spend waiting for the first resource. For example in one of my tests Firebug showed waiting time for the first resource to be 4ms whereas the waiting time for the second resource to be 1m 55s.

    > The SO post says
    >
    > class MyRequestHandler(SimpleHTTPRequestHandler):
    > #protocol_version = "HTTP/1.0" # works
    > protocol_version = "HTTP/1.1" # hangs
    >
    > so this should be some sort of clue. The code says
    >
    > 569 # The version of the HTTP protocol we support.
    > 570 # Set this to HTTP/1.1 to enable automatic keepalive
    > 571 protocol_version = "HTTP/1.0"


    Sure, in version 1.1 of http the connection is not closed by default unless'Connection: Close' header was sent in request. Firefox sends all requestswith 'Connection: keep-alive' header.

    I don't really know how to debug this...
    Piotr Dobrogost, Apr 13, 2013
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. psb

    web page hangs after login

    psb, Apr 6, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    382
  2. Andy Leszczynski
    Replies:
    4
    Views:
    572
    Andy Leszczynski
    Mar 14, 2005
  3. Replies:
    3
    Views:
    792
    Guillermo
    Apr 26, 2007
  4. Tim Shadel

    Gem hangs => TCPSocket.write hangs

    Tim Shadel, Jul 23, 2005, in forum: Ruby
    Replies:
    1
    Views:
    382
    Ville Mattila
    Jul 24, 2005
  5. zyng
    Replies:
    5
    Views:
    678
    Arne Vajhøj
    Jun 21, 2012
Loading...

Share This Page