D
Dan Lenski
Hi all,
I've recently written an HTTP proxy server for the Motorola E815 cell
phone, based on Suzuki Hisao's "Tiny HTTP Proxy" (http://
www.okisoft.co.jp/esc/python/proxy/). This cell phone allows free
Internet access if you change the default proxy server, but it has a
severely buggy HTTP client implementation which will freeze if there's
no Content-Length header. Basically, it's impossible to read normal
web pages without some massaging of the headers.
So my proxy adds the correct Content-Length header and tweaks the
response in a couple of other ways. It works great now, you can get
the code from: http://tonquil.homeip.net/~dlenski/cellphoneproxy/
I've MIT licensed it. It works fine as a proxy for a "normal" web
browser too!
One of the things I had trouble with was getting decent speed out of
this proxy. The original "Tiny HTTP Proxy" didn't support persistent
connections... which seriously slows down web browsing with a proxy,
since it essentially DOUBLES the number of TCP connections that need
to be made for each page access. So my proxy *does* support
persistent connections, using the standard Connection or non-standard
Proxy-Connection headers to determine whether to keep the connection
alive or not.
It's still not perfect though... apparently sometimes a web browser
will SAY that it wants to keep a connection alive, and then just drop
it. The proxy normally runs in a multi-threaded mode, and it counts
the number of times that each connection is reused, printing this when
the RequestHandler.finish() method is eventually called. If I force
the proxy to run in a single thread, and use it as a proxy for
Firefox, it will freeze up after a few page requests... the proxy will
wait around for a new request on the same socket that never comes.
So, my question is... is there any way to improve on this? Is there a
way to make BaseHTTPRequestHandler detect when its connection has been
closed by the peer without waiting for a long timeout? Is there a way
to get the browser to always reuse a connection when it says it's
going to? Any advice on improving the code is appreciated!
Thanks,
Dan Lenski
I've recently written an HTTP proxy server for the Motorola E815 cell
phone, based on Suzuki Hisao's "Tiny HTTP Proxy" (http://
www.okisoft.co.jp/esc/python/proxy/). This cell phone allows free
Internet access if you change the default proxy server, but it has a
severely buggy HTTP client implementation which will freeze if there's
no Content-Length header. Basically, it's impossible to read normal
web pages without some massaging of the headers.
So my proxy adds the correct Content-Length header and tweaks the
response in a couple of other ways. It works great now, you can get
the code from: http://tonquil.homeip.net/~dlenski/cellphoneproxy/
I've MIT licensed it. It works fine as a proxy for a "normal" web
browser too!
One of the things I had trouble with was getting decent speed out of
this proxy. The original "Tiny HTTP Proxy" didn't support persistent
connections... which seriously slows down web browsing with a proxy,
since it essentially DOUBLES the number of TCP connections that need
to be made for each page access. So my proxy *does* support
persistent connections, using the standard Connection or non-standard
Proxy-Connection headers to determine whether to keep the connection
alive or not.
It's still not perfect though... apparently sometimes a web browser
will SAY that it wants to keep a connection alive, and then just drop
it. The proxy normally runs in a multi-threaded mode, and it counts
the number of times that each connection is reused, printing this when
the RequestHandler.finish() method is eventually called. If I force
the proxy to run in a single thread, and use it as a proxy for
Firefox, it will freeze up after a few page requests... the proxy will
wait around for a new request on the same socket that never comes.
So, my question is... is there any way to improve on this? Is there a
way to make BaseHTTPRequestHandler detect when its connection has been
closed by the peer without waiting for a long timeout? Is there a way
to get the browser to always reuse a connection when it says it's
going to? Any advice on improving the code is appreciated!
Thanks,
Dan Lenski