Transparent (redirecting) proxy with BaseHTTPServer

Discussion in 'Python' started by paul koelle, Jan 27, 2005.

  1. paul koelle

    paul koelle Guest

    Hi list,

    My ultimate goal is to have a small HTTP proxy which is able to show a
    message specific to clients name/ip/status then handle the original
    request normally either by redirecting the client, or acting as a proxy.

    I started with a modified[1] version of TinyHTTPProxy postet by Suzuki
    Hisao somewhere in 2003 to this list and tried to extend it to my needs.
    It works quite well if I configure my client to use it, but using
    iptables REDIRECT feature to point the clients transparently to the
    proxy caused some issues.

    Precisely, the "self.path" member variable of baseHTTPRequestHandler is
    missing the <command> and the host (i.e www.python.org) part of the
    request line for REDIRECTed connections:

    without iptables REDIRECT:
    self.path -> GET http://www.python.org/ftp/python/contrib/ HTTP/1.1

    with REDIRECT:
    self.path -> GET /ftp/python/contrib/ HTTP/1.1

    I asked about this on the squid mailing list and was told this is normal
    and I have to reconstuct the request line from the real destination IP,
    the URL-path and the Host header (if any). If the Host header is sent
    it's an (unsafe) nobrainer, but I cannot for the life of me figure out
    where to get the "real destination IP". Any ideas?

    thanks
    Paul

    [1] HTTP Debugging Proxy
    Modified by Xavier Defrang (http://defrang.com/)
    paul koelle, Jan 27, 2005
    #1
    1. Advertising

  2. paul koelle

    aurora Guest

    If you actually want the IP, resolve the host header would give you that.

    In the redirect case you should get a host header like

    Host: www.python.org

    From that you can reconstruct the original URL as
    http://www.python.org/ftp/python/contrib/. With that you can open it using
    urllib and proxy the data to the client.

    The second form of HTTP request without the host part is for compatability
    of pre-HTTP/1.1 standard. All modern web browser should send the Host
    header.


    > Hi list,
    >
    > My ultimate goal is to have a small HTTP proxy which is able to show a
    > message specific to clients name/ip/status then handle the original
    > request normally either by redirecting the client, or acting as a proxy.
    >
    > I started with a modified[1] version of TinyHTTPProxy postet by Suzuki
    > Hisao somewhere in 2003 to this list and tried to extend it to my needs.
    > It works quite well if I configure my client to use it, but using
    > iptables REDIRECT feature to point the clients transparently to the
    > proxy caused some issues.
    >
    > Precisely, the "self.path" member variable of baseHTTPRequestHandler is
    > missing the <command> and the host (i.e www.python.org) part of the
    > request line for REDIRECTed connections:
    >
    > without iptables REDIRECT:
    > self.path -> GET http://www.python.org/ftp/python/contrib/ HTTP/1.1
    >
    > with REDIRECT:
    > self.path -> GET /ftp/python/contrib/ HTTP/1.1
    >
    > I asked about this on the squid mailing list and was told this is normal
    > and I have to reconstuct the request line from the real destination IP,
    > the URL-path and the Host header (if any). If the Host header is sent
    > it's an (unsafe) nobrainer, but I cannot for the life of me figure out
    > where to get the "real destination IP". Any ideas?
    >
    > thanks
    > Paul
    >
    > [1] HTTP Debugging Proxy
    > Modified by Xavier Defrang (http://defrang.com/)
    aurora, Jan 27, 2005
    #2
    1. Advertising

  3. paul koelle

    paul koelle Guest

    Thanks, aurora ;),

    aurora wrote:
    > If you actually want the IP, resolve the host header would give you that.

    I' m only interested in the hostname.

    >
    > The second form of HTTP request without the host part is for
    > compatability of pre-HTTP/1.1 standard. All modern web browser should
    > send the Host header.

    How safe is the assumtion that the Host header will be there? Is it part
    of the HTTP/1.1 spec? And does it mean all "pre 1.1" clients will fail?
    Hmm, maybe I should look on the wire whats really happening...

    thanks again
    Paul
    paul koelle, Jan 27, 2005
    #3
  4. paul koelle

    aurora Guest

    It should be very safe to count on the host header. Maybe some really
    really old browser would not support that. But they probably won't work in
    today's WWW anyway. Majority of today's web site is likely to be virtually
    hosted. One Apache maybe hosting for 50 web addresses. If a client strip
    the host name and not sending the host header either the web server
    wouldn't what address it is really looking for. If you caught some request
    that doesn't have host header it is a good idea to redirect them to a
    browser upgrade page.

    >
    > Thanks, aurora ;),
    >
    > aurora wrote:
    >> If you actually want the IP, resolve the host header would give you
    >> that.

    > I' m only interested in the hostname.
    >
    >> The second form of HTTP request without the host part is for
    >> compatability of pre-HTTP/1.1 standard. All modern web browser should
    >> send the Host header.

    > How safe is the assumtion that the Host header will be there? Is it part
    > of the HTTP/1.1 spec? And does it mean all "pre 1.1" clients will fail?
    > Hmm, maybe I should look on the wire whats really happening...
    >
    > thanks again
    > Paul
    aurora, Jan 28, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael Spahn
    Replies:
    0
    Views:
    1,126
    Michael Spahn
    Dec 7, 2004
  2. Molly
    Replies:
    0
    Views:
    521
    Molly
    Aug 25, 2003
  3. Joshua W. Biagio
    Replies:
    0
    Views:
    595
    Joshua W. Biagio
    Sep 30, 2003
  4. Michael Vondung

    Code for a transparent proxy?

    Michael Vondung, May 9, 2004, in forum: Ruby
    Replies:
    1
    Views:
    141
    Kaspar Schiess
    May 9, 2004
  5. Luca Bongiorni

    Transparent Proxy and Redirecting Sockets

    Luca Bongiorni, Feb 21, 2013, in forum: Python
    Replies:
    2
    Views:
    107
    Luca Bongiorni
    Feb 21, 2013
Loading...

Share This Page