Looking for an http proxy server written in Python

Discussion in 'Python' started by Chris Gray, Nov 18, 2003.

  1. Chris Gray

    Chris Gray Guest

    Is there such a beast? In particular, I'm looking for a
    production-quality proxy server fully compliant with HTTP/1.1 written in
    Python.

    If there isn't, is there anything that comes close?

    Thanks for any help you can throw my way,
    Chris Gray

    "Okey dokey.. free my mind. Right, no problem, free my mind, free my mind,
    no problem, right... " -The Matrix
     
    Chris Gray, Nov 18, 2003
    #1
    1. Advertising

  2. Chris Gray

    Alan Kennedy Guest

    [Chris Gray]
    > I'm looking for a
    > production-quality proxy server fully compliant with HTTP/1.1 written in
    > Python.
    >
    > If there isn't, is there anything that comes close?


    That depends on what you want the proxy to do.

    Check the following page for a list of python http proxies.

    http://xhaus.com/alan/python/proxies.html

    If there's nothing there that catches your eye, then give a clearer
    indication of what you need the proxy to do.

    regards,

    --
    alan kennedy
    -----------------------------------------------------
    check http headers here: http://xhaus.com/headers
    email alan: http://xhaus.com/mailto/alan
     
    Alan Kennedy, Nov 18, 2003
    #2
    1. Advertising

  3. Chris Gray

    Dave Brueck Guest

    Dave Brueck, Nov 18, 2003
    #3
  4. Chris Gray

    Chris Gray Guest

    Thanks, Alan,

    Your resource page is very helpful and your "Why" section explains exactly
    why I'm interested in this. At first blush it looks like httpMonitor is
    most in line with what I'm planning, namely, the ability to do arbitrary
    processing on requests and responses.

    The problem I've run across so far is transparency and performance. I've
    written my own proxy server and experimented with WebDebug and found that
    some things slow down considerably and other things don't seem to pass
    through at all. I want, as much as possible, for the proxy to pass
    requests and responses through in a way that makes no difference to the
    client, but gives me a hook to hang selective logging, storage, and other
    processing on.

    That's why I specified production-quality and HTTP/1.1 compliance. If I
    could just find a proxy server that forwarded requests and responses
    seamlessly, I'd be happy.

    Thanks again,
    Chris

    "Okey dokey.. free my mind. Right, no problem, free my mind, free my mind,
    no problem, right... " -The Matrix

    On Tue, 18 Nov 2003, Alan Kennedy wrote:

    > [Chris Gray]
    > > I'm looking for a
    > > production-quality proxy server fully compliant with HTTP/1.1 written in
    > > Python.
    > >
    > > If there isn't, is there anything that comes close?

    >
    > That depends on what you want the proxy to do.
    >
    > Check the following page for a list of python http proxies.
    >
    > http://xhaus.com/alan/python/proxies.html
    >
    > If there's nothing there that catches your eye, then give a clearer
    > indication of what you need the proxy to do.
    >
    > regards,
    >
    > --
    > alan kennedy
    > -----------------------------------------------------
    > check http headers here: http://xhaus.com/headers
    > email alan: http://xhaus.com/mailto/alan
    >
     
    Chris Gray, Nov 18, 2003
    #4
  5. Chris Gray

    Paul Rubin Guest

    Chris Gray <> writes:
    > Is there such a beast? In particular, I'm looking for a
    > production-quality proxy server fully compliant with HTTP/1.1 written in
    > Python.


    I sometimes use Amit Patel's proxy3 for web surfing, but I don't know
    if I'd call it production quality because it's so slow. I don't know
    if anything can be done about that, because Python itself is pretty
    slow. You may need to bite the bullet and use Squid or the Apache
    proxy module or something like that. I've had reasonable success
    hacking various html rewriting features into the Apache module when
    such things were needed.
     
    Paul Rubin, Nov 18, 2003
    #5
  6. Chris Gray

    John Abel Guest

    You might want to have a look at Willow from www.digitallumber.com.

    Regards

    John

    Chris Gray wrote:

    >Thanks, Alan,
    >
    >Your resource page is very helpful and your "Why" section explains exactly
    >why I'm interested in this. At first blush it looks like httpMonitor is
    >most in line with what I'm planning, namely, the ability to do arbitrary
    >processing on requests and responses.
    >
    >The problem I've run across so far is transparency and performance. I've
    >written my own proxy server and experimented with WebDebug and found that
    >some things slow down considerably and other things don't seem to pass
    >through at all. I want, as much as possible, for the proxy to pass
    >requests and responses through in a way that makes no difference to the
    >client, but gives me a hook to hang selective logging, storage, and other
    >processing on.
    >
    >That's why I specified production-quality and HTTP/1.1 compliance. If I
    >could just find a proxy server that forwarded requests and responses
    >seamlessly, I'd be happy.
    >
    >Thanks again,
    >Chris
    >
    >"Okey dokey.. free my mind. Right, no problem, free my mind, free my mind,
    >no problem, right... " -The Matrix
    >
    >On Tue, 18 Nov 2003, Alan Kennedy wrote:
    >
    >
    >
    >>[Chris Gray]
    >>
    >>
    >>>I'm looking for a
    >>>production-quality proxy server fully compliant with HTTP/1.1 written in
    >>>Python.
    >>>
    >>>If there isn't, is there anything that comes close?
    >>>
    >>>

    >>That depends on what you want the proxy to do.
    >>
    >>Check the following page for a list of python http proxies.
    >>
    >>http://xhaus.com/alan/python/proxies.html
    >>
    >>If there's nothing there that catches your eye, then give a clearer
    >>indication of what you need the proxy to do.
    >>
    >>regards,
    >>
    >>--
    >>alan kennedy
    >>-----------------------------------------------------
    >>check http headers here: http://xhaus.com/headers
    >>email alan: http://xhaus.com/mailto/alan
    >>
    >>
    >>
     
    John Abel, Nov 19, 2003
    #6
  7. Chris Gray

    Chris Gray Guest

    On Tue, 18 Nov 2003, it was written:
    >
    > I sometimes use Amit Patel's proxy3 for web surfing, but I don't know
    > if I'd call it production quality because it's so slow. I don't know
    > if anything can be done about that, because Python itself is pretty
    > slow. You may need to bite the bullet and use Squid or the Apache
    > proxy module or something like that. I've had reasonable success
    > hacking various html rewriting features into the Apache module when
    > such things were needed.


    I may have a look at Squid and mod_proxy (and Jigsaw to boot) at some
    point. I'm not as worried about speed as I am about choking on requests
    that can't be handled. I'm not trying to do the processing in real time;
    that is, the results of the processing don't have to be available to the
    client in real time.

    Thanks for the feedback,
    Chris
     
    Chris Gray, Nov 19, 2003
    #7
  8. Chris Gray

    Chris Gray Guest

    On Wed, 19 Nov 2003, John Abel wrote:

    > You might want to have a look at Willow from www.digitallumber.com.
    >
    > Regards


    Thanks, I hadn't run across this before. Looks interesting.

    Cheers,
    Chris
     
    Chris Gray, Nov 19, 2003
    #8
  9. Chris Gray

    Alan Kennedy Guest

    Chris Gray wrote:
    > Your resource page is very helpful and your "Why" section explains
    > exactly why I'm interested in this. At first blush it looks like
    > httpMonitor is most in line with what I'm planning, namely, the
    > ability to do arbitrary processing on requests and responses.


    I've tried to make the page a little more useful now, by adding
    freshness dates, and a better overview of the features offered by the
    various proxies.

    http://www.xhaus.com/alan/python/proxies.html

    > That's why I specified production-quality and HTTP/1.1 compliance. If I
    > could just find a proxy server that forwarded requests and responses
    > seamlessly, I'd be happy.


    FWIW, I mostly run a very simple proxy, derived from Suzuki Hisao's
    Tiny proxy, to do protocol and resource logging, which I need when I'm
    doing functional and load testing. I store the output in python
    syntax, to be fed straight back into a python program. One of these
    days I might clean it up a little and release it.

    If you have any comments about the page, I'd be happy to hear them.

    One of these days, I'm going to a full-on software review of the
    proxies for a variety of different purposes. Content filtering and
    blocking seems to be much in demand.

    regards,

    --
    alan kennedy
    -----------------------------------------------------
    check http headers here: http://xhaus.com/headers
    email alan: http://xhaus.com/mailto/alan
     
    Alan Kennedy, Nov 19, 2003
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steve C. Orr, MCSD
    Replies:
    1
    Views:
    595
    reaway lee
    Aug 24, 2003
  2. Jack
    Replies:
    6
    Views:
    403
  3. Jean-Paul Calderone
    Replies:
    9
    Views:
    1,147
  4. KaiWen
    Replies:
    102
    Views:
    2,880
    Jorgen Grahn
    Sep 15, 2011
  5. Greg
    Replies:
    4
    Views:
    335
    Steven Simpson
    Aug 16, 2012
Loading...

Share This Page