HTTP traffic over a custom proxy server?

Discussion in 'Java' started by Knorpi, Jun 24, 2011.

  1. Knorpi

    Knorpi Guest

    Hi,

    I would like to setup a server "myserver" that acts as a proxy for its clients. The clients pass the url of the web page they want in a query string:

    - client sends a HTTP request:
    http://myserver?url=www.foreignurl.com&par2=val1

    - Servlet on myserver extracts url from query string:
    www.foreignurl.com

    - Servlet sends HTTP request to other server with query string:
    http://www.foreignurl.com?par1=val1

    - Servlet renders complete response from other server
    into a String "content"

    - Servlet changes every link in the content to point to myserver
    so that all links are accessed the same way

    - Servlet sends response to client

    The most important part would be to change all links in the response.
    Can you give me some hints how to do this?

    Thanks
    Magnus
    Knorpi, Jun 24, 2011
    #1
    1. Advertising

  2. Knorpi

    Stefan Ram Guest

    Knorpi <> writes:
    >Can you give me some hints how to do this?


    http://tools.ietf.org/rfc/rfc2616.txt

    already defines how an HTTP proxy should behave, so
    I would suggest to implement this protocol instead
    of a custom protocol.
    Stefan Ram, Jun 24, 2011
    #2
    1. Advertising

  3. Knorpi

    Tom Anderson Guest

    On Fri, 24 Jun 2011, Knorpi wrote:

    > I would like to setup a server "myserver" that acts as a proxy for its
    > clients. The clients pass the url of the web page they want in a query
    > string:
    >
    > - client sends a HTTP request:
    > http://myserver?url=www.foreignurl.com&par2=val1
    >
    > - Servlet on myserver extracts url from query string:
    > www.foreignurl.com
    >
    > - Servlet sends HTTP request to other server with query string:
    > http://www.foreignurl.com?par1=val1


    This seems a mildly odd way of doing this. As Stefan suggests, it might be
    better to do whatever it is you're doing as a normal, proper, proxy
    server.

    > - Servlet renders complete response from other server
    > into a String "content"
    >
    > - Servlet changes every link in the content to point to myserver
    > so that all links are accessed the same way
    >
    > - Servlet sends response to client
    >
    > The most important part would be to change all links in the response.
    > Can you give me some hints how to do this?


    If you're going to do it properly, you really need to parse the HTML,
    identify all the a elements which have hrefs, rewrite each of those, and
    write out the modified HTML. Doing this in the face of bad HTML, and doing
    it in such a way that it won't radically alter the pages, may be quite
    tricky.

    tom

    --
    IMPORTANCE MEMO: >>> WHEN YOU BUY AN N-GAGE QD <<< PLEASE, please CONTINUE
    TO TALK ON THE SIDE!!$ Note: the other party will not be able to hear you,
    BUT WHO REALLY CRAPS A THING, SIDETALKIN' 2009++!!!
    Tom Anderson, Jun 25, 2011
    #3
  4. Knorpi

    Roedy Green Guest

    On Fri, 24 Jun 2011 03:24:25 -0700 (PDT), Knorpi
    <> wrote, quoted or indirectly quoted
    someone who said :

    >
    >The most important part would be to change all links in the response.
    >Can you give me some hints how to do this?


    What is the advantage of doing this? Without some sort of caching, it
    seems all you get is extra overhead.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    One of the great annoyances in programming derives from the irregularity
    of English spelling especially when you have international teams.
    I want to find a method or variable, but I don't know precisely
    how its is spelled or worded. English is only approximately phonetic.
    Letters are randomly doubled. The dictionary often lists variant spellings.
    British, Canadian and American spellings differ.I would like to see an
    experiment where variable names were spelled in a simplified English, where
    there were no double letters.I also think you could add a number of rules
    about composing variable names so that a variable name for something would
    be highly predictable. You would also need automated enforcement of the
    rules as well as possible.
    Roedy Green, Jun 26, 2011
    #4
  5. Knorpi

    Esmond Pitt Guest

    On 24/06/2011 8:24 PM, Knorpi wrote:
    > The most important part would be to change all links in the response.


    Why? If the client knows about the proxy, the proxy will get all the
    link requests, and it just needs to do what it is already doing.

    And change them to *what* exactly?
    Esmond Pitt, Jun 27, 2011
    #5
  6. Knorpi

    Silvio Guest

    On 06/24/2011 12:24 PM, Knorpi wrote:
    > Hi,
    >
    > I would like to setup a server "myserver" that acts as a proxy for its clients. The clients pass the url of the web page they want in a query string:
    >
    > - client sends a HTTP request:
    > http://myserver?url=www.foreignurl.com&par2=val1
    >
    > - Servlet on myserver extracts url from query string:
    > www.foreignurl.com
    >
    > - Servlet sends HTTP request to other server with query string:
    > http://www.foreignurl.com?par1=val1


    Sounds like a proxy on drugs...

    >
    > - Servlet renders complete response from other server
    > into a String "content"
    >
    > - Servlet changes every link in the content to point to myserver
    > so that all links are accessed the same way


    This is not generally possible. A page might use JavaScript to generate
    URLs dynamcally and you would not be able to hook into that.

    You can use the <base href="..."/> tag for achieving something similar
    for relative URLs.

    Ceating a proper proxy and setting it as the proxy for the user agents
    would solve this automatically.

    Silvio


    >
    > - Servlet sends response to client
    >
    > The most important part would be to change all links in the response.
    > Can you give me some hints how to do this?
    >
    > Thanks
    > Magnus
    Silvio, Jun 28, 2011
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. yoda
    Replies:
    5
    Views:
    4,641
  2. Knorpi
    Replies:
    3
    Views:
    349
  3. Knorpi
    Replies:
    1
    Views:
    329
  4. Dan Kohn
    Replies:
    2
    Views:
    129
    Dan Kohn
    Dec 3, 2005
  5. Greg
    Replies:
    4
    Views:
    292
    Steven Simpson
    Aug 16, 2012
Loading...

Share This Page