simple ruby proxy server?

[Dt Town]

I am trying to right at application which will simply record the headers
of all get requests made from my browser.

It would be ideal to just listen in on port 80, but something tells me
that wont work because of security risks. So I was wondering if there is
a way to do this through a proxy. I am relatively new to ruby, and I
havent been able to find anything in the rdoc.

Any help is greatly appreciated. Thanks!

 

[dusty]

Well, this will work to record the headers. You'll need to install
the mongrel gem.

# gem install mongrel

# headerlog.rb
require 'rubygems'
require 'mongrel'
require 'logger'

class HeaderHandler < Mongrel::HttpHandler

@@logger = Logger.new('headers.log')

def process(request,response)
response.start(200) do |head, out|
head["Content-Type"] = "text/plain"
@@logger.info request.params
end
end
end



server = Mongrel::HttpServer.new("127.0.0.1", "2222")
server.register("/", HeaderHandler.new)
server.register("/favicon.ico", Mongrel::Error404Handler.new(""))
server.run.join


--- Then run it

# ruby headerlog.rb

Its set to listen on port 2222 and will record all the headers to a
file named headers.log. You could change that port to whatever you
want. I see no security implications of listening on port 80.

The only problem would be if you have something else listening on port
80. Are you trying to put something in front of an existing
application to capture the headers, then have it proxy the request to
something else? In that case, what kind of application is it? You
might be able to do that in the app itself. Or, perhaps turn on some
sort of debug logging in apache or whatever server you are using. I
might have some other ideas if this is more complex than the simple
example I listed above.

Hope that is helpful.

 

[Dt Town]

thanks for your help dusty. I haven't gotten a chance to run it yet (I
wont till tomorrow morning).

My approach thus far is to create a proxy server, and have IE or Firefox
route its requests through there. With that in mind I have 3 questions:

1) Is there another (doesn't necessarily have to be better) approach
that you would suggest?
2) If I set IE7 to proxy its requests to port 2222, does the ruby script
you provided return the requested data (i.e. does the web page still
load in the browser)
3) If I set the port to 80 in the script above, will it behave as a
"listener" and not interfere with the request from the browser (my guess
is no on this one).

Thanks for your help, I appreciate it very much.

 

[dusty]

I had a feeling it was more than just capturing the headers.

Do you happen to be running apache on the webserver? You can log all
of it and not worry about a proxy. Here are two ways.

1. mod_dumpio

# Put this in your apache config
DumpIOInput On
LogLevel debug

2. mod_log_forensic

# Put this in your apache config
ForensicLog /some/path/to/a/logfile.log

This would simply capture the traffic on the web server itself.

 

[dusty]

BTW - if you are just looking to do this on your computer so you can
see what is going on while you hit remote sites, then I'd just use
ethereal (or wireshark, I believe its called now). You can see the
whole packet, probably the easiest way.

Although, I'm sure someone on this list has written a ruby proxy
server before, I haven't. Sounds interesting though, I might take a
stab at it if I can find some spare time.

 

[Francis Cianfrocca]

[Note: parts of this message were removed to make it a legal post.]

BTW - if you are just looking to do this on your computer so you can
see what is going on while you hit remote sites, then I'd just use
ethereal (or wireshark, I believe its called now). You can see the
whole packet, probably the easiest way.

Although, I'm sure someone on this list has written a ruby proxy
server before, I haven't. Sounds interesting though, I might take a
stab at it if I can find some spare time.

There was some discussion of HTTP proxies on the Ruby/Eventmachine mailing
list a few weeks back, so you might search through that archive. It sounds
like the OP wants to write a reverse proxy, which isn't hard to do at all
using EventMachine. The thing that made the subject more interesting,
however, was the requirement that the proxy be transparent, so that the
proxied traffic appears to be coming from the original peer's source IP
address. That takes kernel support and is not available on all platforms.

 

[dusty]

Thanks for the heads up, I'll check it out. Been playing with event
machine a little bit recently, its nice!

 

[Dt Town]

thanks for the info guys. I have looked into wireshark, and I have been
back and forth on whether to do a packet level implementation (i.e. run
wireshark command line, and then parse the data), or do a http level
implementation (i.e. the proxy). The thing I dont like about packet
level is the sheer number of packets that I would have to sift through.

 

[GOTO Kentaro]

http://www.ruby-forum.com/topic/139859 may help you.

Gotoken

 

[dusty]

Nice!

 

[an an]

http://www.ruby-forum.com/topic/139859 may help you.

Gotoken

Thanks, that looks very helpful, I will give it a shot tomorrow.

 

[an an]

Hi Gotoken,

At first I thought it was the answer to my problems, but it seems that
not all pages will load with it. i think pages which are heavily ajax
dependent are the ones that dont load. I will try and tinker some more
with it.

 

[GOTO Kentaro]

Hi,

WEBrick::HTTPProxyServer is not fast and maybe too slow in your situation.
Could you report again when you find a simple way to reproduce the problem?
I'll try to help you trouble shooting then.

Thanks,


Gotoken