Using an HTTP proxy to record web traffic

D

Dan Kohn

As part of debugging WWW::Mechanize, I realized that the best way to
compare what my browser is doing to what mechanize is doing would be to
compare their HTTP transactions incoming and outgoing. What I want is
an HTTP proxy that can record the transactions and also supports SSL.

The Ruby project doesn't have any code
http://rubyforge.org/projects/rpp/

This free program doesn't support SSL, and costs $600.
http://www.proxy-sniffer.com/download.html

This doesn't support SSL but is supposed to soon.
http://kevinlangdon.com/serviceCapture/

Any other suggestions?
 
W

why the lucky stiff

Dan said:
As part of debugging WWW::Mechanize, I realized that the best way to
compare what my browser is doing to what mechanize is doing would be to
compare their HTTP transactions incoming and outgoing. What I want is
an HTTP proxy that can record the transactions and also supports SSL.
MouseHole is a scriptable proxy. <http://mousehole.rubyforge.org/>

A script to record traffic would a cinch. Basically,

1. Download MouseHole from the above link. Instructions for running it
are there as well.
2. Add a script named `proxyrecord.user.rb' to your
~/.mouseHole/userScripts/ directory.
Contains:

MouseHole.script do
name "Proxy Record"
namespace "http://yourdomain.com/"
description "Records all traffic, very snoopy."
include_match %r!http://.*!
version "1.0"

rewrite do |req, res|
# store contents of `req' and `res' in the database or a
# log file or something
end
end
3. In your browser, setup MouseHole as your proxy (http://127.0.0.1:37004/)
4. Visit http://127.0.0.1:37004/ and activate the Proxy Record script by
checking its box.

As for SSL, I don't think proxies can actually intercept unencrypted
SSL. It uses the CONNECT method.

_why
 
D

Dan Kohn

Very helpful, why, thanks. I presume I would need to use something
like WATIR to see the unencrypted contents of an SSL connection by
getting it directly from IE.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top