HTTP Proxy via HTTP Layer by Perl?

N

nntp

I am not sure how proxy works. The basic is it got a request then forward
it.

I need a perl program for doing that. There are windows, binary based
proxys. However, I have to have broadband to use it, then it will cost $50 a
month for only fast down, but slow up. If I have a Perl/PHP proxy, and use
web server port 80, then I only need to pay $10 web hosting fee for super
fast connections up and down.

I searched cpan but there is none like that. I went to hotscripts. The only
proxy is web based. But I need http layer so that I just enter the ip and
port in browser then I am ready to go.

Anyone knows such programs?
 
N

nntp

I am not sure how proxy works. The basic is it got a request then forward
it.

I need a perl program for doing that. There are windows, binary based
proxys. However, I have to have broadband to use it, then it will cost $50 a
month for only fast down, but slow up. If I have a Perl/PHP proxy, and use
web server port 80, then I only need to pay $10 web hosting fee for super
fast connections up and down.

I searched cpan but there is none like that. I went to hotscripts. The only
proxy is web based. But I need http layer so that I just enter the ip and
port in browser then I am ready to go.

Anyone knows such programs?

There are some on cpan. Because I don't know much about the technical terms,
so I am not sure which is which

http://search.cpan.org/~rwahby/Net-HTTPTunnel-0.4/HTTPTunnel.pm
http://search.cpan.org/~book/HTTP-Proxy-0.13/lib/HTTP/Proxy.pm

I could not match what it does to what I am trying to do.
 
N

nntp

A. Sinan Unur said:
I am not sure why this is relevant. This is a programming newsgroup. If you
are working on a program, and have a question regarding that program which
we hope is in Perl, then you post it here and get help.
I posted two modules from cpan and ask if they are what I need
I don't understand the program. I don't see any demo either. It looks like
to me a fake clicking software. I am not asking for that. I need about 5-10
private proxy servers set up around the world for a trading business. I need
the lowest cost way.
 
S

Sherm Pendley

nntp said:
I need a perl program for doing that.

Why must it be Perl? Seriously - if you're looking for a canned
solution, then you should be looking for the functionality you need,
period, and ignoring the implementation language.

Also - why are you posting this to a Perl group? Unless you're writing
your own proxy in Perl, this is not the place to post your question.
This group is about *writing* Perl, not discussing applications that
happen to have been written in Perl.
There are windows, binary based
proxys. However, I have to have broadband to use it, then it will cost $50 a
month for only fast down, but slow up. If I have a Perl/PHP proxy, and use
web server port 80, then I only need to pay $10 web hosting fee for super
fast connections up and down.

I don't follow you here, and to be honest I think you're confused. Using
a proxy that's out on the web won't magically make your dialup line as
fast as broadband. The connection between your proxy server and the
target site will be fast, yes, but the connection between you and the
proxy will be your dialup connection.
I searched cpan but there is none like that. I went to hotscripts. The only
proxy is web based. But I need http layer so that I just enter the ip and
port in browser then I am ready to go.

Some serious advice: Learn how the web works before you try to make it
work better. Really. That's not meant as a slam or an attack, just
honest advice. You appear to be very confused, and you're not making
much sense.

Having said that, one of the most popular cacheing servers in use right
now is Squid:

<http://www.squid-cache.org>

The first link on the Squid site's "recommended reading" list looks to
me like a fairly good introduction to cacheing. You *seriously* need to
read this to get better acquainted with the technology:

<http://www.mnot.net/cache_docs/>

There are also a couple of O'Reilly books available, if you prefer dead
trees:

<http://book.web-cache.com>

sherm--
 
N

nntp

Some serious advice: Learn how the web works before you try to make it
work better. Really. That's not meant as a slam or an attack, just
honest advice. You appear to be very confused, and you're not making
much sense.

Having said that, one of the most popular cacheing servers in use right
now is Squid:
I need proxy, not cache.
for example, let's call the script proxy.pl
i make it sitting on root dir, set .htaccess use proxy.pl as default index
then i use the virtual server in apache to proint to this dir....
i enter the url in proxy and port 80 in my IE
then when i go to citibank.com, it go through than proxy.
i need https as well.
 
A

A. Sinan Unur

I posted two modules from cpan and ask if they are what I need

How do you propose that we should be able to figure out what you _really_
need if you refuse to specify it.

The easiest way to see if HTTP::proxy does what you want is to write a
program that uses it to see if it does what you want. It probably does but
then you haven't really explained what you want to do.
I don't understand the program. I don't see any demo either. It looks
like to me a fake clicking software. I am not asking for that.

It is a proxy application written in pure Perl. Thought it might have
hepled you figure out how proxying works. You could also look at the source
code for HTTP::proxy obviously.
I need about 5-10 private proxy servers set up around the world for
a trading business. I need the lowest cost way.

Again, this is still not relevant.

Sinan.
 
N

nntp

I need a perl program for doing that. There are windows, binary based
$50

There are some on cpan. Because I don't know much about the technical terms,
so I am not sure which is which

http://search.cpan.org/~rwahby/Net-HTTPTunnel-0.4/HTTPTunnel.pm
http://search.cpan.org/~book/HTTP-Proxy-0.13/lib/HTTP/Proxy.pm

I could not match what it does to what I am trying to do.

This is the code I got online, but it even crash in shell mod in windows, so
I don't know if it will work in CGI enviroment

#####################
#!/usr/usc/bin/perl
#-------------------------------------------------------------------------
#-- proxy.pl - A simple http proxy server. --
#-- --
#-- To run, type proxy.pl [port-number] at the shell prompt. --
#-- Default port number is 5364. --
#-- --
#-------------------------------------------------------------------------
#require "sys/socket.ph";
use Socket;

srand (time||$$);
#--- Define a friendly exit handler
$SIG{'KILL'} = $SIG{QUIT} = $SIG{INT} = 'exit_handler';
sub exit_handler {
print "\n\n --- Proxy server is dying ...\n\n";
close(SOCKET);
exit;

}
#--- Setup socket

$| = 1;
$proxy_port = shift(@ARGV);
$proxy_port = 5364 unless $proxy_port =~ /\d+/;

$socket_format = 'S n a4 x8';
&listen_to_port(SOCKET, $proxy_port);
$local_host = `hostname`;
chop($local_host);
$local_host_ip = (gethostbyname($local_host))[4];
print " --- Proxy server running on $local_host port: $proxy_port \n\n";
#--- Loop forever taking requests as they come
while (1) {
#--- Wait for request
print " --- Waiting to be of service ...\n";
($addr = accept(CHILD,SOCKET)) || die "accept $!";
($port,$inetaddr) = (unpack($socket_format,$addr))[1,2];
@inetaddr = unpack('C4',$inetaddr);
print "Connection from ", join(".", @inetaddr), " port: $port \n";
#--- Fork a subprocess to handle request.
#--- Parent proces continues listening.
if (fork) {
wait; # For now we wait for the child to finish
next; # We wait so that printouts don't mix
}
#--- Read first line of request and analyze it.
#--- Return and edited version of the first line and the request method.
($first,$method) = &analyze_request;
#--- Send request to remote host
print URL $first;
print $first;
while (<CHILD>) {
print $_;
next if (/Proxy-Connection:/);
print URL $_;
last if ($_ =~ /^[\s\x00]*$/);
}
if ($method eq "POST") {
$data = <CHILD>;
print $data;
print URL $data;
}
print URL "\n";
#--- Wait for response and transfer it to requestor.
print " --- Done sending. Response: \n\n";
$header = 1;
$text = 0;
while (<URL>) {
print CHILD $_;
if ($header || $text) { # Only print header & text lines to STDOUT
print $_;
if ($header && $_ =~ /^[\s\x00]*$/) {
$header = 0;
}
# if ($header && $_ =~ /^Content-type: text/) {
# $text = 1;
# }
}
}
close(URL);
close(CHILD);
exit; # Exit from child process
}
#-------------------------------------------------------------------------
#-- analyze_request --
#-- --
#-- Analyze a new request. First read in first line of request. --
#-- Read URL from it, process URL and open connection. --
#-- Return an edited version of the first line and the request --
#-- method. --
#-------------------------------------------------------------------------
sub analyze_request {
#--- Read first line of HTTP request
$first = <CHILD>;

$url = ($first =~ m|(http://\S+)|)[0];
print "Request for URL: $url \n";

#--- Check if first line is of the form GET http://host-name ...
($method, $remote_host, $remote_port) =
($first =~ m!(GET|POST|HEAD) http://([^/:]+):?(\d*)! );
#--- If not, bad request.

if (!$remote_host) {
print $first;
while (<CHILD>) {
print $_;
last if ($_ =~ /^[\s\x00]*$/);
}
print "Invalid HTTP request from ", join(".", @inetaddr), "\n";
# print CHILD "Content-type: text/plain","\n\n";
print CHILD "I don't understand your request.\n";
close(CHILD);
exit;
}
#--- If requested URL is the proxy server then ignore request
$remote_ip = (gethostbyname($remote_host))[4];
if (($remote_ip eq $local_host_ip) && ($remote_port eq $proxy_port)) {
print $first;
while (<CHILD>) {
print $_;
last if ($_ =~ /^[\s\x00]*$/);
}
print " --- Connection to proxy server ignored.\n";
# print CHILD "Content-type: text/plain","\n\n";
print CHILD "It's not nice to make me loop on myself!.\n";
close(CHILD);
exit;
}
#--- Setup connection to target host and send request
$remote_port = "http" unless ($remote_port);
&open_connection(URL, $remote_host, $remote_port);
#--- Remove remote hostname from URL
$first =~ s/http:\/\/[^\/]+//;
($first, $method);
}
#-------------------------------------------------------------------------
#-- listen_to_port(SOCKET, $port) --
#-- --
#-- Create a socket that listens to a specific port --
#-------------------------------------------------------------------------
sub listen_to_port {
local ($port) = $_[1];
local ($socket_format, $proto, $packed_port, $cur, $max_requests);
$max_requests = 3; # Max number of outstanding requests
$socket_format = 'S n a4 x8';
$proto = (getprotobyname('tcp'))[2];
$packed_port = pack($socket_format, &AF_INET, $port, "\0\0\0\0");
socket($_[0], &PF_INET, &SOCK_STREAM, $proto) || die "socket: $!";
bind($_[0], $packed_port) || die "bind: $!";
listen($_[0], $max_requests) || die "listen: $!";
$cur = select($_[0]);
$| = 1; # Disable buffering on socket.
select($cur);
}

#-------------------------------------------------------------------------
#-- open_connection(SOCKET, $remote_hostname, $port) --
#-- --
#-- Create a socket that connects to a certain host --
#-- $local_host_ip is assumed to be local hostname IP address --
#-------------------------------------------------------------------------
sub open_connection {
local ($remote_hostname, $port) = @_[1,2];
local ($socket_format, $proto, $packed_port, $cur);
local ($remote_addr, @remote_ip, $remote_ip);
local ($local_port, $remote_port);
if ($port !~ /^\d+$/) {
$port = (getservbyname($port, "tcp"))[2];
$port = 80 unless ($port);
}
$proto = (getprotobyname('tcp'))[2];
$remote_addr = (gethostbyname($remote_hostname))[4];
if (!$remote_addr) {
die "Unknown host: $remote_hostname";
}

@remote_ip = unpack("C4", $remote_addr);
$remote_ip = join(".", @remote_ip);
print "Connecting to $remote_ip port $port.\n\n";
$socket_format = 'S n a4 x8';
$local_port = pack($socket_format, &AF_INET, 0, $local_host_ip);
$remote_port = pack($socket_format, &AF_INET, $port, $remote_addr);
socket($_[0], &AF_INET, &SOCK_STREAM, $proto) || die "socket: $!";
bind($_[0], $local_port) || die "bind: $!";
connect($_[0], $remote_port) || die "socket: $!";
$cur = select($_[0]);

$| = 1; # Disable buffering on socket.
select($cur);
}
 
S

Sherm Pendley

nntp said:
I need proxy, not cache.

Please - read the docs I pointed you to. Squid is a caching proxy, but
you can easily - *if* you *read* the *docs* - disable its cache to turn
it into a plain proxy.

And your babble about making your dialup connection faster by putting a
proxy on a $10/month hosting account is still utter nonsense.

sherm--
 
N

nntp

And your babble about making your dialup connection faster by putting a
proxy on a $10/month hosting account is still utter nonsense.

I don't use dialup.

For example, if I buy a high speed internet to use as a windows/linux proxy,
then I need to pay $50 monthly fee, and the speed is 3M down and 300K up.
On a web proxy, the down/up rate is equal. Close to 1M, which is good, as
proxy is two way in and out. So a cable proxy is only good at 300k at most.

Thanks for the doc. I am going to read it Harder!!!.
 
T

Tad McClellan

nntp said:
This is the code I got online,


It is written in an eight year old style.

I wouldn't trust a programmer or program that does it the way it
was done 8 years ago.

It is very likely worth less than what you paid for it...

$local_host = `hostname`;
chop($local_host);


The modern way to remove newlines is chomp().

sub listen_to_port {
local ($port) = $_[1];
local ($socket_format, $proto, $packed_port, $cur, $max_requests);


The modern type of variables are lexical, rather than package, variables.

my($port) = $_[1];
my($socket_format, $proto, $packed_port, $cur, $max_requests);
 
S

Sherm Pendley

nntp said:
For example, if I buy a high speed internet to use as a windows/linux proxy,
then I need to pay $50 monthly fee, and the speed is 3M down and 300K up.
On a web proxy, the down/up rate is equal. Close to 1M, which is good, as
proxy is two way in and out. So a cable proxy is only good at 300k at most.

Okay, let me see if I can sort this out. I like a challenge. ;-)

You want to set up a web server. If you put it on your home cable
connection, it won't be as quick because that connection limits the
outgoing bandwidth. (Also, although you don't mention it, most cable
providers strictly prohibit servers.)

Now, on this web server of yours, you're looking for a CGI that forwards
the requests it receives to another server, retrieves the results, and
then sends the results to the browser.

You *might* be able to get away with a simple little script that uses
the HTTP::proxy module. But, you'd need to run that - or most any other
"canned" proxy application I can think of - as a daemon (aka server)
process. That's not likely to be something you'll be allowed to do on a
budget $10/month hosting account. You could do it with a dedicated
server, but the cheapest I've seen that for is $79/month.

If you want a CGI, you'll probably need to write one. You'll need to
send the request to the remote server and retrieve the results - use
LWP::UserAgent for that. Of course, if you want links on the returned
pages to work, you'll need to parse the returned HTML and fix those -
HTML::parser will help with that.

sherm--
 
J

Joe Smith

nntp said:
I am not sure how proxy works. The basic is it got a request then forward
it.

Yes, that is what a proxy server does.
The standard http proxy requires that you inform your browser that
you are using a proxy, so that the browser (MSIE, Netscape) will use
the proxy for all requests.
However, I have to have broadband to use it, then it will cost $50 a
month for only fast down, but slow up.

That is typical for residential broadband access to the Internet.
It allows high speed for download (to your computer from the Internet)
but has slow speed for upload (from your computer to the Internet).

If I have a Perl/PHP proxy, and use web server port 80, then I only need to
pay $10 web hosting fee for super fast connections up and down.

I seriously doubt that. Most $10/month web sites do not allow CGI.
Even if you are allowed to run programs there, the super fast connections
are between the web hosting server and the rest of the Internet.
It will *not* make the connection between your computer and anywhere
else faster.
I need about 5-10 private proxy servers set up around the
world for a trading business. I need the lowest cost way.

It would make sense to have mirror servers around the world, not proxy
servers. A proxy server may make things faster if you and everyone who
does business with you are all on the same high speed network. But since
you're not using broadband, a proxy won't help.

If you are serious about doing business, you should sign up for a
business-grade shared server at a colocation site. More than $50/month.
Perl is not the solution.
-Joe
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,769
Messages
2,569,579
Members
45,053
Latest member
BrodieSola

Latest Threads

Top