How do I PUSH an HTTP::POST using perl LWP?


J

joemacbusiness

Hi All,

I want to use http POST to push several gzipped files
to another apache server. We need to use this since our
customers only allow access to port 80, so no ssh and no ftp.

I am using perl LWP modules for this.
The problem is that it seems like the POST works like a get (pull)
when I really want it to *push* the many *gz files to another
host.

Here is the code I have so far. This POST works like a "get".

========================= snip =========================

use HTTP::Request::Common qw(POST);
use LWP 5.64; # get the most recent stuff.
$ua = LWP::UserAgent->new;

#########################################################
# this builds a list of files that need to be transferred.
# it creates a local file: /tmp/zipped.list
#########################################################
#
my $req = HTTP::Request->new(POST => 'http://somehostname.company.com/
~user/zipped.list');
$req->content_type('text');

open(F,">/tmp/zipped.list") || print "cannot open /tmp/zipped.list";
#binmode F;
print F $ua->request($req)->content;
close(F);

#########################################################
# this loops through the list and (supposedly) copies the
# files **to** the desthost (ie a push).
# But it really pulls the files down from the desthost
# I want it to push the files to desthost.
#########################################################
#

my $infile = "/tmp/zipped.list";
open(FILELIST,"$infile") || print "cannot open $infile $!";
my @filearray = <FILELIST>;
close(FILELIST);

chdir("/tmp/shipping");
my $pwd = `pwd`;
print " ================================ working directory = $pwd \n";
foreach my $fname (@filearray){
chomp($fname);
print "transferring $fname .... \n";

my $req = HTTP::Request->new(POST => "http://desthost.company2.com/
~user/$fname");
$req->content_type('application/gzip');

open(F,">/tmp/shipping/$fname") || print "cannot open /tmp/
shipping/$fname";
binmode F;
print F $ua->request($req)->content;
close(F);
}

========================= end snip ======================

How do I make the code PUSH the gzipped files to desthost?
This assumes that I have permission on desthost for ~user, which I do.

Any help would be appreciated,

Thanks, Joe M.
 
Ad

Advertisements

B

Ben Morrow

Quoth (e-mail address removed):
I want to use http POST to push several gzipped files
to another apache server. We need to use this since our
customers only allow access to port 80, so no ssh and no ftp.

What do you have running on the server end (that is, the host you are
POSTing to)? What have you set up to handle the POST requests?

Note that you can't just POST to some random URL and expect it to
overwrite the file at that URL. Firstly, that would be a huge security
hole; secondly, the HTTP method for doing that is PUT, not POST; and
thirdly, unless you've installed something like WebDAV at the receiving
end Apache won't be willing to honour the request.
I am using perl LWP modules for this.
The problem is that it seems like the POST works like a get (pull)
when I really want it to *push* the many *gz files to another
host.

POST sends some data to the remote end, and then gets a reply.
Here is the code I have so far. This POST works like a "get".

========================= snip =========================

use HTTP::Request::Common qw(POST);
use LWP 5.64; # get the most recent stuff.
$ua = LWP::UserAgent->new;

#########################################################
# this builds a list of files that need to be transferred.
# it creates a local file: /tmp/zipped.list
#########################################################
#
my $req = HTTP::Request->new(POST => 'http://somehostname.company.com/
~user/zipped.list');
$req->content_type('text');

Is this supposed to be a push or a pull of zipped.list? If it's supposed
to be a push, then you need to set $req->content. Otherwise, nothing
will get sent.

Also, 'text' is not a valid Content-Type. You probably mean
'text/plain', and you should specify a charset.
open(F,">/tmp/zipped.list") || print "cannot open /tmp/zipped.list";

Use lexical filehandles.

Use 3-arg open.

Include the reason the open failed in the error message.

Errors should go to STDOUT, and unless you have some more sophisticated
error-handling, should 'die'.

You name that file at least three times: anything you name more than
once should go in a variable.

my $ZIPLIST = '/tmp/zipped.list';

open(my $F, '>', $ZIPLIST) || die "cannot open $ZIPLIST: $!";
#binmode F;
print F $ua->request($req)->content;
close(F);

Again, are you pushing or pulling zipped.list? If you're trying to push
it, why are you opening the local copy for writing?
#########################################################
# this loops through the list and (supposedly) copies the
# files **to** the desthost (ie a push).
# But it really pulls the files down from the desthost
# I want it to push the files to desthost.
#########################################################
#

my $infile = "/tmp/zipped.list";
open(FILELIST,"$infile") || print "cannot open $infile $!";
my @filearray = <FILELIST>;
close(FILELIST);

You've just written this data out. Keep hold of it rather than reading
it back in again.
chdir("/tmp/shipping");
my $pwd = `pwd`;

use Cwd qw/cwd/;

my $pwd = cwd;
print " ================================ working directory = $pwd \n";
foreach my $fname (@filearray){
chomp($fname);

'chomp' is not usually the right tool for stripping newlines from
something received from the network. Typically, data received from the
network has CRLF line endings, and unless you've changed $/ chomp will
be removing LF only.
print "transferring $fname .... \n";

my $req = HTTP::Request->new(POST => "http://desthost.company2.com/
~user/$fname");

You will need a CGI (or some other form of server-side intelligence) set
up at this URL to handle the POST. Otherwise, Apache will just drop the
POSTed data on the floor and return the contents of the file.
$req->content_type('application/gzip');

application/gzip is not a registered Content-Type. Either use
application/x-gzip or stick to application/octet-stream.
open(F,">/tmp/shipping/$fname") || print "cannot open /tmp/
shipping/$fname";
binmode F;
print F $ua->request($req)->content;
close(F);

Again, I'm confused about whether you think you're pushing or pulling;
you're overwriting the local file, which suggests you're trying to pull.
If you want to push data you have to *read* from the local file and then
set $req->content to its contents *before* you make the request.

Ben
 
Ad

Advertisements

M

Martijn Lievaart

Hi All,

I want to use http POST to push several gzipped files to another apache
server. We need to use this since our customers only allow access to
port 80, so no ssh and no ftp.

If the webserver is only used for this, run sshd on port 80. :)

M4
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top