How do I PUSH an HTTP::POST using perl LWP?

Discussion in 'Perl Misc' started by joemacbusiness@gmail.com, Jul 23, 2008.

  1. Guest

    Hi All,

    I want to use http POST to push several gzipped files
    to another apache server. We need to use this since our
    customers only allow access to port 80, so no ssh and no ftp.

    I am using perl LWP modules for this.
    The problem is that it seems like the POST works like a get (pull)
    when I really want it to *push* the many *gz files to another
    host.

    Here is the code I have so far. This POST works like a "get".

    ========================= snip =========================

    use HTTP::Request::Common qw(POST);
    use LWP 5.64; # get the most recent stuff.
    $ua = LWP::UserAgent->new;

    #########################################################
    # this builds a list of files that need to be transferred.
    # it creates a local file: /tmp/zipped.list
    #########################################################
    #
    my $req = HTTP::Request->new(POST => 'http://somehostname.company.com/
    ~user/zipped.list');
    $req->content_type('text');

    open(F,">/tmp/zipped.list") || print "cannot open /tmp/zipped.list";
    #binmode F;
    print F $ua->request($req)->content;
    close(F);

    #########################################################
    # this loops through the list and (supposedly) copies the
    # files **to** the desthost (ie a push).
    # But it really pulls the files down from the desthost
    # I want it to push the files to desthost.
    #########################################################
    #

    my $infile = "/tmp/zipped.list";
    open(FILELIST,"$infile") || print "cannot open $infile $!";
    my @filearray = <FILELIST>;
    close(FILELIST);

    chdir("/tmp/shipping");
    my $pwd = `pwd`;
    print " ================================ working directory = $pwd \n";
    foreach my $fname (@filearray){
    chomp($fname);
    print "transferring $fname .... \n";

    my $req = HTTP::Request->new(POST => "http://desthost.company2.com/
    ~user/$fname");
    $req->content_type('application/gzip');

    open(F,">/tmp/shipping/$fname") || print "cannot open /tmp/
    shipping/$fname";
    binmode F;
    print F $ua->request($req)->content;
    close(F);
    }

    ========================= end snip ======================

    How do I make the code PUSH the gzipped files to desthost?
    This assumes that I have permission on desthost for ~user, which I do.

    Any help would be appreciated,

    Thanks, Joe M.
     
    , Jul 23, 2008
    #1
    1. Advertising

  2. Ben Morrow Guest

    Quoth :
    >
    > I want to use http POST to push several gzipped files
    > to another apache server. We need to use this since our
    > customers only allow access to port 80, so no ssh and no ftp.


    What do you have running on the server end (that is, the host you are
    POSTing to)? What have you set up to handle the POST requests?

    Note that you can't just POST to some random URL and expect it to
    overwrite the file at that URL. Firstly, that would be a huge security
    hole; secondly, the HTTP method for doing that is PUT, not POST; and
    thirdly, unless you've installed something like WebDAV at the receiving
    end Apache won't be willing to honour the request.

    > I am using perl LWP modules for this.
    > The problem is that it seems like the POST works like a get (pull)
    > when I really want it to *push* the many *gz files to another
    > host.


    POST sends some data to the remote end, and then gets a reply.

    > Here is the code I have so far. This POST works like a "get".
    >
    > ========================= snip =========================
    >
    > use HTTP::Request::Common qw(POST);
    > use LWP 5.64; # get the most recent stuff.
    > $ua = LWP::UserAgent->new;
    >
    > #########################################################
    > # this builds a list of files that need to be transferred.
    > # it creates a local file: /tmp/zipped.list
    > #########################################################
    > #
    > my $req = HTTP::Request->new(POST => 'http://somehostname.company.com/
    > ~user/zipped.list');
    > $req->content_type('text');


    Is this supposed to be a push or a pull of zipped.list? If it's supposed
    to be a push, then you need to set $req->content. Otherwise, nothing
    will get sent.

    Also, 'text' is not a valid Content-Type. You probably mean
    'text/plain', and you should specify a charset.

    > open(F,">/tmp/zipped.list") || print "cannot open /tmp/zipped.list";


    Use lexical filehandles.

    Use 3-arg open.

    Include the reason the open failed in the error message.

    Errors should go to STDOUT, and unless you have some more sophisticated
    error-handling, should 'die'.

    You name that file at least three times: anything you name more than
    once should go in a variable.

    my $ZIPLIST = '/tmp/zipped.list';

    open(my $F, '>', $ZIPLIST) || die "cannot open $ZIPLIST: $!";

    > #binmode F;
    > print F $ua->request($req)->content;
    > close(F);


    Again, are you pushing or pulling zipped.list? If you're trying to push
    it, why are you opening the local copy for writing?

    > #########################################################
    > # this loops through the list and (supposedly) copies the
    > # files **to** the desthost (ie a push).
    > # But it really pulls the files down from the desthost
    > # I want it to push the files to desthost.
    > #########################################################
    > #
    >
    > my $infile = "/tmp/zipped.list";
    > open(FILELIST,"$infile") || print "cannot open $infile $!";
    > my @filearray = <FILELIST>;
    > close(FILELIST);


    You've just written this data out. Keep hold of it rather than reading
    it back in again.

    > chdir("/tmp/shipping");
    > my $pwd = `pwd`;


    use Cwd qw/cwd/;

    my $pwd = cwd;

    > print " ================================ working directory = $pwd \n";
    > foreach my $fname (@filearray){
    > chomp($fname);


    'chomp' is not usually the right tool for stripping newlines from
    something received from the network. Typically, data received from the
    network has CRLF line endings, and unless you've changed $/ chomp will
    be removing LF only.

    > print "transferring $fname .... \n";
    >
    > my $req = HTTP::Request->new(POST => "http://desthost.company2.com/
    > ~user/$fname");


    You will need a CGI (or some other form of server-side intelligence) set
    up at this URL to handle the POST. Otherwise, Apache will just drop the
    POSTed data on the floor and return the contents of the file.

    > $req->content_type('application/gzip');


    application/gzip is not a registered Content-Type. Either use
    application/x-gzip or stick to application/octet-stream.

    > open(F,">/tmp/shipping/$fname") || print "cannot open /tmp/
    > shipping/$fname";
    > binmode F;
    > print F $ua->request($req)->content;
    > close(F);


    Again, I'm confused about whether you think you're pushing or pulling;
    you're overwriting the local file, which suggests you're trying to pull.
    If you want to push data you have to *read* from the local file and then
    set $req->content to its contents *before* you make the request.

    Ben

    --
    The Earth is degenerating these days. Bribery and corruption abound.
    Children no longer mind their parents, every man wants to write a book,
    and it is evident that the end of the world is fast approaching.
    Assyrian stone tablet, c.2800 BC
     
    Ben Morrow, Jul 24, 2008
    #2
    1. Advertising

  3. On Wed, 23 Jul 2008 15:26:05 -0700, joemacbusiness wrote:

    > Hi All,
    >
    > I want to use http POST to push several gzipped files to another apache
    > server. We need to use this since our customers only allow access to
    > port 80, so no ssh and no ftp.


    If the webserver is only used for this, run sshd on port 80. :)

    M4
     
    Martijn Lievaart, Aug 2, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. XyZaa
    Replies:
    0
    Views:
    573
    XyZaa
    Jul 19, 2007
  2. samppi
    Replies:
    27
    Views:
    494
    David A. Black
    Dec 5, 2007
  3. Mark Mykkanen

    LWP - Upload via HTTP POST

    Mark Mykkanen, Jul 2, 2003, in forum: Perl Misc
    Replies:
    1
    Views:
    370
    J. Gleixner
    Jul 2, 2003
  4. Zapanaz
    Replies:
    4
    Views:
    136
    Tad McClellan
    Oct 12, 2005
  5. pratap

    HTTP POST without LWP

    pratap, Dec 26, 2006, in forum: Perl Misc
    Replies:
    11
    Views:
    527
    Tad McClellan
    Dec 29, 2006
Loading...

Share This Page