retrieve webpage with POST

Discussion in 'Perl Misc' started by Dean Banko, Dec 28, 2003.

  1. Dean Banko

    Dean Banko Guest

    Hello!

    Does anyone know how to retreive a page generated as form request?
    I need to post some data like in the row:
    my $req=HTTP::Request->new(POST=>$url);
    but there are also some parameters that I must put into POST, but I don't
    know the syntax :-(
    I know that must be something like:
    my $req=HTTP::Request->new(POST=>$url, [field=>'1', valid=>'yes');



    $url=http://www.somelink.com"
    my $ua=LWP::UserAgent->new();
    my $req=HTTP::Request->new(POST=>$url);
    my $response=$ua->request($req);
    if ($response->is_error()){$response->error_as_HTML()}
    $content=$response->content();

    Can anyone help me?

    Regards,
    Dean
    Dean Banko, Dec 28, 2003
    #1
    1. Advertising

  2. Dean Banko wrote:
    > Does anyone know how to retreive a page generated as form request?
    > I need to post some data like in the row:
    > my $req=HTTP::Request->new(POST=>$url);
    > but there are also some parameters that I must put into POST, but I
    > don't know the syntax :-(
    > I know that must be something like:
    > my $req=HTTP::Request->new(POST=>$url, [field=>'1', valid=>'yes');


    Assuming you are talking about two fields named 'field' and 'valid',
    and whose values you want to be '1' respective 'yes', I'd try:

    my $req=HTTP::Request->new(POST=>$url);
    $req->content_type('application/x-www-form-urlencoded');
    $req->content('field=1&valid=yes');

    (untested)

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Hjalmarsson, Dec 28, 2003
    #2
    1. Advertising

  3. Dean Banko <> wrote:

    > I know that must be something like:
    > my $req=HTTP::Request->new(POST=>$url, [field=>'1', valid=>'yes');



    use HTTP::Request::Common;
    $ua = LWP::UserAgent->new;
    $ua->request(POST $url, [field => '1', valid => 'yes']);


    > Can anyone help me?



    The documentation for HTTP::Request says to see also HTTP::Request::Common.

    If you'd paid close attention while reading the docs for the module
    that you are using, you wouldn't _need_ anyone else's help. :)


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Dec 28, 2003
    #3
  4. Tad McClellan wrote:
    > Dean Banko <> wrote:
    >>
    >> I know that must be something like:
    >> my $req=HTTP::Request->new(POST=>$url, [field=>'1', valid=>'yes');

    >
    > use HTTP::Request::Common;
    > $ua = LWP::UserAgent->new;
    > $ua->request(POST $url, [field => '1', valid => 'yes']);
    >
    >> Can anyone help me?

    >
    > The documentation for HTTP::Request says to see also
    > HTTP::Request::Common.


    Does it? I for one am not able to find that reference.

    Anyway, thanks for the tip. I suppose it results in the same HTTP
    request as my suggestion would do.

    > If you'd paid close attention while reading the docs for the module
    > that you are using, you wouldn't _need_ anyone else's help. :)


    Hrrrm...

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Hjalmarsson, Dec 29, 2003
    #4
  5. Dean Banko

    Matt Garrish Guest

    "Gunnar Hjalmarsson" <> wrote in message
    news:bsnsun$ejjdt$-berlin.de...
    > Tad McClellan wrote:
    > >
    > > The documentation for HTTP::Request says to see also
    > > HTTP::Request::Common.

    >
    > Does it? I for one am not able to find that reference.
    >


    Umm, it's right in the See Also section of the documentation. A quick skim
    through Poe's The Purloined Letter might be in order... ; )

    Matt
    Matt Garrish, Dec 29, 2003
    #5
  6. Gunnar Hjalmarsson <> wrote:
    > Tad McClellan wrote:


    >> The documentation for HTTP::Request says to see also
    >> HTTP::Request::Common.

    >
    > Does it? I for one am not able to find that reference.



    http://search.cpan.org/~gaas/libwww-perl-5.76/lib/HTTP/Request.pm


    > I suppose it results in the same HTTP
    > request as my suggestion would do.



    I do not know. I did not see your suggestion.


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Dec 29, 2003
    #6
  7. Tad McClellan wrote:
    > Gunnar Hjalmarsson <> wrote:
    >> Tad McClellan wrote:
    >>> The documentation for HTTP::Request says to see also
    >>> HTTP::Request::Common.

    >>
    >> Does it? I for one am not able to find that reference.

    >
    > http://search.cpan.org/~gaas/libwww-perl-5.76/lib/HTTP/Request.pm


    There it is, obviously... Thanks!

    To *not* find it, I searched CPAN for HTTP::Request and ended up at
    http://search.cpan.org/~rse/lcwa-1.0.0/lib/lwp/lib/HTTP/Request.pm
    -----------------------------------------------------^^^^^^^^^^^^

    I should probably have been more attentive, but...

    http://search.cpan.org/~rse/lcwa-1.0.0/ seems to be a 'historical
    archive' of past versions of quite a few modules. :(

    Is that in accordance with the PAUSE rules?

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Hjalmarsson, Dec 29, 2003
    #7
  8. Dean Banko

    Dean Banko Guest

    Hello Gunnar!

    I thank you a lot for your assistance!
    The syntax that you offered is functional, I've tested it and it works!

    Thank you very much!
    Happy New Year,
    Dean

    "Gunnar Hjalmarsson" <> wrote in message
    news:bsnnce$ekeav$-berlin.de...
    > Dean Banko wrote:
    > > Does anyone know how to retreive a page generated as form request?
    > > I need to post some data like in the row:
    > > my $req=HTTP::Request->new(POST=>$url);
    > > but there are also some parameters that I must put into POST, but I
    > > don't know the syntax :-(
    > > I know that must be something like:
    > > my $req=HTTP::Request->new(POST=>$url, [field=>'1', valid=>'yes');

    >
    > Assuming you are talking about two fields named 'field' and 'valid',
    > and whose values you want to be '1' respective 'yes', I'd try:
    >
    > my $req=HTTP::Request->new(POST=>$url);
    > $req->content_type('application/x-www-form-urlencoded');
    > $req->content('field=1&valid=yes');
    >
    > (untested)
    >
    > --
    > Gunnar Hjalmarsson
    > Email: http://www.gunnar.cc/cgi-bin/contact.pl
    >
    Dean Banko, Dec 29, 2003
    #8
  9. Dean Banko

    Dean Banko Guest

    Hello Gunnar and Tad!

    Thank you both for your assistance!
    I've read all refferences you gave, but nowhere is written so clear as
    Gunnar posted as the untested version to my original posting.

    The thing works now correctly and appreciate your help, both!

    I wish Happy New Year to you and your families,
    many success and health
    Dean Banko, Croatia

    "Gunnar Hjalmarsson" <> wrote in message
    news:bsoji2$etv7f$-berlin.de...
    > Tad McClellan wrote:
    > > Gunnar Hjalmarsson <> wrote:
    > >> Tad McClellan wrote:
    > >>> The documentation for HTTP::Request says to see also
    > >>> HTTP::Request::Common.
    > >>
    > >> Does it? I for one am not able to find that reference.

    > >
    > > http://search.cpan.org/~gaas/libwww-perl-5.76/lib/HTTP/Request.pm

    >
    > There it is, obviously... Thanks!
    >
    > To *not* find it, I searched CPAN for HTTP::Request and ended up at
    > http://search.cpan.org/~rse/lcwa-1.0.0/lib/lwp/lib/HTTP/Request.pm
    > -----------------------------------------------------^^^^^^^^^^^^
    >
    > I should probably have been more attentive, but...
    >
    > http://search.cpan.org/~rse/lcwa-1.0.0/ seems to be a 'historical
    > archive' of past versions of quite a few modules. :(
    >
    > Is that in accordance with the PAUSE rules?
    >
    > --
    > Gunnar Hjalmarsson
    > Email: http://www.gunnar.cc/cgi-bin/contact.pl
    >
    Dean Banko, Dec 29, 2003
    #9
  10. [ Please do not top post! ]

    Dean Banko wrote:
    > Gunnar Hjalmarsson wrote:
    >>
    >> my $req=HTTP::Request->new(POST=>$url);
    >> $req->content_type('application/x-www-form-urlencoded');
    >> $req->content('field=1&valid=yes');
    >>
    >> (untested)

    >
    > I thank you a lot for your assistance!
    > The syntax that you offered is functional, I've tested it and it
    > works!


    I'm glad it works, Dean.

    It should be noted that the field names and values in the example
    consist of letters and digits only. If you can't tell that that's
    always the case, I recommend that you take a look at
    HTTM::Request::Common as Tad suggested. That module takes care of URI
    escaping when needed. Otherwise, with my 'direct' method, you'd need
    to handle the URI escaping separately.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Hjalmarsson, Dec 29, 2003
    #10
  11. Dean Banko

    Peter Scott Guest

    In article <bsnnce$ekeav$-berlin.de>,
    Gunnar Hjalmarsson <> writes:
    >Dean Banko wrote:
    >> Does anyone know how to retreive a page generated as form request?
    >> I need to post some data like in the row:
    >> my $req=HTTP::Request->new(POST=>$url);
    >> but there are also some parameters that I must put into POST, but I
    >> don't know the syntax :-(
    >> I know that must be something like:
    >> my $req=HTTP::Request->new(POST=>$url, [field=>'1', valid=>'yes' );

    >
    >Assuming you are talking about two fields named 'field' and 'valid',
    >and whose values you want to be '1' respective 'yes', I'd try:
    >
    > my $req=HTTP::Request->new(POST=>$url);
    > $req->content_type('application/x-www-form-urlencoded');
    > $req->content('field=1&valid=yes');


    This is unnecessarily fraught with danger. It will fail the first time
    anyone tries to use values that ought to be URL escaped. And can you be
    certain without checking the source whether the Content-length header
    will be set correctly ? There is a clearly better alternative:

    use HTTP::Request::Common;
    my $req = POST $url, [ field=>'1', valid=>'yes' ];

    The OP was almost there.

    >(untested)


    No need to test.

    --
    Peter Scott
    http://www.perldebugged.com/
    *** NEW *** http//www.perlmedic.com/
    Peter Scott, Dec 29, 2003
    #11
  12. Peter Scott wrote:
    > Gunnar Hjalmarsson writes:
    >>
    >> my $req=HTTP::Request->new(POST=>$url);
    >> $req->content_type('application/x-www-form-urlencoded');
    >> $req->content('field=1&valid=yes');

    >
    > This is unnecessarily fraught with danger.


    Danger? In that case, the POD for LWP.pm is dangerous, because its
    first example includes just those lines.

    > It will fail the first time anyone tries to use values that ought
    > to be URL escaped.


    Yep, i.e. if you don't escape them first. I had already posted that
    warning.

    > And can you be certain without checking the source whether the
    > Content-length header will be set correctly ?


    Which source? I don't think it is set. Does it need to be? Again, see
    LWP.pm.

    I have at least noticed that the the remote server sets the
    content-length environment variable.

    > There is a clearly better alternative:
    >
    > use HTTP::Request::Common;
    > my $req = POST $url, [ field=>'1', valid=>'yes' ];


    Yes, it seems to be a more convenient way that I wasn't aware of
    before Tad mentioned it.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Hjalmarsson, Dec 30, 2003
    #12
  13. Dean Banko

    Peter Scott Guest

    In article <bsqn5g$eog5$-berlin.de>,
    Gunnar Hjalmarsson <> writes:
    >Peter Scott wrote:
    >> Gunnar Hjalmarsson writes:
    >>>
    >>> my $req=HTTP::Request->new(POST=>$url);
    >>> $req->content_type('application/x-www-form-urlencoded');
    >>> $req->content('field=1&valid=yes');

    >>
    >> This is unnecessarily fraught with danger.

    >
    >Danger? In that case, the POD for LWP.pm is dangerous, because its
    >first example includes just those lines.


    So it does. Perusal of Backpan reveals that those lines precede
    the introduction of HTTP::Request::Common. I shall submit a patch.
    Thank you for pointing them out.

    >> And can you be certain without checking the source whether the
    >> Content-length header will be set correctly ?

    >
    >Which source? I don't think it is set. Does it need to be?


    That's my point; you're left wondering.

    --
    Peter Scott
    http://www.perldebugged.com/
    *** NEW *** http//www.perlmedic.com/
    Peter Scott, Dec 30, 2003
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. wonder

    Retrieve current webpage url.

    wonder, Aug 18, 2004, in forum: Python
    Replies:
    0
    Views:
    302
    wonder
    Aug 18, 2004
  2. Replies:
    1
    Views:
    401
  3. mxywp
    Replies:
    1
    Views:
    285
    Paul Rubin
    Oct 22, 2006
  4. Vanessa
    Replies:
    5
    Views:
    269
    Vanessa
    Sep 22, 2005
  5. Arti Singh
    Replies:
    2
    Views:
    88
    Arti Singh
    Aug 12, 2009
Loading...

Share This Page