how to use POST to call cgi from Perl

Discussion in 'Perl Misc' started by michal.shmueli@gmail.com, May 3, 2005.

  1. Guest

    i'm trying to use agent to retrieve data from some websites. the
    website is using POST method. so i tried to follow some examples i
    found on the web...

    Eventually, i did the follow, but i'm getting error that says that it
    couldn't fetch any data....
    any idea?

    #!/usr/bin/perl
    use strict;
    use warnings;
    use LWP::UserAgent;
    my $browser = LWP::UserAgent->new;

    $browser->env_proxy;

    my $url = 'http://www.carprices.com/cgi-bin/n_page3.cgi';
    my $response = $browser->post( $url,
    [ "vehicle_type" =>"new,used",
    "conf_id" =>"carprices",
    "onpage" =>"1",
    "frompage" =>"basicsearch",
    "bodystyle_buff" =>
    "convertible,coupe,hatchback,pickup,sedan,suv,van,wagon",
    "price_min_buff" => "1050",
    "price_max_buff" => "149990",
    "year_min_buff" => "1986",
    "year_max_buff" => "2005" ]
    );

    thanks a lot,
    Michal
    , May 3, 2005
    #1
    1. Advertising

  2. wrote in news:1115082439.407836.128200
    @f14g2000cwb.googlegroups.com:

    > Eventually, i did the follow, but i'm getting error that says that it
    > couldn't fetch any data....
    > any idea?


    ....

    > my $url = 'http://www.carprices.com/cgi-bin/n_page3.cgi';


    What is the URL for the form?

    Sinan
    --
    A. Sinan Unur <>
    (reverse each component and remove .invalid for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
    A. Sinan Unur, May 3, 2005
    #2
    1. Advertising

  3. Guest

    , May 3, 2005
    #3
  4. wrote in news:1115087231.275936.34460
    @g14g2000cwa.googlegroups.com:

    [ please provide some context when replying ]

    > it the form of "used cars":
    > http://inventory.carprices.com/basicsearch.html


    I don't think so. In your original post, you had a variable for body
    type, which clearly is not an option with the basic search. So, I guess
    you are using the form at

    http://inventory.carprices.com/advanced.html

    Posting that form, and examining the traffic shows some tracking
    happening, and some cookies being set. There is also some Javascript
    involved in form submission.

    You might want to Google for "web scraping proxy", and use that tool to
    figure out exactly what information is being exchanged.

    You might also want to look at WWW::Mechanize. I don't think it is going
    to be of much help in this case, as the form submission goes through
    Javascript, and the site does not provide an alternative when Javascript
    is not enabled.

    To sum up, I do not have a quick solution. You are going to need to look
    at the Javascript to see what is being done with cookies, and form
    parameters, and possibly inspect the traffic to figure out exactly what
    information the server expects, and submit that.

    Sinan

    --
    A. Sinan Unur <>
    (reverse each component and remove .invalid for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
    A. Sinan Unur, May 3, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Wet Basement
    Replies:
    1
    Views:
    2,501
    BobMonk
    Jul 15, 2003
  2. Jürgen Exner

    Re: CGI Perl "use CGI" statement fail

    Jürgen Exner, Jul 31, 2003, in forum: Perl
    Replies:
    0
    Views:
    1,223
    Jürgen Exner
    Jul 31, 2003
  3. Ken Smith
    Replies:
    9
    Views:
    249
    Ken Smith
    Jul 30, 2004
  4. Guy
    Replies:
    4
    Views:
    761
    Malcolm Dew-Jones
    Dec 11, 2003
  5. kath
    Replies:
    4
    Views:
    533
    J. Gleixner
    Apr 9, 2007
Loading...

Share This Page