Script to insert data to a web page?

[Chad]

Hi there,
I am a Perl novice and am looking for a simple script to access a web
page, enter some data, and examine the response.

Does anyone have anything like this?

 

[Tim Greer]

Hi there,
I am a Perl novice and am looking for a simple script to access a web
page, enter some data, and examine the response.

Does anyone have anything like this?

There are a lot of scripts that accept data entered and then respond
accordingly. I'd wager that a high majority of scripts for the web are
designed for that specific task. What exactly are you looking to
do/have passed to it and process (and display a result, I assume)?
There are likely existing scripts to do exactly what you want.

 

[Chad]

There are a lot of scripts that accept data entered and then respond
accordingly.  I'd wager that a high majority of scripts for the web are
designed for that specific task.  What exactly are you looking to
do/have passed to it and process (and display a result, I assume)?
There are likely existing scripts to do exactly what you want.
--
Tim Greer, CEO/Founder/CTO, BurlyHost.com, Inc.
Shared Hosting, Reseller Hosting, Dedicated & Semi-Dedicated servers
and Custom Hosting.  24/7 support, 30 day guarantee, secure servers.
Industry's most experienced staff! -- Web Hosting With Muscle!

Hi there,
Wow, thanks for the quick response. I have a large list of objects
that I want to enter on a web page to see if they exist. So the basic
flow would be read file, send object name to the web page, hit search,
return the result.

 

[Vicky Conlan]

(snip)
Wow, thanks for the quick response. I have a large list of objects
that I want to enter on a web page to see if they exist. So the basic
flow would be read file, send object name to the web page, hit search,
return the result.

Sorry, just to get confirmation, cos I think I'm unsure what you're looking
for - from your first post I thought you wanted something which would scan
a web page with a form in it, then fill in the form fields and submit; from
your second it sounded more like downloading the source and doing some kind
of text search on it.

(apologies if this sounds a bit of a daft question, my brain is still on
holiday atm

--

 

[News123]

Hi,


This can be anything from quite trivial to quite annoying, depending on
the web server and the amount of Cookies, Javascript, Flash Plugins,
Java Applets being sued by your page.


You should probably look at following modules
- LWP::UserAgent # to access the page, enter the data and post/get it
- WWW::Mechanize # sub class of LWP::UserAgent, probably simpler,
# but I never used ut

- HTML:arser # to examine the response. If the response is very simple
you could also just use a regular expression.

- HTTP::Cookies # Only needed if browsing the page doesn't work without
cookies



Please note, that these modules are not necessarily installed on your PC,


To find out whether a module exists, you coulcd check,
You can try to executa an empty perl script, which uses this module:
perl -MLWP::UserAgent -e ''

or you try to read the module's documentation:
perldoc LWP::UserAgent # if there's no doc the module doesn't exist

Installing these modules depends on your OS / perl distribution


For examples you can look at:

http://search.cpan.org/~gaas/libwww-perl-5.822/lib/LWP/UserAgent.pm
http://search.cpan.org/~petdance/WWW-Mechanize-1.52/lib/WWW/Mechanize.pm




bye


N

 

[Chad]

Sorry, just to get confirmation, cos I think I'm unsure what you're looking
for - from your first post I thought you wanted something which would scan
a web page with a form in it, then fill in the form fields and submit;  from
your second it sounded more like downloading the source and doing some kind
of text search on it.

(apologies if this sounds a bit of a daft question, my brain is still on
holiday atm

--

Hi again,
I will read a value from an input file, input that to the web page,
and examine the result from the web.

 

[Chad]

Hi,

This can be anything from quite trivial to quite annoying, depending on
the web server and the amount of Cookies, Javascript, Flash Plugins,
Java Applets being sued by your page.

You should probably look at following modules
- LWP::UserAgent # to access the page, enter the data and post/get it
- WWW::Mechanize # sub class of LWP::UserAgent, probably simpler,
                 # but I never used ut

- HTML:arser # to examine the response. If the response is very simple
you could also just use a regular expression.

- HTTP::Cookies # Only needed if browsing the page doesn't work without
cookies

Please note, that these modules are not necessarily installed on your PC,

To find out whether a module exists, you coulcd check,
You can try to executa an empty perl script, which uses this module:
 perl -MLWP::UserAgent -e ''

or you try to read the module's documentation:
perldoc LWP::UserAgent # if there's no doc the module doesn't exist

Installing these modules depends on your OS / perl distribution

For examples you can look at:

http://search.cpan.org/~gaas/libwww...dance/WWW-Mechanize-1.52/lib/WWW/Mechanize.pm

bye

N





- Show quoted text -

Thanks again. I have been trying that but the documentation isn't the
best. I am looking for a little better working example to play with.

 

[Gunnar Hjalmarsson]

Hi there,
I am a Perl novice

http://learn.perl.org/

 

[Tad J McClellan]

I am a Perl novice and am looking for a simple script to access a web

^^^^^^^^^^^

If you are instead "looking to write" such a script, then this
is the right place to ask.

page, enter some data, and examine the response.

use WWW::Mechanize

 

[Henry Law]

Hi there,
I am a Perl novice and am looking for a simple script to access a web
page, enter some data, and examine the response.

Does anyone have anything like this?

I've got just the resource for you: http://tinyurl.com/a28wgc

 

[News123]

.. . .

Thanks again. I have been trying that but the documentation isn't the
best. I am looking for a little better working example to play with.

As mentioned before web pages can be very different.
This means in other words, that nobody can really help you if you don't
give us more information.

Where do you fail exactly?
- Fetching the url?
- sending the post / get request with the valeus you filled in?
- analyzing the result ?

It might be useful if you show what you have already written (or tried
to write) so far and if you gave a concrete example.
( url to fetch, values to fill in, etc.)


bye


N

 

[Chad]

. . .


As mentioned before web pages can be very different.
This means in other words, that nobody can really help you if you don't
give us more information.

Where do you fail exactly?
- Fetching  the url?
- sending the post / get request with the valeus you filled in?
- analyzing the result ?

It might be useful if you show what you have already written (or tried
to write) so far and if you gave a concrete example.
( url to fetch, values to fill in, etc.)

bye

N- Hide quoted text -

- Show quoted text -

I can get the script to work going against sometrhing like google. My
problem is assessing an internal company web page. I have security to
it through IE so I don't know what's wrong. Here is the code:

#!c:\\perl\\bin
use strict;
use LWP;
use LWP::UserAgent;
use WWW::Mechanize;
use HTTP::Cookies;


my $url = "http://compassapps.amfam.com/isdoc/tab_function.asp?
CALLING_ASP=Objects&DOM_ID=48&FUNCTION_ID=6572";
my $username = "<my userid>";
my $password = "<my password>";

my $searchstring = "Perl xml";
my $outfile = "out.htm";
my $mech = WWW::Mechanize->new();
$mech->cookie_jar(HTTP::Cookies->new());
$mech->get($url,$username, $password);
$mech->form_name('vForm');
$mech->field(txtSearchString => $searchstring);
$mech->field(login => $username);
$mech->field(passwd => $password);
##$mech->field(my $query => "$searchstring");
$mech->click();
my $output_page = $mech->content();
open(OUTFILE, ">$outfile");
print OUTFILE "$output_page";
close(OUTFILE);

 

[J. Gleixner]

Chad wrote:
[...]

I can get the script to work going against sometrhing like google. My
problem is assessing an internal company web page. I have security to
it through IE so I don't know what's wrong. Here is the code:

#!c:\\perl\\bin
use strict;
use LWP;
use LWP::UserAgent;
use WWW::Mechanize;
use HTTP::Cookies;


my $url = "http://compassapps.amfam.com/isdoc/tab_function.asp?
CALLING_ASP=Objects&DOM_ID=48&FUNCTION_ID=6572";
my $username = "<my userid>";
my $password = "<my password>";

my $searchstring = "Perl xml";
my $outfile = "out.htm";
my $mech = WWW::Mechanize->new();
$mech->cookie_jar(HTTP::Cookies->new());
$mech->get($url,$username, $password);
$mech->form_name('vForm');
$mech->field(txtSearchString => $searchstring);
$mech->field(login => $username);
$mech->field(passwd => $password);
##$mech->field(my $query => "$searchstring");
$mech->click();
my $output_page = $mech->content();
open(OUTFILE, ">$outfile");
print OUTFILE "$output_page";
close(OUTFILE);

Do some error checking!

We have no idea nor can anyone guess how your internal page works.

Is the get() successful?
Is there a form named 'vForm'?
Does it have a input fields for login, passwd, and txtSearchString?
etc.

I'd start with the get(). Check the content() or success(),
to ensure that's correct. I'm pretty sure you'll need to
read up on how to do Basic authentication, using WWW::Mechanize.