Newbie Question: Best way to Extract Post Hashes?

[Andrew]

Hello,

I am a Perl newbie and I was wondering if anyone could tell me the
best way to extract expected post keys and values from websites? is
there an easy way to extract the expected post/value from a web site
with the intention of using the information to write an agent or
spider script for? Does anyone know of way to automatically gather it
and post dynamically?


Thanks!

 

[Tad McClellan]

I am a Perl newbie

You are expected to check the Perl FAQs before posting to
the Perl newsgroup.

and I was wondering if anyone could tell me the
best way to extract expected post keys and values from websites?

Use the CGI module for that:

perldoc CGI

is
there an easy way to extract the expected post/value from a web site
with the intention of using the information to write an agent or
spider script for?

I'm afraid I cannot make sense of your question there. Sorry.

Does anyone know of way to automatically gather it
and post dynamically?

perldoc -q CGI

What is the correct form of response from a CGI script?

How do I decode a CGI form?


perldoc -q HTML

How do I fetch an HTML file?

How do I automate an HTML form submission?

 

[Ben Liddicott]

See the CGI group of modules.

# perldoc CGI

In general, the documentation which comes with a distribution is very good, and is accessible through the perldoc command.

For more information:

# perldoc perldoc

 

[James Willmore]

[THIS is top posting - please DON'T do this]

See the CGI group of modules.

# perldoc CGI

This is a *single* module - not a "group". There are other modules
that have the prefix "CGI", but this command will provide
documentation for a *single* module.

<snip>

--
Jim

Copyright notice: all code written by the author in this post is
released under the GPL. http://www.gnu.org/licenses/gpl.txt
for more information.

a fortune quote ...
Those of you who think you know everything are very annoying to
those of us who do.

 

[Eric J. Roode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(e-mail address removed) (Andrew) wrote in @posting.google.com:

Hello,

I am a Perl newbie and I was wondering if anyone could tell me the
best way to extract expected post keys and values from websites? is
there an easy way to extract the expected post/value from a web site
with the intention of using the information to write an agent or
spider script for? Does anyone know of way to automatically gather it
and post dynamically?

I have been a web programmer for eight years, and I have no clue what you
mean by "expected post keys and values". Can you give some more details
about what you are trying to do?

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBP79Wl2PeouIeTNHoEQK1hACfX+Z6OXkItewefV39bPOaMIBAAj4AoMP+
iKKbBa3dkinz64meSFD/DG6c
=Xn84
-----END PGP SIGNATURE-----

 

[Juha Laiho]

(e-mail address removed) (Andrew) wrote in news:[email protected]:

I am a Perl newbie and I was wondering if anyone could tell me the
best way to extract expected post keys and values from websites?

[...]
I have been a web programmer for eight years, and I have no clue what you
mean by "expected post keys and values". Can you give some more details
about what you are trying to do?

For some reason his wording made slight problems for me as well, but
what he apparently wants to write is software that:
- downloads a HTML form
- parses the form to determine which fill-in elements it contains
- somehow generate data for those elements
- submits the filled-in form appropriately

Now, the hard parts here are
- to determine what data is requested by the form
- what are the allowable values for free-text fields
- what all can be filled in to the form, if it happens to be something
that is partially generated by client-side technologies while the user
is filling it

 

[Andrew]

(e-mail address removed) (Andrew) wrote in news:[email protected]:

I am a Perl newbie and I was wondering if anyone could tell me the
best way to extract expected post keys and values from websites?

[...]
I have been a web programmer for eight years, and I have no clue what you
mean by "expected post keys and values". Can you give some more details
about what you are trying to do?

For some reason his wording made slight problems for me as well, but
what he apparently wants to write is software that:
- downloads a HTML form
- parses the form to determine which fill-in elements it contains
- somehow generate data for those elements
- submits the filled-in form appropriately

Now, the hard parts here are
- to determine what data is requested by the form
- what are the allowable values for free-text fields
- what all can be filled in to the form, if it happens to be something
that is partially generated by client-side technologies while the user
is filling it

Hello All,

Sorry about the confusion in my wording. What I was looking for tips
on how to automatically detect all necessary post values the form is
expecting. For example, When I look at the source code for
my.yahoo.com, there are several hidden attributes that it is
requesting aside form the obvious username and password fields. I was
wondering if anyone knew of a module that could help identify these
hidden values.

Thanks for the idea of checking out the CGI module. I havent used
that module yet and I am unaware of what it contains. I will check
out the perldocs on that.

 

[Eric J. Roode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(e-mail address removed) (Andrew) wrote in

Sorry about the confusion in my wording. What I was looking for tips
on how to automatically detect all necessary post values the form is
expecting. For example, When I look at the source code for
my.yahoo.com, there are several hidden attributes that it is
requesting aside form the obvious username and password fields. I was
wondering if anyone knew of a module that could help identify these
hidden values.

Ah. Sounds like you need a module for parsing HTML, such as
HTML::TokeParser.

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBP8H3sGPeouIeTNHoEQIJpQCg2aK/tSBPzeK0N8+uPrMaaElrPdkAoK+b
wWeoC/ZiZYUbNottDhOoNuli
=Eqi2
-----END PGP SIGNATURE-----

 

[Clayton L. Scott]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(e-mail address removed) (Andrew) wrote in

Ah. Sounds like you need a module for parsing HTML, such as
HTML::TokeParser.

Or a way to submit forms WWW::Mechanise

CLayton

 

[kokit]

(e-mail address removed) (Andrew) wrote in

I am a Perl newbie and I was wondering if anyone could tell me the
best way to extract expected post keys and values from websites?

[...]

I have been a web programmer for eight years, and I have no clue what you
mean by "expected post keys and values". Can you give some more details
about what you are trying to do?

For some reason his wording made slight problems for me as well, but
what he apparently wants to write is software that:
- downloads a HTML form
- parses the form to determine which fill-in elements it contains
- somehow generate data for those elements
- submits the filled-in form appropriately

Now, the hard parts here are
- to determine what data is requested by the form
- what are the allowable values for free-text fields
- what all can be filled in to the form, if it happens to be something
that is partially generated by client-side technologies while the user
is filling it

use CGI.pm module