Newbie Question: Best way to Extract Post Hashes?

Discussion in 'Perl Misc' started by Andrew, Nov 21, 2003.

  1. Andrew

    Andrew Guest

    Hello,

    I am a Perl newbie and I was wondering if anyone could tell me the
    best way to extract expected post keys and values from websites? is
    there an easy way to extract the expected post/value from a web site
    with the intention of using the information to write an agent or
    spider script for? Does anyone know of way to automatically gather it
    and post dynamically?


    Thanks!
     
    Andrew, Nov 21, 2003
    #1
    1. Advertising

  2. Andrew <> wrote:

    > I am a Perl newbie



    You are expected to check the Perl FAQs before posting to
    the Perl newsgroup.


    > and I was wondering if anyone could tell me the
    > best way to extract expected post keys and values from websites?



    Use the CGI module for that:

    perldoc CGI


    > is
    > there an easy way to extract the expected post/value from a web site
    > with the intention of using the information to write an agent or
    > spider script for?



    I'm afraid I cannot make sense of your question there. Sorry.


    > Does anyone know of way to automatically gather it
    > and post dynamically?



    perldoc -q CGI

    What is the correct form of response from a CGI script?

    How do I decode a CGI form?


    perldoc -q HTML

    How do I fetch an HTML file?

    How do I automate an HTML form submission?



    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Nov 21, 2003
    #2
    1. Advertising

  3. See the CGI group of modules.

    # perldoc CGI

    In general, the documentation which comes with a distribution is very good, and is accessible through the perldoc command.

    For more information:

    # perldoc perldoc

    --
    Cheers,
    Ben Liddicott

    "Andrew" <> wrote in message news:...
    > Hello,
    >
    > I am a Perl newbie and I was wondering if anyone could tell me the
    > best way to extract expected post keys and values from websites?
     
    Ben Liddicott, Nov 22, 2003
    #3
  4. [THIS is top posting - please DON'T do this]

    On Sat, 22 Nov 2003 10:34:27 +0000 (UTC)
    "Ben Liddicott" <> wrote:
    > See the CGI group of modules.
    >
    > # perldoc CGI


    This is a *single* module - not a "group". There are other modules
    that have the prefix "CGI", but this command will provide
    documentation for a *single* module.

    <snip>

    --
    Jim

    Copyright notice: all code written by the author in this post is
    released under the GPL. http://www.gnu.org/licenses/gpl.txt
    for more information.

    a fortune quote ...
    Those of you who think you know everything are very annoying to
    those of us who do.
     
    James Willmore, Nov 22, 2003
    #4
  5. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    (Andrew) wrote in news:9b7dcecd.0311211429.3c7a7735
    @posting.google.com:

    > Hello,
    >
    > I am a Perl newbie and I was wondering if anyone could tell me the
    > best way to extract expected post keys and values from websites? is
    > there an easy way to extract the expected post/value from a web site
    > with the intention of using the information to write an agent or
    > spider script for? Does anyone know of way to automatically gather it
    > and post dynamically?


    I have been a web programmer for eight years, and I have no clue what you
    mean by "expected post keys and values". Can you give some more details
    about what you are trying to do?

    - --
    Eric
    $_ = reverse sort $ /. r , qw p ekca lre uJ reh
    ts p , map $ _. $ " , qw e p h tona e and print

    -----BEGIN PGP SIGNATURE-----
    Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

    iQA/AwUBP79Wl2PeouIeTNHoEQK1hACfX+Z6OXkItewefV39bPOaMIBAAj4AoMP+
    iKKbBa3dkinz64meSFD/DG6c
    =Xn84
    -----END PGP SIGNATURE-----
     
    Eric J. Roode, Nov 22, 2003
    #5
  6. Andrew

    Juha Laiho Guest

    "Eric J. Roode" <> said:
    > (Andrew) wrote in news::
    >> I am a Perl newbie and I was wondering if anyone could tell me the
    >> best way to extract expected post keys and values from websites?

    [...]
    >I have been a web programmer for eight years, and I have no clue what you
    >mean by "expected post keys and values". Can you give some more details
    >about what you are trying to do?


    For some reason his wording made slight problems for me as well, but
    what he apparently wants to write is software that:
    - downloads a HTML form
    - parses the form to determine which fill-in elements it contains
    - somehow generate data for those elements
    - submits the filled-in form appropriately

    Now, the hard parts here are
    - to determine what data is requested by the form
    - what are the allowable values for free-text fields
    - what all can be filled in to the form, if it happens to be something
    that is partially generated by client-side technologies while the user
    is filling it
    --
    Wolf a.k.a. Juha Laiho Espoo, Finland
    (GC 3.0) GIT d- s+: a C++ ULSH++++$ P++@ L+++ E- W+$@ N++ !K w !O !M V
    PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
    "...cancel my subscription to the resurrection!" (Jim Morrison)
     
    Juha Laiho, Nov 23, 2003
    #6
  7. Andrew

    Andrew Guest

    Juha Laiho <> wrote in message news:<bpr0ri$k4v$-int>...
    > "Eric J. Roode" <> said:
    > > (Andrew) wrote in news::
    > >> I am a Perl newbie and I was wondering if anyone could tell me the
    > >> best way to extract expected post keys and values from websites?

    > [...]
    > >I have been a web programmer for eight years, and I have no clue what you
    > >mean by "expected post keys and values". Can you give some more details
    > >about what you are trying to do?

    >
    > For some reason his wording made slight problems for me as well, but
    > what he apparently wants to write is software that:
    > - downloads a HTML form
    > - parses the form to determine which fill-in elements it contains
    > - somehow generate data for those elements
    > - submits the filled-in form appropriately
    >
    > Now, the hard parts here are
    > - to determine what data is requested by the form
    > - what are the allowable values for free-text fields
    > - what all can be filled in to the form, if it happens to be something
    > that is partially generated by client-side technologies while the user
    > is filling it


    Hello All,

    Sorry about the confusion in my wording. What I was looking for tips
    on how to automatically detect all necessary post values the form is
    expecting. For example, When I look at the source code for
    my.yahoo.com, there are several hidden attributes that it is
    requesting aside form the obvious username and password fields. I was
    wondering if anyone knew of a module that could help identify these
    hidden values.

    Thanks for the idea of checking out the CGI module. I havent used
    that module yet and I am unaware of what it contains. I will check
    out the perldocs on that.
     
    Andrew, Nov 24, 2003
    #7
  8. -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    (Andrew) wrote in
    news::

    > Sorry about the confusion in my wording. What I was looking for tips
    > on how to automatically detect all necessary post values the form is
    > expecting. For example, When I look at the source code for
    > my.yahoo.com, there are several hidden attributes that it is
    > requesting aside form the obvious username and password fields. I was
    > wondering if anyone knew of a module that could help identify these
    > hidden values.


    Ah. Sounds like you need a module for parsing HTML, such as
    HTML::TokeParser.

    - --
    Eric
    $_ = reverse sort $ /. r , qw p ekca lre uJ reh
    ts p , map $ _. $ " , qw e p h tona e and print

    -----BEGIN PGP SIGNATURE-----
    Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

    iQA/AwUBP8H3sGPeouIeTNHoEQIJpQCg2aK/tSBPzeK0N8+uPrMaaElrPdkAoK+b
    wWeoC/ZiZYUbNottDhOoNuli
    =Eqi2
    -----END PGP SIGNATURE-----
     
    Eric J. Roode, Nov 24, 2003
    #8
  9. Eric J. Roode <> wrote:
    > -----BEGIN PGP SIGNED MESSAGE-----
    > Hash: SHA1


    > (Andrew) wrote in
    > news::


    >> Sorry about the confusion in my wording. What I was looking for tips
    >> on how to automatically detect all necessary post values the form is
    >> expecting. For example, When I look at the source code for
    >> my.yahoo.com, there are several hidden attributes that it is
    >> requesting aside form the obvious username and password fields. I was
    >> wondering if anyone knew of a module that could help identify these
    >> hidden values.


    > Ah. Sounds like you need a module for parsing HTML, such as
    > HTML::TokeParser.


    Or a way to submit forms WWW::Mechanise

    CLayton
     
    Clayton L. Scott, Nov 26, 2003
    #9
  10. Andrew

    kokit Guest

    Juha Laiho wrote:
    > "Eric J. Roode" <> said:
    >
    >> (Andrew) wrote in news::
    >>
    >>>I am a Perl newbie and I was wondering if anyone could tell me the
    >>>best way to extract expected post keys and values from websites?

    >
    > [...]
    >
    >>I have been a web programmer for eight years, and I have no clue what you
    >>mean by "expected post keys and values". Can you give some more details
    >>about what you are trying to do?

    >
    >
    > For some reason his wording made slight problems for me as well, but
    > what he apparently wants to write is software that:
    > - downloads a HTML form
    > - parses the form to determine which fill-in elements it contains
    > - somehow generate data for those elements
    > - submits the filled-in form appropriately
    >
    > Now, the hard parts here are
    > - to determine what data is requested by the form
    > - what are the allowable values for free-text fields
    > - what all can be filled in to the form, if it happens to be something
    > that is partially generated by client-side technologies while the user
    > is filling it


    use CGI.pm module
     
    kokit, Jan 16, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ben Holness

    Hashes of Hashes via subs

    Ben Holness, Oct 5, 2003, in forum: Perl
    Replies:
    8
    Views:
    593
    Ben Holness
    Oct 8, 2003
  2. Steven Arnold

    using hashes as keys in hashes

    Steven Arnold, Nov 23, 2005, in forum: Ruby
    Replies:
    3
    Views:
    186
    Mauricio Fernández
    Nov 23, 2005
  3. djacober

    Question about hashes of hashes

    djacober, Jul 24, 2005, in forum: Perl Misc
    Replies:
    10
    Views:
    204
    Ian Wilson
    Jul 25, 2005
  4. Tim O'Donovan

    Hash of hashes, of hashes, of arrays of hashes

    Tim O'Donovan, Oct 27, 2005, in forum: Perl Misc
    Replies:
    5
    Views:
    237
  5. mrstevegross
    Replies:
    3
    Views:
    152
    Tad J McClellan
    Nov 11, 2008
Loading...

Share This Page