using LWP to POST data via "javascript:document.form.submit()" link

Discussion in 'Perl Misc' started by schruthensis, Jan 3, 2006.

  1. schruthensis

    schruthensis Guest

    Hello.

    Can anybody solve this problem?

    I need to POST data to a HTTPS login script that doesn't have a
    standard "submit" button. The form appears to use javascript to submit
    the document via the browser.

    <form action='loginScript.cgi' method="post" name="loginForm">
    <input name="userid" />
    <input name="password" />
    <a href="javascript:document.loginForm.submit()" >
    <img src="someImage.gif" >
    </a>
    </form>

    I have verified this by
    1) disabling javascript in my a browser and replicating the error of
    being unable to submit ... and ...
    2) creating my own simple form (roughly the same as the above code)
    on my own site to submit the data (to the site of interest) without
    problem!

    Any ideas on how to use perl to submit via this javascript dependent
    form?

    the only think I can think of is to have perl create and run a
    temporary javascript enabled form/webpage and capture the results...
    but this would be highly inelegant..

    Thanks in advance,

    Dave
    schruthensis, Jan 3, 2006
    #1
    1. Advertising

  2. schruthensis

    Eric Bohlman Guest

    "schruthensis" <> wrote in news:1136280041.510531.133830
    @o13g2000cwo.googlegroups.com:

    > I need to POST data to a HTTPS login script that doesn't have a
    > standard "submit" button. The form appears to use javascript to submit
    > the document via the browser.
    >
    > <form action='loginScript.cgi' method="post" name="loginForm">
    > <input name="userid" />
    > <input name="password" />
    > <a href="javascript:document.loginForm.submit()" >
    > <img src="someImage.gif" >
    > </a>
    > </form>


    The only thing Javascript is being used for here is to allow submitting the
    form by clicking a link image rather than a button. All the code in
    question does is a normal POST submit with items "userid" and "password,"
    and that's all you need to do with LWP; just treat it as if it were an
    ordinary form with a submit button.
    Eric Bohlman, Jan 3, 2006
    #2
    1. Advertising

  3. schruthensis

    John Bokma Guest

    Eric Bohlman <> wrote:

    > "schruthensis" <> wrote in
    > news:1136280041.510531.133830 @o13g2000cwo.googlegroups.com:
    >
    >> I need to POST data to a HTTPS login script that doesn't have a
    >> standard "submit" button. The form appears to use javascript to
    >> submit the document via the browser.
    >>
    >> <form action='loginScript.cgi' method="post" name="loginForm">
    >> <input name="userid" />
    >> <input name="password" />
    >> <a href="javascript:document.loginForm.submit()" >
    >> <img src="someImage.gif" >
    >> </a>
    >> </form>

    >
    > The only thing Javascript is being used for here is to allow
    > submitting the form by clicking a link image rather than a button.


    Which is odd, since this can be done without JavaScript if I am not
    mistaken :-D.

    --
    John Small Perl scripts: http://johnbokma.com/perl/
    Perl programmer available: http://castleamber.com/
    I ploink googlegroups.com :)
    John Bokma, Jan 3, 2006
    #3
  4. schruthensis <> wrote:

    > I need to POST data to a HTTPS login script that doesn't have a
    > standard "submit" button. The form appears to use javascript to submit
    > the document via the browser.



    > Any ideas on how to use perl to submit via this javascript dependent
    > form?



    Reverse engineer what the javascript does, or simply spy on
    the output from the javascript by using the

    Web Scraping Proxy

    http://www.research.att.com/~hpk/wsp/

    and then duplicate that output.


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Jan 3, 2006
    #4
  5. schruthensis

    schruthensis Guest

    John Bokma wrote:
    > Eric Bohlman <> wrote:


    > > The only thing Javascript is being used for here is to allow
    > > submitting the form by clicking a link image rather than a button.

    >
    > Which is odd, since this can be done without JavaScript if I am not
    > mistaken :-D.


    right, thats kinda the point. I have no problem with a standard
    "submit button" (no javascript) version of any form
    schruthensis, Jan 3, 2006
    #5
  6. schruthensis

    schruthensis Guest

    you are right! (ill explain below)
    schruthensis, Jan 3, 2006
    #6
  7. schruthensis

    schruthensis Guest

    schruthensis wrote:

    > I have verified this by
    > 1) disabling javascript in my a browser and replicating the error of
    > being unable to submit ... and ...
    > 2) creating my own simple form (roughly the same as the above code)
    > on my own site to submit the data (to the site of interest) without
    > problem!
    >
    > Any ideas on how to use perl to submit via this javascript dependent
    > form?
    >


    Well, looks like i didn't try an important permutation of submission
    possibilities:

    submitting from my own form with a submit button instead of the
    javascript. (which does work)...... ,however, this does NOT work if i
    disable javascript... which implies that somehow javascript is
    required AFTER the login at some point and perhaps that the javascript
    login is just a way to make sure only javascript enabled browsers
    enter....

    now, what do i do?

    I need to figure out how javascript is working after the login.... I
    would scrape, but doesn't that require that the communications are
    clear text? i have been sniffing the transactions but this is a
    secure, encrypted connection.. and i can't read anything..
    schruthensis, Jan 3, 2006
    #7
  8. schruthensis

    Matt Garrish Guest

    "schruthensis" <> wrote in message
    news:...
    > you are right! (ill explain below)
    >


    Who is right? Explain below what?

    I'm really starting to get a hate on for Google...

    Matt
    Matt Garrish, Jan 3, 2006
    #8
  9. schruthensis

    Samwyse Guest

    Re: using LWP to POST data via "javascript:document.form.submit()"link

    schruthensis wrote:

    > submitting from my own form with a submit button instead of the
    > javascript. (which does work)...... ,however, this does NOT work if i
    > disable javascript... which implies that somehow javascript is
    > required AFTER the login at some point and perhaps that the javascript
    > login is just a way to make sure only javascript enabled browsers
    > enter....


    I'll wager that it's setting a cookie. You should be able to look at
    the javascript and mentally execute it; I'd be surprised if it's more
    that a few lines long.
    Samwyse, Jan 5, 2006
    #9
  10. schruthensis

    schruthensis Guest

    Samwyse wrote:
    > schruthensis wrote:
    >
    >> which implies that somehow javascript is
    > > required AFTER the login at some point

    >
    > I'll wager that it's setting a cookie. You should be able to look at
    > the javascript and mentally execute it; I'd be surprised if it's more
    > that a few lines long.


    i think youre right, samwyse, the only javascript on the page (after
    loging in) thats any different from the previous pages is the following
    code snippet:

    <script>
    function getCookie(name)
    {
    //uses document.cookie.indexOf();
    }
    function setCookie(name, value, expires, path, domain, secure)
    {
    //uses something like:
    document.cookie=(name,value,expires,path,domain,secure)
    }
    setCookie("cookiename0","eKIQUeS7L3meAjSRyCChvstAojxX7q3GUduqdSxE9dhylKsekTtVTnOSwawkYsSQEZ7JvnqbfzmUCfeUk-HczPbJDwiXjkWwHhUYVHRnUxCEUKPQlV5BqA8XgE9OHg3A7AUhvVTHqu+Mp2gtbNKvLaIKd8Jb5Aznc5i6s-G2lv8n0nW-wpgl3nkTG59e3Uw5i4mDRhm28KGjh3r-shVXOUZgsbd+Wsrp+r6u+aUmX1TrsyA+52mC057ArY12i0JlfijUI4Fhcpnt-e0rdPAcD32xRAC1BMdAVuSOzW13RAFVLA6QFKm2WaqIX1XTHXv4wMKW7ELlw6B6T-q8VxpaGV2dlTSXXhNphVR73ZjWhXBPfg+b7jPjaIUttZhRRGKZdEeREeEX+YOdPZcXcOB2IFaNK6tKCLjP-LlLxn1ugzmDAEvxDJ63YD2YQ5UAtUBRJuI6ul-u38JRL2Qs7vZ5snVj3soyZhJv164z51mquua1szFR-t-C0IovEFgkFF2I22KGC3QE+A35NZBBusvVPttyUJksYlpqTagPkA5aLGSSLLJ017xICP8kAAeWD39AJA",
    "-1", "path", ".domain.com");
    setCookie("cookiename1","codenumber=i3NOHiv7DS5gS", "-1", "path",
    ".domain.com");
    setCookie("cookiename2","name=name", "expireTime", "path",
    ".domain.com");
    </script>

    I am thinking of using HTTP::cookies

    But what about these randomly generated cookie values?? (like:
    eKIQUeS7L3meAjS...etc) how am i supposed to reverse engineer that??
    Does HTTP::cookies allow values to be set from javascript? I am
    doubting it...

    Thanks again,
    d
    schruthensis, Jan 5, 2006
    #10
  11. schruthensis

    Samwyse Guest

    Re: using LWP to POST data via "javascript:document.form.submit()"link

    schruthensis wrote:

    > setCookie("cookiename0","eKIQUeS7L3meAjSRyCChvstAojxX7q3GUduqdSxE9dhylKsekTtVTnOSwawkYsSQEZ7JvnqbfzmUCfeUk-HczPbJDwiXjkWwHhUYVHRnUxCEUKPQlV5BqA8XgE9OHg3A7AUhvVTHqu+Mp2gtbNKvLaIKd8Jb5Aznc5i6s-G2lv8n0nW-wpgl3nkTG59e3Uw5i4mDRhm28KGjh3r-shVXOUZgsbd+Wsrp+r6u+aUmX1TrsyA+52mC057ArY12i0JlfijUI4Fhcpnt-e0rdPAcD32xRAC1BMdAVuSOzW13RAFVLA6QFKm2WaqIX1XTHXv4wMKW7ELlw6B6T-q8VxpaGV2dlTSXXhNphVR73ZjWhXBPfg+b7jPjaIUttZhRRGKZdEeREeEX+YOdPZcXcOB2IFaNK6tKCLjP-LlLxn1ugzmDAEvxDJ63YD2YQ5UAtUBRJuI6ul-u38JRL2Qs7vZ5snVj3soyZhJv164z51mquua1szFR-t-C0IovEFgkFF2I22KGC3QE+A35NZBBusvVPttyUJksYlpqTagPkA5aLGSSLLJ017xICP8kAAeWD39AJA",
    > "-1", "path", ".domain.com");
    > setCookie("cookiename1","codenumber=i3NOHiv7DS5gS", "-1", "path",
    > ".domain.com");
    > setCookie("cookiename2","name=name", "expireTime", "path",
    > ".domain.com");
    > </script>
    >
    > I am thinking of using HTTP::cookies
    >
    > But what about these randomly generated cookie values?? (like:
    > eKIQUeS7L3meAjS...etc) how am i supposed to reverse engineer that??
    > Does HTTP::cookies allow values to be set from javascript? I am
    > doubting it...


    I'd GET the initial page, then just scan the text for a regex:
    $regex = quotemeta('setCookie("X", "X", "X", "X", "X");');
    $regex =~ s/X/([^"]*)/g;
    $regex =~ s/ /\\s*/g;
    while (@args = ($page =~ /$regex/og)) { ... }

    Once you capture the cookies, feed the information to a Perl routine
    that you hand-translated from Javascript. Possibly you can get away
    with just calling HTTP::Cookies to create them.
    Samwyse, Jan 6, 2006
    #11
  12. schruthensis

    robic0 Guest

    On Fri, 06 Jan 2006 03:36:59 GMT, Samwyse <> wrote:

    >schruthensis wrote:
    >
    >> setCookie("cookiename0","eKIQUeS7L3meAjSRyCChvstAojxX7q3GUduqdSxE9dhylKsekTtVTnOSwawkYsSQEZ7JvnqbfzmUCfeUk-HczPbJDwiXjkWwHhUYVHRnUxCEUKPQlV5BqA8XgE9OHg3A7AUhvVTHqu+Mp2gtbNKvLaIKd8Jb5Aznc5i6s-G2lv8n0nW-wpgl3nkTG59e3Uw5i4mDRhm28KGjh3r-shVXOUZgsbd+Wsrp+r6u+aUmX1TrsyA+52mC057ArY12i0JlfijUI4Fhcpnt-e0rdPAcD32xRAC1BMdAVuSOzW13RAFVLA6QFKm2WaqIX1XTHXv4wMKW7ELlw6B6T-q8VxpaGV2dlTSXXhNphVR73ZjWhXBPfg+b7jPjaIUttZhRRGKZdEeREeEX+YOdPZcXcOB2IFaNK6tKCLjP-LlLxn1ugzmDAEvxDJ63YD2YQ5UAtUBRJuI6ul-u38JRL2Qs7vZ5snVj3soyZhJv164z51mquua1szFR-t-C0IovEFgkFF2I22KGC3QE+A35NZBBusvVPttyUJksYlpqTagPkA5aLGSSLLJ017xICP8kAAeWD39AJA",
    >> "-1", "path", ".domain.com");
    >> setCookie("cookiename1","codenumber=i3NOHiv7DS5gS", "-1", "path",
    >> ".domain.com");
    >> setCookie("cookiename2","name=name", "expireTime", "path",
    >> ".domain.com");
    >> </script>
    >>
    >> I am thinking of using HTTP::cookies
    >>
    >> But what about these randomly generated cookie values?? (like:
    >> eKIQUeS7L3meAjS...etc) how am i supposed to reverse engineer that??
    >> Does HTTP::cookies allow values to be set from javascript? I am
    >> doubting it...

    >
    >I'd GET the initial page, then just scan the text for a regex:
    > $regex = quotemeta('setCookie("X", "X", "X", "X", "X");');
    > $regex =~ s/X/([^"]*)/g;
    > $regex =~ s/ /\\s*/g;
    > while (@args = ($page =~ /$regex/og)) { ... }
    >
    >Once you capture the cookies, feed the information to a Perl routine
    >that you hand-translated from Javascript. Possibly you can get away
    >with just calling HTTP::Cookies to create them.


    You will never ever get sucess from obscure modules. You think cause
    its got a "::" in it its gonna be nice and save you from actually having
    to think. When it doesen't do for you what you want, your think revolves
    around workarounds and "if's" and trying to examine flawed code to
    comment 1 line so it will magically work.

    The energy and loss of time is unbelievable. Perl modules all just fucking
    suck. There's no other way to put it. If I'm totally off base and Cpan "::"
    is somehow God syntax, then by all means knock yourself out. I feel sorry
    for you.
    -robic0-
    robic0, Jan 6, 2006
    #12
  13. schruthensis

    schruthensis Guest

    Samwyse wrote:
    > schruthensis wrote:
    >
    > > But what about these randomly generated cookie values?? (like:
    > > eKIQUeS7L3meAjS...etc) how am i supposed to reverse engineer that??
    > > Does HTTP::cookies allow values to be set from javascript? I am
    > > doubting it...

    >
    > I'd GET the initial page, then just scan the text for a regex:
    > $regex = quotemeta('setCookie("X", "X", "X", "X", "X");');


    > Once you capture the cookies, feed the information to a Perl routine
    > that you hand-translated from Javascript. Possibly you can get away
    > with just calling HTTP::Cookies to create them.


    that would be nice if i could get the page at all in the first place...
    I am getting 302 (page moved) errors and no HTML to work with.
    The only thing i can think of is that there is communication in the
    pre-HTML headers that I am not properly handling.

    I finally got the ATT scraping proxy to work (not on windows....(a
    waste of an entire day), but via an x-window of mozilla from my linux
    server to my windows machine.... long story) and it seems to indicate
    that cookies are being used extensively and perhaps to help build the
    URL.. ...
    schruthensis, Jan 6, 2006
    #13
  14. schruthensis

    schruthensis Guest

    >>> schruthensis wrote:
    >>> I am thinking of using HTTP::cookies


    > robic0 wrote:
    > You will never ever get sucess from obscure modules. You think cause
    > its got a "::" in it its gonna be nice and save you from actually having
    > to think.


    That's funny, I've had success with just about every module I've
    ever tried. and yes, it has saved some brain time. If nothing else,
    I learn from looking at their code. What exactly defines an "obscure"
    module anyway? If you have a list of modules that you think I should
    steer clear of that might be nice, but blanket statements like these
    are hardly useful (except, perhaps to make you feel better about
    yourself)

    > robic0 wrote:
    > The energy and loss of time is unbelievable. Perl modules all just fucking
    > suck. There's no other way to put it.


    Okay, I can be a (coding) army of one. I'll just write all of this
    code from scratch. I was thinking of writing my own operating system
    while I'm at it.. and manufacturing my own mother boards and chips. I
    was thinking of doing it all blind folded and with one arm tied behind
    my back too. Afterwards I thought I might try and re-invent the wheel
    a couple of times.

    > If I'm totally off base and Cpan "::"
    > is somehow God syntax, then by all means knock yourself out. I feel sorry
    > for you.


    I'm not expecting miracles here, i was merely asking for input and
    suggestions on what I might try.... to see if someone has already
    wrestled with the problem a bit and bounce ideas off of them.

    If you are such a coding genius, perhaps you could do something useful
    like help me solve my problem or submit better modules to CPAN.
    Instead of verbally abusing a newbie (like me) or a particular faction
    of the community (like CPAN) to boost your own jaded ego try doing
    something that somebody appreciates.
    schruthensis, Jan 6, 2006
    #14
  15. schruthensis

    J. Gleixner Guest

    Re: using LWP to POST data via "javascript:document.form.submit()"link

    schruthensis wrote:

    > that would be nice if i could get the page at all in the first place...
    > I am getting 302 (page moved) errors and no HTML to work with.
    > The only thing i can think of is that there is communication in the
    > pre-HTML headers that I am not properly handling.


    LWP::UserAgent doesn't handle redirect for a POST, by default. See
    perldoc LWP::UserAgent, specifically the requests_redirectable method.

    I think I post this answer every other month. Maybe the docs should
    make this more prominent or be included in lwpcook.pod, but then
    someone would actually have to read them. :)
    J. Gleixner, Jan 6, 2006
    #15
  16. schruthensis

    schruthensis Guest

    J. Gleixner wrote:

    > LWP::UserAgent doesn't handle redirect for a POST, by default. See
    > perldoc LWP::UserAgent, specifically the requests_redirectable method.


    I already have that code in place, it just allows me to be redirected
    back to the main page. There is more than just redirection going on.
    schruthensis, Jan 6, 2006
    #16
  17. schruthensis

    Samwyse Guest

    Re: using LWP to POST data via "javascript:document.form.submit()"link

    schruthensis wrote:
    > J. Gleixner wrote:
    >
    >>LWP::UserAgent doesn't handle redirect for a POST, by default. See
    >>perldoc LWP::UserAgent, specifically the requests_redirectable method.

    >
    > I already have that code in place, it just allows me to be redirected
    > back to the main page. There is more than just redirection going on.


    Well, if all else fails, you can set up a proxy server and send all of
    your browser requests through it. Afterwards, check the log files for
    what was going on.

    P.S. Setting up a proxy server is outside the scope of this group.
    Samwyse, Jan 7, 2006
    #17
  18. schruthensis

    schruthensis Guest

    Samwyse wrote:
    > Well, if all else fails, you can set up a proxy server and send all of
    > your browser requests through it. Afterwards, check the log files for
    > what was going on.
    >
    > P.S. Setting up a proxy server is outside the scope of this group.


    I kind of already did that (as mentioned two posts ago). I did just
    as Tad McClellan (of the second response to my original posting)
    suggested to do: I downloaded, installed, and ran the ATT Web Scraping
    Proxy.

    I guess i just have to do the hard work of looking through all the
    headers, cookies and (sometimes purposefully obfuscated) JavaScript to
    figure this out. I was just wondering if anybody had seen this before,
    if I was missing something really obvious, or if some sites are just
    plain impossible to PERL into because of certain known limitations.

    Thanks for all the help anyway
    schruthensis, Jan 7, 2006
    #18
  19. schruthensis

    Matt Garrish Guest

    "schruthensis" <> wrote in message
    news:...
    >> robic0 wrote:

    >
    >> If I'm totally off base and Cpan "::"
    >> is somehow God syntax, then by all means knock yourself out. I feel sorry
    >> for you.

    >
    > If you are such a coding genius, perhaps you could do something useful
    > like help me solve my problem or submit better modules to CPAN.
    > Instead of verbally abusing a newbie (like me) or a particular faction
    > of the community (like CPAN) to boost your own jaded ego try doing
    > something that somebody appreciates.
    >


    Trust me, you *don't* want him to help you with your problem; it'll just put
    you in a worse position. I think the general perception is he's a bored kid
    who's acting out on usenet to get attention (classic troll). The more you
    give him that attention the more it keeps him coming back.

    Matt
    Matt Garrish, Jan 7, 2006
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Vishal
    Replies:
    1
    Views:
    649
    Steve C. Orr [MVP, MCSD]
    Dec 21, 2004
  2. Mark Mykkanen

    LWP - Upload via HTTP POST

    Mark Mykkanen, Jul 2, 2003, in forum: Perl Misc
    Replies:
    1
    Views:
    359
    J. Gleixner
    Jul 2, 2003
  3. Andrew
    Replies:
    1
    Views:
    339
    Gunnar Hjalmarsson
    Dec 5, 2003
  4. Zapanaz
    Replies:
    4
    Views:
    120
    Tad McClellan
    Oct 12, 2005
  5. Anon
    Replies:
    4
    Views:
    352
    schruthensis
    Jan 10, 2006
Loading...

Share This Page