Pressing A Webpage Button

[Elliot Temple]

How do I make Python press a button on a webpage? I looked at
urllib, but I only see how to open a URL with that. I searched
google but no luck.

For example, google has a button <input type=submit value="Google
Search" name=btnG> how would i make a script to press that button?

Just for fun, is there any way to do the equivalent of typing into a
text field like the google search field before hitting the button?
(I don't actually need to do this.)

If someone could point me in the right direction it'd be appreciated.

-- Elliot Temple
http://www.curi.us/

 

[Brian Beck]

How do I make Python press a button on a webpage? I looked at
urllib, but I only see how to open a URL with that. I searched
google but no luck.

Check out mechanize: http://wwwsearch.sourceforge.net/mechanize/

 

[Steve M]

Do you actually need to 'press' the button? Or do you just need the
effect that pressing the button would bring about (e.g., submitting a
Google search query and receiving the results page)?

If the latter, then you might want to search for, e.g., "html form get
post" and check out some results. Pushing the button is often just
loading a URL with parameters.

For example, go to Google and type "html form get post" into the search
box and press Submit. Now look at the URL you are visiting in your
location bar, the URL of the search results. It will be something like:

http://www.google.com/search?hl=en&q=html+form+get+post&btnG=Google+Search

If you were to load that URL directly (without having gone to the
Google homepage, typed "html form get post" in the text entry box and
pressed submit) the exact same effect would happen. Filling in the box
and clicking the submit button is just the user-friendly way of
constructing that URL.

 

[J Correia]

How do I make Python press a button on a webpage? I looked at
urllib, but I only see how to open a URL with that. I searched
google but no luck.

For example, google has a button <input type=submit value="Google
Search" name=btnG> how would i make a script to press that button?

Just for fun, is there any way to do the equivalent of typing into a
text field like the google search field before hitting the button?
(I don't actually need to do this.)

You don't say which OS... if you're running IE on Windows you
can use COM as follows...

from win32com.client import Dispatch
from time import sleep

ie = Dispatch("InternetExplorer.Application")
ie.Visible = 1
ie.Navigate("http://www.google.com")
while ie.ReadyState != 4: # Wait for browser to finish loading.
sleep(1)
doc = ie.Document
doc.f.q.value = "qwerty" # form name is 'f'; search field name is 'q'
doc.f.btnG.click() # click on button 'btnG'

 

[Tim Roberts]

How do I make Python press a button on a webpage? I looked at
urllib, but I only see how to open a URL with that. I searched
google but no luck.

For example, google has a button <input type=submit value="Google
Search" name=btnG> how would i make a script to press that button?

Just for fun, is there any way to do the equivalent of typing into a
text field like the google search field before hitting the button?
(I don't actually need to do this.)

Both things are done the same way. The Google Search button is a field
named "btnG" with the value "Google Search". The query field itself is
named "q". Then, if you look at the HTML, you'll see that this is wrapped
in a <form> with the action "/search".

So, all you need to do, then, is to encode all this in a URL:

http://www.google.com/search?btnG=Google Search&q=python urllib

This only works because the Google "search" page accepts parameters using
the "GET" method, which is what you get when you send parameters in the
URL. Many forms only accept parameters using the "POST" method, in which
you send the encoded parameters as the body of the HTTP request.

You probably need to do some reading on HTTP, and the GET and POST methods
of transmitting parameters.

 

[John J. Lee]

How do I make Python press a button on a webpage? I looked at
urllib, but I only see how to open a URL with that. I searched
google but no luck.

[...]

You might find the FAQ list and hints below useful after you get over
the initial barriers (and the modules at the same site):

http://wwwsearch.sourceforge.net/bits/GeneralFAQ.html

http://wwwsearch.sourceforge.net/ClientCookie/doc.html#debugging


John

 

[jerky]

hi,
urllib does not work when search from google.com,
since google.com have some limitations to developer ,
you can get more details from www.google.com/apis/

 

[Grant Edwards]

How do I make Python press a button on a webpage?

You just do whatever action is specified for the form
containing the button.

I looked at urllib, but I only see how to open a URL with
that.

Guess what happens when you push that button: the browser
opens a URL.

I searched google but no luck.

For example, google has a button <input type=submit value="Google
Search" name=btnG> how would i make a script to press that button?

Find the <form> containing the button, and look to see what the
URL is specified. For Google, it looks something like this:

<form action="/search" naem="f">

So, /search is the URL you open.

Just for fun, is there any way to do the equivalent of typing
into a text field like the google search field before hitting
the button? (I don't actually need to do this.)

Sure. Just send back the field value in the normal manner
using a GET.

If someone could point me in the right direction it'd be appreciated.

You need an introductory book on HTTP and HTML.

If all you care about is a google query here's a python program
that prints the URL you need to open for a google query:

#!/usr/bin/python
import urllib,sys,os
queryString="whatever you're searching for"
print 'http://www.google.com/search?'+urllib.urlencode({'q':queryString})

I presume you can figure out how to open the URL instead of
printing it?

 

[Grant Edwards]

You just do whatever action is specified for the form
containing the button.


Guess what happens when you push that button: the browser
opens a URL.


Find the <form> containing the button, and look to see what the
URL is specified. For Google, it looks something like this:

<form action="/search" naem="f">

So, /search is the URL you open.


Sure. Just send back the field value in the normal manner
using a GET.


You need an introductory book on HTTP and HTML.

If all you care about is a google query here's a python program
that prints the URL you need to open for a google query:

#!/usr/bin/python
import urllib,sys,os
queryString="whatever you're searching for"
print 'http://www.google.com/search?'+urllib.urlencode({'q':queryString})

I presume you can figure out how to open the URL instead of
printing it?

Ah, never mind. That doesn't work. Google somehow detects
you're not sending the query from a browser and bonks you.

 

[Paul Rubin]

Ah, never mind. That doesn't work. Google somehow detects
you're not sending the query from a browser and bonks you.

Try setting the User-agent: header to one that looks like a browser.

 

[Esben Pedersen]

You don't say which OS... if you're running IE on Windows you
can use COM as follows...

from win32com.client import Dispatch
from time import sleep

ie = Dispatch("InternetExplorer.Application")
ie.Visible = 1
ie.Navigate("http://www.google.com")
while ie.ReadyState != 4: # Wait for browser to finish loading.
sleep(1)
doc = ie.Document
doc.f.q.value = "qwerty" # form name is 'f'; search field name is 'q'
doc.f.btnG.click() # click on button 'btnG'

How do i know which methods the ie object has? dir(ie) doesn't show
Navigate.

/Esben

 

[J Correia]

How do i know which methods the ie object has? dir(ie) doesn't show
Navigate.

For ie object:
http://msdn.microsoft.com/workshop/browser/webbrowser/reference/ifaces/IWebBrowser2/IWebBrowser2.asp

For document object:
http://msdn.microsoft.com/workshop/author/dhtml/reference/objects/obj_document.asp

 

[calfdog]

You also may want to fire an event when a button is pressed such as:

ie.Document.forms['f'].elements['btnG'].FireEvent('onClick')


Events:
onclick
Fires when the user clicks the left mouse button on the object.

onsubmit
Fires when a FORM is about to be submitted.

For more go here:
http://msdn.microsoft.com/library/d...hor/dhtml/reference/dhtml_reference_entry.asp