Pulling data from a .asps site

[hall.jeff]

There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...

 

[Todd O'Bryan]

---------- Forwarded message ----------
From: Todd O'Bryan <[email protected]>
Date: Nov 27, 2007 1:48 PM
Subject: Re: Pulling data from a .asps site
To: (e-mail address removed)


Check out Selenium Remote Control! It's very easy to use.

 

[Zepo Len]

There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...

Well, Python supports moving the mouse pointer so it's should be quite
simple to write such a program.

 

[Diez B. Roggisch]

There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...

There needs no browser to be involved - unless there is AJAX in the
game, and even then - after alle, it's all HTTP-requests.

Whichfor there are several options to work with - one popular is

http://wwwsearch.sourceforge.net/mechanize/

Diez

 

[Tim Williams]

There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...

Amongst others, IEC will allow you to open an IE browser window,
navigate around sites (including pushing buttons), and retrieve text.

http://www.mayukhbose.com/python/IEC/index.php

HTH