web client in Python Q

[Jive Dadson]

Hey folks. I know approximately zero about web clients. There's a
simple task I want to do. (I think it's probably simple.) And I figure a
Python script ought to be just the ticket.

Various web services for daily stock ticker info. For example,

http://finance.google.com/finance/historical?q=NYSE:APC&output=csv

and

http://download.finance.yahoo.com/d/quotes.csv?s=APC&f=sl1d1t1c1ohgv&e=.csv

Those will serve up data for the stock APC. But the problem is, a
file-selection dialog pops up. I would like to have a program that
would supply a file name and download the data automatically, without
human or mouse intervention. Is that clear?

Can someone point me in the right direction?

Thankee.

 

[Jerry Hill]

Hey folks. I know approximately zero about web clients. There's a simple
task I want to do. (I think it's probably simple.) And I figure a Python
script ought to be just the ticket.

Various web services for daily stock ticker info. For example,

http://finance.google.com/finance/historical?q=NYSE:APC&output=csv

and

http://download.finance.yahoo.com/d/quotes.csv?s=APC&f=sl1d1t1c1ohgv&e=.csv

That should be a pretty straightforward script. Take a look at this,
and see if it helps:

import urllib
page = urllib.urlopen('http://finance.google.com/finance/historical?q=NYSE:APC&output=csv')
data = page.readlines()
data[0] 'Date,Open,High,Low,Close,Volume\n'
data[1] '7-May-08,76.13,77.94,75.60,76.68,9501300\n'
data[1].strip().split(',')

['7-May-08', '76.13', '77.94', '75.60', '76.68', '9501300']

page = urllib.urlopen('http://download.finance.yahoo.com/d/quotes.csv?s=APC&f=sl1d1t1c1ohgv&e=.csv')
data = page.readlines()
data[0] '"APC",76.68,"5/7/2008","4:01pm",+2.15,76.14,77.94,75.60,9501342\n'
data[1]

Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
data[1]
IndexError: list index out of range
The urllib documentation is here: http://docs.python.org/lib/module-urllib.html
Both of those formats look pretty straightforward to parse, but if
you're dealing with more complicated CSV files, take a look at the csv
module.

 

[Jive Dadson]

Thanks, Jerry! That's so cool. I actually managed to blunder through
with sockets and so forth, but this is much cleaner.

I wonder why it does not work with

http://stockcharts.com/webcgi/wb.exe?Data.web+SLW

I get a connection reset by peer error.

 

[Jerry Hill]

I wonder why it does not work with

http://stockcharts.com/webcgi/wb.exe?Data.web+SLW

It looks like that is a subscription site. That makes things more
complicated, because it means you'll need to figure out how to log in,
then probably store cookies related to your session and offer them
back up to the web server on subsequent requests. Python has modules
that help with these things (mostly in the urllib2 module if i recall
correctly), but it's not quite as straightforward as just fetching a
URL.

I don't have time to dig up an example at the moment, but if you're
still having trouble with it, I can try to pull something together
this weekend.

 

[Jive Dadson]

It looks like that is a subscription site. That makes things more
complicated, because it means you'll need to figure out how to log in,
then probably store cookies related to your session and offer them
back up to the web server on subsequent requests. Python has modules
that help with these things (mostly in the urllib2 module if i recall
correctly), but it's not quite as straightforward as just fetching a
URL.

I don't have time to dig up an example at the moment, but if you're
still having trouble with it, I can try to pull something together
this weekend.

Thanks for the help, Jerry. I do subscribe to that service, but tapping
the data via Python may be more trouble than it's worth. I can get all
the info for stocks and ETF's from finance.google. All I am missing is
indexes. I can probably live without those. But if you want a
challenge, have at it. It would be appreciated.

Jive