yahoo finance api:Timeout::Error: execution expired

[Pen Ttt]

i want to download usa stock quote data via ruby-yahoo finance api,
here is my program:
require 'rubygems'
require 'yahoofinance'
file1=open('/home/pt/usacode','r')
file2=open('/home/pt/usadata','a+')
i=0
while line=file1.gets
line=line.chomp
i=i+1
print line," ",i,"\n"
YahooFinance::get_historical_quotes( line,
Date.parse( '2005-09-09' ),
Date.today() ) do |row|
file2.puts "#{line},#{row.join(',')}"
end
end

there are 6000 companies in the file1 ,each company one line ,i use "i"
to count company,the program can run,but each time i can only download
about ten companies data,the output is

Timeout::Error: execution expired
from /usr/lib/ruby/1.8/timeout.rb:60:in `rbuf_fill'
from /usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
from /usr/lib/ruby/1.8/net/protocol.rb:86:in `read'
from /usr/lib/ruby/1.8/net/http.rb:2236:in `read_chunked'
from /usr/lib/ruby/1.8/net/http.rb:2211:in `read_body_0'
from /usr/lib/ruby/1.8/net/http.rb:2177:in `read_body'
from /usr/lib/ruby/1.8/net/http.rb:773:in `get'
from /usr/lib/ruby/1.8/net/http.rb:1053:in `request'
from /usr/lib/ruby/1.8/net/http.rb:2140:in `reading_body'
from /usr/lib/ruby/1.8/net/http.rb:1052:in `request'
from /usr/lib/ruby/1.8/net/http.rb:772:in `get'
from
/var/lib/gems/1.8/gems/yahoofinance-1.2.2/lib/yahoofinance.rb:419:in
`retrieve_raw_historical_quotes'
from /usr/lib/ruby/1.8/net/http.rb:543:in `start'
from /usr/lib/ruby/1.8/net/http.rb:440:in `start'
from
/var/lib/gems/1.8/gems/yahoofinance-1.2.2/lib/yahoofinance.rb:409:in
`retrieve_raw_historical_quotes'
from
/var/lib/gems/1.8/gems/yahoofinance-1.2.2/lib/yahoofinance.rb:451:in
`get_historical_quotes'
from (irb):10
from :0irb(main):016:0>
what's wrong??

 

[Brian Candler]

i want to download usa stock quote data via ruby-yahoo finance api,
here is my program:

I don't know anything about the Yahoo Finance API, but I know that if I
were running such a public service, I would probably include some
protection against one person trying to hit it 6000 times repeatedly.
Are you sure you're using the right API call to do what you want?

OTOH, a quick search suggests that Yahoo rate limiting is typically done
over a larger period:
http://developer.yahoo.com/search/rate.html

Anyway, I'd suggest you try the following:

(1) Re-order the list of companies you search. Does it always barf on
the same one, or always the 10th in the list?

(2) Insert a "sleep 1" into the loop, see if the problem goes away

(3) Check that the YahooFinance client module is properly closing each
socket, by using "netstat -n" in another window, and checking how many
sockets are in ESTABLISHED state. If there are lots of ESTABLISHED
sockets then the client code isn't closing sockets, and the server may
have a limit on the number of concurrent requests from the same IP.