Help with Screen Scraper!

[Ta St]

Hi,

I need some help with a simple screen scraper app. Here's how it works:

1) I open a connection to a url
2) I look for a specific string
3) When I get the string, I do something with it
4) Important: I'm polling a url looking for data changes.

Problem:
Everything works fine, until the website is updated with new content. At
that point, my code throws an "private method `scan' called for
nil:NilClass (NoMethodError)"

I can get the code to work using
begin
....
rescue
retry
end

But, the problem is that it causes the connection to close and re-open.
This causes a further delay in grabbing the data.

Is there a way to handle this? Again, the polling works fine until the
server updates the new page that I'm scanning. At that time, the error
is thrown and I either have to re-open a connection or fail on error. My
Java application seems to be handling this more gracefully, but, Ruby is
faster on the polling. Thanks for any insight.

Regards.


******************
require 'open-uri'
pval = "14"
url="http://www.someurl.com"
stop = false

while stop == false
#Create timestamp
sts="S " << Time.now.strftime("%m/%d/%y %H %M
%S.")<<Time.now.usec.to_s
puts sts
lines=open(url).read.split("\n")
#Get Lines of data and find the SearchValue
tdate=lines.detect {|line| line =~ /SearchValue/}.scan(/\d+/)
#Compare SearchValue to pval
if tdate[1] != pdate

puts tdate[0]
ts="F "<<Time.now.strftime("%m/%d/%y %H %M
%S.")<<Time.now.usec.to_s
else
puts ".."
end
end