open-uril ECONNREFUSED problem

[Cyrus Ghalambor]

I'm working on a simple scraping tool and the open() method has suddenly
stopped working! This 2 line program fails with a ECONNREFUSED exception
in open-uri:

require "open-uri"
open("http://www.yahoo.com")

from /usr/lib/ruby/1.8/open-uri.rb:134:in `open_uri'
from /usr/lib/ruby/1.8/open-uri.rb:424:in `open'
from /usr/lib/ruby/1.8/open-uri.rb:85:in `open'
from /Users/cyrus/Desktop/test.rb:3

/usr/lib/ruby/1.8/timeout.rb:41:in `initialize'
/usr/lib/ruby/1.8/net/protocol.rb:83:in `new'
/usr/lib/ruby/1.8/net/protocol.rb:83:in `connect'
/usr/lib/ruby/1.8/net/protocol.rb:82:in `timeout'
/usr/lib/ruby/1.8/timeout.rb:55:in `timeout'
/usr/lib/ruby/1.8/net/protocol.rb:82:in `connect'
/usr/lib/ruby/1.8/net/protocol.rb:64:in `initialize'
/usr/lib/ruby/1.8/net/http.rb:430:in `open'
/usr/lib/ruby/1.8/net/http.rb:430:in `do_start'
/usr/lib/ruby/1.8/net/http.rb:419:in `start'
/usr/lib/ruby/1.8/net/http.rb:324:in `start'
/usr/lib/ruby/1.8/open-uri.rb:544:in `proxy_open'
/usr/lib/ruby/1.8/open-uri.rb:525:in `direct_open'
/usr/lib/ruby/1.8/open-uri.rb:169:in `open_loop'
/usr/lib/ruby/1.8/open-uri.rb:164:in `catch'
/usr/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
/usr/lib/ruby/1.8/open-uri.rb:134:in `open_uri'
/usr/lib/ruby/1.8/open-uri.rb:424:in `open'
/usr/lib/ruby/1.8/open-uri.rb:85:in `open'
/Users/cyrus/Desktop/test.rb:3

 

[Chris Pearl]

WFM.

Appearantly the Yahoo server is refusing connections from your
development box (maybe too much testing on your part? ;-)

 

[Cyrus Ghalambor]

I've tried different URLs from different locations. Do you think setting
some header information would get around this problem?

 

[Cyrus Ghalambor]

The following still has the same problem:

open("http://www.google.com", "User-Agent" => "Mozilla/4.0")

 

[Eric Hodel]

I'm working on a simple scraping tool and the open() method has
suddenly
stopped working! This 2 line program fails with a ECONNREFUSED
exception
in open-uri:

require "open-uri"
open("http://www.yahoo.com")

Are you behind a proxy?

 

[ara.t.howard]

The following still has the same problem:

open("http://www.google.com", "User-Agent" => "Mozilla/4.0")

firewall?

-a

 

[Cyrus Ghalambor]

Hi everyone,

Thanks for responding to this post. Here's a recap of what's going on:

I had no problems with the open() call for almost two days, but then
suddenly it stopped working. I have no problems with my internet
connection and the site I want to reach is definitely up. I've tried
this test from three difeerent locations with the same result.

Regarding Eric Hodel's question, I'm not sure if I am behind a proxy (or
firewall for that matter). How can I find out?

Also, are there other ways of testing something like this in Ruby? Will
it help to upgrade to 1.8.5?

Thanks again.

 

[Eric Hodel]

Thanks for responding to this post. Here's a recap of what's going on:

I had no problems with the open() call for almost two days, but then
suddenly it stopped working. I have no problems with my internet
connection and the site I want to reach is definitely up. I've tried
this test from three difeerent locations with the same result.

So what changed when it stopped working?

Regarding Eric Hodel's question, I'm not sure if I am behind a
proxy (or
firewall for that matter). How can I find out?

If you're at work, you might be behind a proxy. Check your browser
settings for proxy configuration. If you're at home, unlikely.

Also, are there other ways of testing something like this in Ruby?
Will
it help to upgrade to 1.8.5?

Unlikely.

 

[James Moore]

I have no problems with my internet

connection and the site I want to reach is definitely up.

When you're looking at problems like this, make sure you're using fairly
simple tools. Don't use browsers, for one thing; something like

telnet the.destination.system.com 80

to talk straight to the right port on the destination system is a better
choice than a browser since you know telnet won't play games with things
like proxies and caches.

Regarding Eric Hodel's question, I'm not sure if I am behind a proxy
(or firewall for that matter). How can I find out?

This is the bit that makes me suspicious; if you don't know how to find out
whether or not you're behind a proxy of some sort, what did you do to figure
out that the site you want to reach is "definitely up?"

If you're just testing that you can connect to the remote site by hitting
refresh on a browser, there are possible things between you and the target
that could give you misleading symptoms.

- James Moore

 

[Cyrus Ghalambor]

Thanks James. Actually, I've opened another topic called "Komodo issue"
and I'd appreciate it if you could take a look there. Essentially, I can
make the http connection from within a Terminal session (on my Mac OS X
10.4.6) using the Ruby interpreter. All the tests described above were
being run from within Komodo...

 

[Cyrus Ghalambor]

I reinstalled Komodo (and made sure to remove all the prefs) and now it
works.

Thanks everyone for your help in resolving this.