J
James Mulholland
I'm not sure if this is the best place to ask this, but I hope someone
will be able to help, or at least point me somewhere else.
I've written a screen-scrapper (in Perl) for digg.com. It uses
HTTP::Lite to retrieve the page and regexp's to parse information. It
works, but I'd like to create a Ruby version to help me learn Ruby.
Here is the code I'm trying to use:
require 'net/http'
require 'uri'
Net::HTTP.start( 'www.digg.com', 80 ) do |http|
print( http.get( '/' ).body )
end
If I use this to get another site (eg slashdot.org) it returns all the
HTML, as expected. With digg.com, I get this:
<BR clear=3D"all">
<HR noshade size=3D"1px">
<ADDRESS>
Generated Mon, 28 Nov 2005 20:22:05 GMT by Prolexic.com (SI2LON1/2.0)
</ADDRESS>
</BODY></HTML>
That looks like (I'm guessing) some kind of return message from a
load-balancer or other proxy. I've tried this from 3 different systems
(which use different ISPs) so I don't think it's my system.
Does anyone have any ideas about this? Why does the Perl code work,
but not the Ruby? Is there a fix?
Using Ruby 1.8.3 under Linux, also tried it with Ruby 1.8.2 on Mac OS X.
TIA
will be able to help, or at least point me somewhere else.
I've written a screen-scrapper (in Perl) for digg.com. It uses
HTTP::Lite to retrieve the page and regexp's to parse information. It
works, but I'd like to create a Ruby version to help me learn Ruby.
Here is the code I'm trying to use:
require 'net/http'
require 'uri'
Net::HTTP.start( 'www.digg.com', 80 ) do |http|
print( http.get( '/' ).body )
end
If I use this to get another site (eg slashdot.org) it returns all the
HTML, as expected. With digg.com, I get this:
<BR clear=3D"all">
<HR noshade size=3D"1px">
<ADDRESS>
Generated Mon, 28 Nov 2005 20:22:05 GMT by Prolexic.com (SI2LON1/2.0)
</ADDRESS>
</BODY></HTML>
That looks like (I'm guessing) some kind of return message from a
load-balancer or other proxy. I've tried this from 3 different systems
(which use different ISPs) so I don't think it's my system.
Does anyone have any ideas about this? Why does the Perl code work,
but not the Ruby? Is there a fix?
Using Ruby 1.8.3 under Linux, also tried it with Ruby 1.8.2 on Mac OS X.
TIA