open-uri and ftp problem

[Henrik Horneber]

Hi!
following script (with an existing ftp address) throws an exception.

#file simple_uri
require 'open-uri'

ftp = "ftp://" # ftp address left out in this mail,
# but one that I can acces
open( ftp ){ |f|
f.read
}
# end of file
produces:

:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in `initialize': No
such file or directory - /dev/null (Errno::ENOENT)
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in
`open_uri_original_open'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in `open'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/net/ftp.rb:497:in
`getbinaryfile'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:600:in
`direct_open'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:169:in `open_loop'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:164:in `catch'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:134:in `open_uri'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:424:in `open'
from h:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:85:in `open'
from simple_uri.rb:3

Exit code: 1

]


RTFS'ing *cough* I find that open-uri does this:

[snip]
ftp = Net::FTP.open(self.host)
ftp.login(user, passwd)
# line 600 in open-uri
ftp.getbinaryfile(self.path, '/dev/null', Net::FTP:EFAULT_BLOCKSIZE)
[snap]


Docs for Net::FTP#getbinaryfile say:
--
getbinaryfile(remotefile, localfile = File.basename(remotefile),
blocksize = DEFAULT_BLOCKSIZE) {|data| ...}

Retrieves remotefile in binary mode, storing the result in localfile. If
a block is supplied, it is passed the retrieved data in blocksize chunks.
---


Since I am on Windows, there is no /dev/null to 'store' the file in.

any clues on what I am doing wrong?

regards,
Henrik

 

[Tanaka Akira]

RTFS'ing *cough* I find that open-uri does this:

[snip]
ftp = Net::FTP.open(self.host)
ftp.login(user, passwd)
# line 600 in open-uri
ftp.getbinaryfile(self.path, '/dev/null', Net::FTP:EFAULT_BLOCKSIZE)
[snap]


Docs for Net::FTP#getbinaryfile say:
--
getbinaryfile(remotefile, localfile = File.basename(remotefile),
blocksize = DEFAULT_BLOCKSIZE) {|data| ...}

Retrieves remotefile in binary mode, storing the result in localfile. If
a block is supplied, it is passed the retrieved data in blocksize chunks.
---


Since I am on Windows, there is no /dev/null to 'store' the file in.

any clues on what I am doing wrong?

I think Net::FTP#getbinaryfile should accept nil as the localfile
argument.

If so, open-uri can avoid /dev/null.

 

[Henrik Horneber]

I think Net::FTP#getbinaryfile should accept nil as the localfile
argument.

If so, open-uri can avoid /dev/null.

Hi!

Thanks for your reply!

Unfortunately, Net::FTP#getbinaryfile does not accept nil, as far as I
understand the source, it passes the parameter to open, and dies with

H:/ruby/ruby_install/ruby/lib/ruby/1.8/open-uri.rb:87:in
`open_uri_original_open': cannot convert nil into String (TypeError)

When I change

ftp.getbinaryfile(self.path, '/dev/null', Net::FTP:EFAULT_BLOCKSIZE)
{|str|

to

ftp.getbinaryfile(self.path) {|str|

in open-uri.rb, it only falls back to Net::FTP#getbinaryfile's default
value, which simply is the basename of the remote file, so that I end up
with the file on my hard drive. (This sounds strange if you don't know
the context, but this gets called from open("ftp://") with a block, so I
want to handle the data myself without temporary storage on disk) .

I'm stuck. Would this not be a problem if I were using the cygwin build,
because there might exist some fake /dev/null?

regards,
Henrik

 

[Markus]

You could just modify the definition of FTP.getbinaryfile (either by
modifying the source or overriding it in your code) like so:

def getbinaryfile(remotefile, localfile = File.basename(remotefile),
blocksize = DEFAULT_BLOCKSIZE, &block) # :yield:
data
if @resume
rest_offset = File.size?(localfile)
f = open(localfile, "a")
elsif localfile == '/dev/nul' or localfile == nil
rest_offset = nil
f = nil
else
rest_offset = nil
f = open(localfile, "w")
end
begin
f.binmode if f
retrbinary("RETR " + remotefile, blocksize, rest_offset) do
|data|
f.write(data) if f
yield(data) if block
end
ensure
f.close if f
end
end

And that should do what you want. If this is something you will be
using for a while, I'd suggest negotiating/propagating the change to the
appropriate maintainers, so you don't have to revisit the fix every time
you upgrade.

-- Markus

 

[Henrik Horneber]

You could just modify the definition of FTP.getbinaryfile (either by
modifying the source or overriding it in your code) like so:

[snip code]

And that should do what you want. If this is something you will be
using for a while, I'd suggest negotiating/propagating the change to the
appropriate maintainers, so you don't have to revisit the fix every time
you upgrade.

Hi!

To be honest, I do not have a real use for it at all. I stumbled over it
while trying to hack a download progress display into gems. I simply
wondered if my code worked for ftp downloads, too. So, if I understand
this correctly, as soon as somebody starts to publish gems per anonymous
ftp, some of us windows users are in for a surprise, because gems uses
open-uri like I did in my example.

Given my track record, it's perfectly possible that I misunderstand
something here tho.

regards,
Henrik