Transfering files using drb (distributed Ruby)

[Francis Joanis]

Hello everyone,

I'd like to know if it is possible to use drb to transfer files from the
server to the client (and vice versa)?

For example, a remotely callable method could return a byte array then,
client side, that array could be written back on disk.

If this is possible, how would the drb infrastructure "behave" when
transfering rather big arrays (say, 5~10 megabytes)?

Thanks,

Francis

 

[Kirk Haines]

Hello everyone,

I'd like to know if it is possible to use drb to transfer files from
the server to the client (and vice versa)?

For example, a remotely callable method could return a byte array
then, client side, that array could be written back on disk.

If this is possible, how would the drb infrastructure "behave" when
transfering rather big arrays (say, 5~10 megabytes)?

It's definitely possible. As to have the infrastructure behaves, you will
have processing costs because the data has to be marshalled for transfer and
then unmarshalled at the other end. I regularly pump hundreds of k per
second of marshalled data around though, with no problems.

I'd say just try it. DRb is extremely simple to use. Put a prototype
together and test it out!


Kirk Haines

 

[Joel VanderWerf]

On Sat, 5 Jun 2004 00:33:39 +0900, Francis Joanis wrote



It's definitely possible. As to have the infrastructure behaves, you will
have processing costs because the data has to be marshalled for transfer and
then unmarshalled at the other end. I regularly pump hundreds of k per
second of marshalled data around though, with no problems.

I'd say just try it. DRb is extremely simple to use. Put a prototype
together and test it out!

And if the file just gets slurped into a string, the marshalling cost
will be next to nothing.

irb(main):002:0> str = (1..9).entries.to_s*10
=>
"123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"
irb(main):003:0> Marshal.dump(str)
=>
"\004\010\"_123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"

 

[Francis Joanis]

And if the file just gets slurped into a string, the marshalling cost
will be next to nothing.

irb(main):002:0> str = (1..9).entries.to_s*10
=>
"123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"
irb(main):003:0> Marshal.dump(str)
=>
"\004\010\"_123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"

Thanks guys, I'm going to try it out and give you some news.

FJ

 

[Masatoshi SEKI]

Thanks guys, I'm going to try it out and give you some news.

I wrote a small sample code.

# setup server
host1% ruby dcp.rb
druby://host1:12345

# and client
host2% irb -r dcp.rb
irb(main):001:0> DRb.start_service
irb(main):002:0> there = DRbObject.new_with_uri('druby://host1:12345')
irb(main):003:0> here = DCP::Site.new
# transfer a file
irb(main):004:0> here.store_from('filename.jpg', there)
# with progress(?)
irb(main):005:0> here.store_from('filename.jpg', there) {|x| p x}

## dcp.rb
require 'drb/drb'

module DCP
class Site
include DRbUndumped

def size(fname)
File.lstat(fname).size
end

def fetch(fname)
File.open(fname, 'rb') do |fp|
while buf = fp.read(4096)
yield(buf)
end
end
nil
end

def store_from(fname, there)
size = there.size(fname)
wrote = 0

File.rename(fname, fname + '.bak') if File.exists? fname
File.open(fname, 'wb') do |fp|
yield([wrote, size]) if block_given?
there.fetch(fname) do |buf|
wrote += fp.write(buf)
yield([wrote, size]) if block_given?
nil
end
end

wrote
end
end
end

if __FILE__ == $0
DRb.start_service(ARGV.shift, DCP::Site.new)
puts DRb.uri
DRb.thread.join
end