Transfering files using drb (distributed Ruby)

Discussion in 'Ruby' started by Francis Joanis, Jun 4, 2004.

  1. Hello everyone,

    I'd like to know if it is possible to use drb to transfer files from the
    server to the client (and vice versa)?

    For example, a remotely callable method could return a byte array then,
    client side, that array could be written back on disk.

    If this is possible, how would the drb infrastructure "behave" when
    transfering rather big arrays (say, 5~10 megabytes)?

    Thanks,

    Francis
    Francis Joanis, Jun 4, 2004
    #1
    1. Advertising

  2. Francis Joanis

    Kirk Haines Guest

    On Sat, 5 Jun 2004 00:33:39 +0900, Francis Joanis wrote
    > Hello everyone,
    >
    > I'd like to know if it is possible to use drb to transfer files from
    > the server to the client (and vice versa)?
    >
    > For example, a remotely callable method could return a byte array
    > then, client side, that array could be written back on disk.
    >
    > If this is possible, how would the drb infrastructure "behave" when
    > transfering rather big arrays (say, 5~10 megabytes)?


    It's definitely possible. As to have the infrastructure behaves, you will
    have processing costs because the data has to be marshalled for transfer and
    then unmarshalled at the other end. I regularly pump hundreds of k per
    second of marshalled data around though, with no problems.

    I'd say just try it. DRb is extremely simple to use. Put a prototype
    together and test it out!


    Kirk Haines
    Kirk Haines, Jun 4, 2004
    #2
    1. Advertising

  3. Kirk Haines wrote:
    > On Sat, 5 Jun 2004 00:33:39 +0900, Francis Joanis wrote
    >
    >>Hello everyone,
    >>
    >>I'd like to know if it is possible to use drb to transfer files from
    >>the server to the client (and vice versa)?
    >>
    >>For example, a remotely callable method could return a byte array
    >>then, client side, that array could be written back on disk.
    >>
    >>If this is possible, how would the drb infrastructure "behave" when
    >>transfering rather big arrays (say, 5~10 megabytes)?

    >
    >
    > It's definitely possible. As to have the infrastructure behaves, you will
    > have processing costs because the data has to be marshalled for transfer and
    > then unmarshalled at the other end. I regularly pump hundreds of k per
    > second of marshalled data around though, with no problems.
    >
    > I'd say just try it. DRb is extremely simple to use. Put a prototype
    > together and test it out!


    And if the file just gets slurped into a string, the marshalling cost
    will be next to nothing.

    irb(main):002:0> str = (1..9).entries.to_s*10
    =>
    "123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"
    irb(main):003:0> Marshal.dump(str)
    =>
    "\004\010\"_123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"
    Joel VanderWerf, Jun 4, 2004
    #3
  4. On Sat, 5 Jun 2004 05:46:04 +0900, Joel VanderWerf
    <> wrote:

    > Kirk Haines wrote:
    >> On Sat, 5 Jun 2004 00:33:39 +0900, Francis Joanis wrote
    >>
    >>> Hello everyone,
    >>>
    >>> I'd like to know if it is possible to use drb to transfer files from
    >>> the server to the client (and vice versa)?
    >>>
    >>> For example, a remotely callable method could return a byte array
    >>> then, client side, that array could be written back on disk.
    >>>
    >>> If this is possible, how would the drb infrastructure "behave" when
    >>> transfering rather big arrays (say, 5~10 megabytes)?

    >>
    >>
    >> It's definitely possible. As to have the infrastructure behaves, you
    >> will have processing costs because the data has to be marshalled for
    >> transfer and then unmarshalled at the other end. I regularly pump
    >> hundreds of k per second of marshalled data around though, with no
    >> problems.
    >>
    >> I'd say just try it. DRb is extremely simple to use. Put a prototype
    >> together and test it out!

    >
    > And if the file just gets slurped into a string, the marshalling cost
    > will be next to nothing.
    >
    > irb(main):002:0> str = (1..9).entries.to_s*10
    > =>
    > "123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"
    > irb(main):003:0> Marshal.dump(str)
    > =>
    > "\004\010\"_123456789123456789123456789123456789123456789123456789123456789123456789123456789123456789"
    >
    >


    Thanks guys, I'm going to try it out and give you some news.

    FJ
    Francis Joanis, Jun 5, 2004
    #4
  5. On 2004/06/05, at 22:53, Francis Joanis wrote:

    > On Sat, 5 Jun 2004 05:46:04 +0900, Joel VanderWerf
    > <> wrote:
    >
    >> Kirk Haines wrote:
    >>> On Sat, 5 Jun 2004 00:33:39 +0900, Francis Joanis wrote
    >>>
    >>>> Hello everyone,
    >>>>
    >>>> I'd like to know if it is possible to use drb to transfer files
    >>>> from the server to the client (and vice versa)?
    >>>>
    >>>> For example, a remotely callable method could return a byte array
    >>>> then, client side, that array could be written back on disk.
    >>>>
    >>>> If this is possible, how would the drb infrastructure "behave" when
    >>>> transfering rather big arrays (say, 5~10 megabytes)?
    >>>
    >>>
    >>> It's definitely possible. As to have the infrastructure behaves,
    >>> you will have processing costs because the data has to be marshalled
    >>> for transfer and then unmarshalled at the other end. I regularly
    >>> pump hundreds of k per second of marshalled data around though, with
    >>> no problems.
    >>>
    >>> I'd say just try it. DRb is extremely simple to use. Put a
    >>> prototype together and test it out!

    >>
    >> And if the file just gets slurped into a string, the marshalling cost
    >> will be next to nothing.
    >>
    >> irb(main):002:0> str = (1..9).entries.to_s*10
    >> =>
    >> "123456789123456789123456789123456789123456789123456789123456789123456
    >> 789123456789123456789"
    >> irb(main):003:0> Marshal.dump(str)
    >> =>
    >> "\004\010\"_1234567891234567891234567891234567891234567891234567891234
    >> 56789123456789123456789123456789"
    >>
    >>

    >
    > Thanks guys, I'm going to try it out and give you some news.


    I wrote a small sample code.

    # setup server
    host1% ruby dcp.rb
    druby://host1:12345

    # and client
    host2% irb -r dcp.rb
    irb(main):001:0> DRb.start_service
    irb(main):002:0> there = DRbObject.new_with_uri('druby://host1:12345')
    irb(main):003:0> here = DCP::Site.new
    # transfer a file
    irb(main):004:0> here.store_from('filename.jpg', there)
    # with progress(?)
    irb(main):005:0> here.store_from('filename.jpg', there) {|x| p x}

    ## dcp.rb
    require 'drb/drb'

    module DCP
    class Site
    include DRbUndumped

    def size(fname)
    File.lstat(fname).size
    end

    def fetch(fname)
    File.open(fname, 'rb') do |fp|
    while buf = fp.read(4096)
    yield(buf)
    end
    end
    nil
    end

    def store_from(fname, there)
    size = there.size(fname)
    wrote = 0

    File.rename(fname, fname + '.bak') if File.exists? fname
    File.open(fname, 'wb') do |fp|
    yield([wrote, size]) if block_given?
    there.fetch(fname) do |buf|
    wrote += fp.write(buf)
    yield([wrote, size]) if block_given?
    nil
    end
    end

    wrote
    end
    end
    end

    if __FILE__ == $0
    DRb.start_service(ARGV.shift, DCP::Site.new)
    puts DRb.uri
    DRb.thread.join
    end
    Masatoshi SEKI, Jun 6, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ilmari Heikkinen

    Tutorial: Distributed pipes with DRb

    Ilmari Heikkinen, Jan 10, 2005, in forum: Ruby
    Replies:
    0
    Views:
    103
    Ilmari Heikkinen
    Jan 10, 2005
  2. Miles Keaton
    Replies:
    3
    Views:
    179
    Miles Keaton
    Mar 30, 2005
  3. Kirk Haines

    More DRb; SSL & DRB & errors

    Kirk Haines, Jul 1, 2005, in forum: Ruby
    Replies:
    0
    Views:
    123
    Kirk Haines
    Jul 1, 2005
  4. J. Wook
    Replies:
    16
    Views:
    279
    Robert Klemme
    May 16, 2007
  5. Ittay Dror
    Replies:
    1
    Views:
    132
    Ittay Dror
    Oct 21, 2008
Loading...

Share This Page