[ANN] forkoff-0.0.4

Discussion in 'Ruby' started by ara howard, Sep 13, 2008.

  1. ara howard

    ara howard Guest

    NAME

    forkoff

    SYNOPSIS

    brain-dead simple parallel processing for ruby

    URI

    http://rubyforge.org/projects/codeforpeople

    INSTALL

    gem install forkoff

    DESCRIPTION

    forkoff works for any enumerable object, iterating a code block to
    run in a
    child process and collecting the results. forkoff can limit the
    number of
    child processes which is, by default, 2.

    HISTORY
    0.0.4
    - code re-org
    - add :strategy option
    - default number of processes is 2, not 8


    SAMPLES

    <========< samples/a.rb >========>

    ~ > cat samples/a.rb

    # forkoff makes it trivial to do parallel processing with ruby,
    the following
    # prints out each word in a separate process
    #

    require 'forkoff'

    %w( hey you ).forkoff!{|word| puts "#{ word } from
    #{ Process.pid }"}

    ~ > ruby samples/a.rb

    hey from 1032
    you from 1033


    <========< samples/b.rb >========>

    ~ > cat samples/b.rb

    # for example, this takes only 4 seconds or so to complete (8
    iterations
    # running in two processes = twice as fast)
    #

    require 'forkoff'

    a = Time.now.to_f

    results =
    (0..7).forkoff do |i|
    sleep 1
    i ** 2
    end

    b = Time.now.to_f

    elapsed = b - a

    puts "elapsed: #{ elapsed }"
    puts "results: #{ results.inspect }"

    ~ > ruby samples/b.rb

    elapsed: 4.25545883178711
    results: [0, 1, 4, 9, 16, 25, 36, 49]


    <========< samples/c.rb >========>

    ~ > cat samples/c.rb

    # forkoff does *NOT* spawn processes in batches, waiting for each
    batch to
    # complete. rather, it keeps a certain number of processes busy
    until all
    # results have been gathered. in otherwords the following will
    ensure that 3
    # processes are running at all times, until the list is complete.
    note that
    # the following will take about 3 seconds to run (3 sets of 3 @ 1
    second).
    #

    require 'forkoff'

    pid = Process.pid

    a = Time.now.to_f

    pstrees =
    %w( a b c d e f g h i ).forkoff! :processes => 3 do |letter|
    sleep 1
    { letter => ` pstree -l 2 #{ pid } ` }
    end


    b = Time.now.to_f

    puts
    puts "pid: #{ pid }"
    puts "elapsed: #{ b - a }"
    puts

    require 'yaml'

    pstrees.each do |pstree|
    y pstree
    end

    ~ > ruby samples/c.rb


    pid: 1048
    elapsed: 3.14415812492371

    ---
    a: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01049 ahoward ruby -Ilib samples/c.rb
    |-+- 01050 ahoward ruby -Ilib samples/c.rb
    \-+- 01051 ahoward ruby -Ilib samples/c.rb

    ---
    b: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01049 ahoward (ruby)
    |-+- 01050 ahoward ruby -Ilib samples/c.rb
    \-+- 01051 ahoward ruby -Ilib samples/c.rb

    ---
    c: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01049 ahoward ruby -Ilib samples/c.rb
    |-+- 01050 ahoward ruby -Ilib samples/c.rb
    \-+- 01051 ahoward ruby -Ilib samples/c.rb

    ---
    d: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01061 ahoward ruby -Ilib samples/c.rb
    |-+- 01062 ahoward ruby -Ilib samples/c.rb
    \-+- 01063 ahoward ruby -Ilib samples/c.rb

    ---
    e: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01061 ahoward (ruby)
    |-+- 01062 ahoward ruby -Ilib samples/c.rb
    \-+- 01063 ahoward ruby -Ilib samples/c.rb

    ---
    f: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01061 ahoward ruby -Ilib samples/c.rb
    |-+- 01062 ahoward ruby -Ilib samples/c.rb
    \-+- 01063 ahoward ruby -Ilib samples/c.rb

    ---
    g: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01090 ahoward ruby -Ilib samples/c.rb
    |-+- 01091 ahoward ruby -Ilib samples/c.rb
    \-+- 01092 ahoward ruby -Ilib samples/c.rb

    ---
    h: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01090 ahoward ruby -Ilib samples/c.rb
    |-+- 01091 ahoward ruby -Ilib samples/c.rb
    \-+- 01092 ahoward ruby -Ilib samples/c.rb

    ---
    i: |
    -+- 01048 ahoward ruby -Ilib samples/c.rb
    |-+- 01090 ahoward ruby -Ilib samples/c.rb
    |-+- 01091 ahoward ruby -Ilib samples/c.rb
    \-+- 01092 ahoward ruby -Ilib samples/c.rb



    <========< samples/d.rb >========>

    ~ > cat samples/d.rb

    # forkoff supports two strategies of reading the result from the
    child: via
    # pipe (the default) or via file. you can select which to use
    using the
    # :strategy option.
    #

    require 'forkoff'

    %w( hey you guys ).forkoff :strategy => :file do |word|
    puts "#{ word } from #{ Process.pid }"
    end

    ~ > ruby samples/d.rb

    hey from 1102
    you from 1103
    guys from 1104




    a @ http://codeforpeople.com/
    --
    we can deny everything, except that we have the possibility of being
    better. simply reflect on that.
    h.h. the 14th dalai lama
     
    ara howard, Sep 13, 2008
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mike Sampson [MSFT]

    [ANN]: NNTP Server slow downs.

    Mike Sampson [MSFT], Oct 7, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    425
    Mike Sampson [MSFT]
    Oct 7, 2003
  2. Michael Livsey
    Replies:
    3
    Views:
    447
    Michael Livsey
    May 27, 2004
  3. ara howard
    Replies:
    20
    Views:
    346
    Jeremy Hinegardner
    Apr 30, 2008
  4. ara howard

    [ANN] forkoff-0.0.1

    ara howard, Apr 22, 2008, in forum: Ruby
    Replies:
    0
    Views:
    110
    ara howard
    Apr 22, 2008
  5. ara.t.howard

    [ANN] forkoff-1.1.0

    ara.t.howard, Oct 12, 2009, in forum: Ruby
    Replies:
    0
    Views:
    110
    ara.t.howard
    Oct 12, 2009
Loading...

Share This Page