[ANN] open4-0.8.0

Discussion in 'Ruby' started by ara.t.howard@noaa.gov, Sep 25, 2006.

  1. Guest

    URIS

    http://rubyforge.org/projects/codeforpeople/
    http://www.codeforpeople.com/lib/ruby/

    SYNOPSIS

    open child process with handles on pid, stdin, stdout, and stderr: manage
    child processes and their io handles easily.

    HISTORY

    0.8.0:

    - fixed a critical bug whereby a process producing tons of stdout, but for
    which the stdout was not handled, would cause the child process to
    become blocked/hung writing to the pipe. eg, this command would cause a
    hang

    include Open4

    spawn 'ruby -e" puts Array.new(65536){ 42 } "'

    whereas this one would not

    include Open4

    spawn 'ruby -e" puts Array.new(65536){ 42 } "', :stdout=>StringIO.new

    this version handles the former by spawning a 'null' thread which reads,
    but does not process stdout/stderr. that way commands which generate
    tons of output will never become blocked.


    INSTALL

    ~> gem install open4

    SAMPLES

    ----------------------------------------------------------------------------
    simple usage
    ----------------------------------------------------------------------------

    harp: > cat sample/simple.rb
    require "open4"

    pid, stdin, stdout, stderr = Open4::popen4 "sh"

    stdin.puts "echo 42.out"
    stdin.puts "echo 42.err 1>&2"
    stdin.close

    ignored, status = Process::waitpid2 pid

    puts "pid : #{ pid }"
    puts "stdout : #{ stdout.read.strip }"
    puts "stderr : #{ stderr.read.strip }"
    puts "status : #{ status.inspect }"
    puts "exitstatus : #{ status.exitstatus }"


    harp: > ruby sample/simple.rb
    pid : 17273
    stdout : 42.out
    stderr : 42.err
    status : #<Process::Status: pid=17273,exited(0)>
    exitstatus : 0


    ----------------------------------------------------------------------------
    in block form - the child process is automatically waited for
    ----------------------------------------------------------------------------

    harp: > cat sample/block.rb
    require 'open4'

    status =
    Open4::popen4("sh") do |pid, stdin, stdout, stderr|
    stdin.puts "echo 42.out"
    stdin.puts "echo 42.err 1>&2"
    stdin.close

    puts "pid : #{ pid }"
    puts "stdout : #{ stdout.read.strip }"
    puts "stderr : #{ stderr.read.strip }"
    end

    puts "status : #{ status.inspect }"
    puts "exitstatus : #{ status.exitstatus }"


    harp: > ruby sample/block.rb
    pid : 17295
    stdout : 42.out
    stderr : 42.err
    status : #<Process::Status: pid=17295,exited(0)>
    exitstatus : 0

    ----------------------------------------------------------------------------
    exceptions are marshaled from child to parent if fork/exec fails
    ----------------------------------------------------------------------------

    harp: > cat sample/exception.rb
    require "open4"
    Open4::popen4 "noexist"


    harp: > ruby sample/exception.rb
    /dmsp/reference/ruby-1.8.1//lib/ruby/site_ruby/open4.rb:100:in `popen4': No such file or directory - noexist (Errno::ENOENT)
    from sample/exception.rb:3

    ----------------------------------------------------------------------------
    the spawn method provides and even more convenient method of running a
    process, allowing any object that supports 'each', 'read', or 'to_s' to be
    given as stdin and any objects that support '<<' to be given as
    stdout/stderr. an exception is thrown if the exec'd cmd fails (nonzero
    exitstatus) unless the option 'raise'=>false is given
    ----------------------------------------------------------------------------

    harp: > cat sample/spawn.rb
    require 'open4'
    include Open4

    cat = ' ruby -e" ARGF.each{|line| STDOUT << line} " '

    stdout, stderr = '', ''
    status = spawn cat, 'stdin' => '42', 'stdout' => stdout, 'stderr' => stderr
    p status
    p stdout
    p stderr

    stdout, stderr = '', ''
    status = spawn cat, 0=>'42', 1=>stdout, 2=>stderr
    p status
    p stdout
    p stderr


    harp: > RUBYLIB=lib ruby sample/spawn.rb
    0
    "42"
    ""
    0
    "42"
    ""


    ----------------------------------------------------------------------------
    the bg/background method is similar to spawn, but the process is
    automatically set running in a thread. the returned thread has several
    methods added dynamically which return the pid and blocking calls to the
    exitstatus.
    ----------------------------------------------------------------------------

    harp: > cat sample/bg.rb
    require 'yaml'
    require 'open4'
    include Open4

    stdin = '42'
    stdout = ''
    stderr = ''

    t = bg 'ruby -e"sleep 4; puts ARGF.read"', 0=>stdin, 1=>stdout, 2=>stderr

    waiter = Thread.new{ y t.pid => t.exitstatus } # t.exitstatus is a blocking call!

    while((status = t.status))
    y "status" => status
    sleep 1
    end

    waiter.join

    y "stdout" => stdout


    harp: > ruby sample/bg.rb
    ---
    status: run
    ---
    status: sleep
    ---
    status: sleep
    ---
    status: sleep
    ---
    21357: 0
    ---
    stdout: "42\n"

    ----------------------------------------------------------------------------
    the timeout methods can be used to ensure execution is preceding at the
    desired interval. note also how to setup a 'pipeline'
    ----------------------------------------------------------------------------

    harp: > cat sample/stdin_timeout.rb
    require 'open4'

    producer = 'ruby -e" STDOUT.sync = true; loop{sleep(rand+rand) and puts 42} "'

    consumer = 'ruby -e" STDOUT.sync = true; STDIN.each{|line| puts line} "'

    open4(producer) do |pid, i, o, e|

    open4.spawn consumer, :stdin=>o, :stdout=>STDOUT, :stdin_timeout => 1.4

    end


    harp: > ruby sample/stdin_timeout.rb
    42
    42
    42
    42
    42
    /dmsp/reference/ruby-1.8.1//lib/ruby/1.8/timeout.rb:42:in `relay': execution expired (Timeout::Error)

    AUTHOR



    LICENSE

    ruby's

    -a
    --
    in order to be effective truth must penetrate like an arrow - and that is
    likely to hurt. -- wei wu wei
    , Sep 25, 2006
    #1
    1. Advertising

  2. Corey Jewett Guest

    $ sudo gem install open4
    ERROR: While executing gem ... (OpenURI::HTTPError)
    404 Not Found

    :(

    Corey


    On Sep 25, 2006, at 10:24 AM, wrote:

    >
    > URIS
    >
    > http://rubyforge.org/projects/codeforpeople/
    > http://www.codeforpeople.com/lib/ruby/
    >
    > SYNOPSIS
    >
    > open child process with handles on pid, stdin, stdout, and
    > stderr: manage
    > child processes and their io handles easily.
    >
    > HISTORY
    >
    > 0.8.0:
    >
    > - fixed a critical bug whereby a process producing tons of
    > stdout, but for
    > which the stdout was not handled, would cause the child
    > process to
    > become blocked/hung writing to the pipe. eg, this command
    > would cause a
    > hang
    >
    > include Open4
    >
    > spawn 'ruby -e" puts Array.new(65536){ 42 } "'
    >
    > whereas this one would not
    >
    > include Open4
    >
    > spawn 'ruby -e" puts Array.new(65536){ 42 }
    > "', :stdout=>StringIO.new
    >
    > this version handles the former by spawning a 'null' thread
    > which reads,
    > but does not process stdout/stderr. that way commands which
    > generate
    > tons of output will never become blocked.
    >
    >
    > INSTALL
    >
    > ~> gem install open4
    >
    > SAMPLES
    >
    >
    > ----------------------------------------------------------------------
    > ------
    > simple usage
    >
    > ----------------------------------------------------------------------
    > ------
    >
    > harp: > cat sample/simple.rb
    > require "open4"
    >
    > pid, stdin, stdout, stderr = Open4::popen4 "sh"
    >
    > stdin.puts "echo 42.out"
    > stdin.puts "echo 42.err 1>&2"
    > stdin.close
    >
    > ignored, status = Process::waitpid2 pid
    >
    > puts "pid : #{ pid }"
    > puts "stdout : #{ stdout.read.strip }"
    > puts "stderr : #{ stderr.read.strip }"
    > puts "status : #{ status.inspect }"
    > puts "exitstatus : #{ status.exitstatus }"
    >
    >
    > harp: > ruby sample/simple.rb
    > pid : 17273
    > stdout : 42.out
    > stderr : 42.err
    > status : #<Process::Status: pid=17273,exited(0)>
    > exitstatus : 0
    >
    >
    >
    > ----------------------------------------------------------------------
    > ------
    > in block form - the child process is automatically waited for
    >
    > ----------------------------------------------------------------------
    > ------
    >
    > harp: > cat sample/block.rb
    > require 'open4'
    >
    > status =
    > Open4::popen4("sh") do |pid, stdin, stdout, stderr|
    > stdin.puts "echo 42.out"
    > stdin.puts "echo 42.err 1>&2"
    > stdin.close
    >
    > puts "pid : #{ pid }"
    > puts "stdout : #{ stdout.read.strip }"
    > puts "stderr : #{ stderr.read.strip }"
    > end
    >
    > puts "status : #{ status.inspect }"
    > puts "exitstatus : #{ status.exitstatus }"
    >
    >
    > harp: > ruby sample/block.rb
    > pid : 17295
    > stdout : 42.out
    > stderr : 42.err
    > status : #<Process::Status: pid=17295,exited(0)>
    > exitstatus : 0
    >
    >
    > ----------------------------------------------------------------------
    > ------
    > exceptions are marshaled from child to parent if fork/exec fails
    >
    > ----------------------------------------------------------------------
    > ------
    >
    > harp: > cat sample/exception.rb
    > require "open4"
    > Open4::popen4 "noexist"
    >
    >
    > harp: > ruby sample/exception.rb
    > /dmsp/reference/ruby-1.8.1//lib/ruby/site_ruby/open4.rb:100:in
    > `popen4': No such file or directory - noexist (Errno::ENOENT)
    > from sample/exception.rb:3
    >
    >
    > ----------------------------------------------------------------------
    > ------
    > the spawn method provides and even more convenient method of
    > running a
    > process, allowing any object that supports 'each', 'read', or
    > 'to_s' to be
    > given as stdin and any objects that support '<<' to be given as
    > stdout/stderr. an exception is thrown if the exec'd cmd fails
    > (nonzero
    > exitstatus) unless the option 'raise'=>false is given
    >
    > ----------------------------------------------------------------------
    > ------
    >
    > harp: > cat sample/spawn.rb
    > require 'open4'
    > include Open4
    >
    > cat = ' ruby -e" ARGF.each{|line| STDOUT << line} " '
    >
    > stdout, stderr = '', ''
    > status = spawn cat, 'stdin' => '42', 'stdout' => stdout,
    > 'stderr' => stderr
    > p status
    > p stdout
    > p stderr
    >
    > stdout, stderr = '', ''
    > status = spawn cat, 0=>'42', 1=>stdout, 2=>stderr
    > p status
    > p stdout
    > p stderr
    >
    >
    > harp: > RUBYLIB=lib ruby sample/spawn.rb
    > 0
    > "42"
    > ""
    > 0
    > "42"
    > ""
    >
    >
    >
    > ----------------------------------------------------------------------
    > ------
    > the bg/background method is similar to spawn, but the process is
    > automatically set running in a thread. the returned thread has
    > several
    > methods added dynamically which return the pid and blocking calls
    > to the
    > exitstatus.
    >
    > ----------------------------------------------------------------------
    > ------
    >
    > harp: > cat sample/bg.rb
    > require 'yaml'
    > require 'open4'
    > include Open4
    >
    > stdin = '42'
    > stdout = ''
    > stderr = ''
    >
    > t = bg 'ruby -e"sleep 4; puts ARGF.read"', 0=>stdin, 1=>stdout,
    > 2=>stderr
    >
    > waiter = Thread.new{ y t.pid => t.exitstatus } # t.exitstatus
    > is a blocking call!
    >
    > while((status = t.status))
    > y "status" => status
    > sleep 1
    > end
    >
    > waiter.join
    >
    > y "stdout" => stdout
    >
    >
    > harp: > ruby sample/bg.rb
    > ---
    > status: run
    > ---
    > status: sleep
    > ---
    > status: sleep
    > ---
    > status: sleep
    > ---
    > 21357: 0
    > ---
    > stdout: "42\n"
    >
    >
    > ----------------------------------------------------------------------
    > ------
    > the timeout methods can be used to ensure execution is preceding
    > at the
    > desired interval. note also how to setup a 'pipeline'
    >
    > ----------------------------------------------------------------------
    > ------
    >
    > harp: > cat sample/stdin_timeout.rb
    > require 'open4'
    >
    > producer = 'ruby -e" STDOUT.sync = true; loop{sleep(rand+rand)
    > and puts 42} "'
    >
    > consumer = 'ruby -e" STDOUT.sync = true; STDIN.each{|line| puts
    > line} "'
    >
    > open4(producer) do |pid, i, o, e|
    >
    > open4.spawn
    > consumer, :stdin=>o, :stdout=>STDOUT, :stdin_timeout => 1.4
    >
    > end
    >
    >
    > harp: > ruby sample/stdin_timeout.rb
    > 42
    > 42
    > 42
    > 42
    > 42
    > /dmsp/reference/ruby-1.8.1//lib/ruby/1.8/timeout.rb:42:in
    > `relay': execution expired (Timeout::Error)
    >
    > AUTHOR
    >
    >
    >
    > LICENSE
    >
    > ruby's
    >
    > -a
    > --
    > in order to be effective truth must penetrate like an arrow - and
    > that is
    > likely to hurt. -- wei wu wei
    >
    Corey Jewett, Sep 25, 2006
    #2
    1. Advertising

  3. Guest

    On Tue, 26 Sep 2006, Corey Jewett wrote:

    > $ sudo gem install open4
    > ERROR: While executing gem ... (OpenURI::HTTPError)
    > 404 Not Found
    >
    > :(


    gems sometimes take quite a while to propagate. in the meantime you can grab
    it directly (no mirrors) using

    http://rubyforge.org/frs/?group_id=1024&release_id=7103

    or

    http://codeforpeople.com/lib/ruby/open4/
    http://codeforpeople.com/lib/ruby/open4/open4-0.8.0

    regards.

    -a
    --
    in order to be effective truth must penetrate like an arrow - and that is
    likely to hurt. -- wei wu wei
    , Sep 25, 2006
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. [ANN] open4-0.2.0

    , Feb 14, 2006, in forum: Ruby
    Replies:
    0
    Views:
    79
  2. [ANN] open4-0.5.0

    , Aug 8, 2006, in forum: Ruby
    Replies:
    0
    Views:
    71
  3. [ANN] open4-0.5.1

    , Aug 22, 2006, in forum: Ruby
    Replies:
    0
    Views:
    107
  4. [ANN] open4-0.6.0

    , Aug 31, 2006, in forum: Ruby
    Replies:
    0
    Views:
    75
  5. [ANN] open4-0.7.0

    , Sep 1, 2006, in forum: Ruby
    Replies:
    0
    Views:
    63
Loading...

Share This Page