[ANN] posix-spawn 0.3.0 -- first public release (codename, "tigersblood")


R

Ryan Tomayko

<https://github.com/rtomayko/posix-spawn>
$ gem install posix-spawn

tmm1 and I are pleased to announce the initial release of posix-spawn,
a small extension library that implements a subset of Ruby 1.9's new
Process::spawn [1] in a way that takes advantage of fast process
spawning (IEEE Std 1003.1 posix_spawn(2) systems interfaces [2]) where
available and runs on all MRI Rubys >= 1.8.7.

- Fast, constant time process spawning across a variety of platforms
- A largish compatible subset of Ruby 1.9's Process::spawn interface
as well as 1.9 enhancements to Kernel#system, Kernel#`, etc. under
Ruby >= 1.8.7.
- High level and hopefully portable POSIX::Spawn::Child class for
quick and dirty (but correct!) non-streaming IPC scenarios.

See the README for usage and graphs of benchmark results on Linux and
Darwin, or run them yourself:

$ uname -a
Linux aux1 2.6.26-2-xen-amd64 #1 SMP Thu Aug 20 2009 x86_64 GNU/Linux
$ ruby --version
ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
$ gem install posix-spawn
$ posix-spawn-benchmark
benchmarking fork/exec vs. posix_spawn over 1000 runs at 100M res
user system total real
fspawn (fork/exec): 0.080000 14.920000 38.040000 ( 39.029493)
pspawn (posix_spawn): 0.040000 0.010000 0.560000 ( 0.939422)

Work on the library started when tmm1 found, through the use of his
brilliant rbtrace [3] program, a number of slow points in the GitHub
codebase where fork/exec is used heavily to spawn processes. In some
cases, a single fork() system call was using >30ms while in others
using only ~1ms. Our testsuite fork()'d especially slowly. Hmmm.

On Linux, fork(2) slows down as the parent process uses more memory
due to the need to copy page tables for COW. In many common uses of
fork(), where it is followed by one of the exec family of functions to
spawn child processes (Kernel#system, IO::popen, Process::spawn,
etc.), this overhead can be removed by using posix_spawn() or vfork()
instead.

After implementing a simple fast process spawner extension using
posix_spawn() and gaining some familiarity with the posix_spawn family
of C functions, we noticed that it could potentially be used to
implement a large subset of features provided by Ruby 1.9's
Process::spawn.

We love Process::spawn.

We love Process::spawn so much in fact that over the past few months,
even before surfacing any of the issues with Linux fork() slowness, an
effort had been underway at GitHub to move two key libraries (Grit,
the Ruby interface to Git, and Albino, a Ruby wrapper around the
excellent Pygments syntax highlighter) to use Process::spawn
compatible method invocations (implemented with fork/exec under
Ruby 1.8.7) so that we could take advantage of Process::spawn under
Ruby 1.9.

Once we had a basic Process::spawn interface implemented on top of
posix_spawn(), we were able to take some higher level utility classes
from this work on the Grit and Albino projects and include them in
posix-spawn as a nice POSIX::Spawn::Child class. It is:

- Simple, requiring little code for simple stream input and capture
- Internally non-blocking (uses select(2)), so it handles all pipe
hang cases due to exceeding PIPE_BUF limits on one or more streams
- Potentially portable, due to the abstraction over lower-level
process and stream management APIs

We hope to now remove large bodies of Ruby 1.8.7 spawn emulation code
and replace it with posix-spawn.

As the project continued to take shape, we noticed how much more
feature-rich the Kernel#system, IO.popen, etc. methods were in Ruby
1.9. Having been built on the foundation of the new Process::spawn,
they allow for setting up the child's environment, redirecting
arbitrary fds, and all the other great stuff in Process::spawn. We
were able to write Ruby 1.8.7 compatible subset implementations of
those as well and put them under the POSIX::Spawn module.

Now, about that subset. As of this initial release, we were able to
implement the following arguments and options to spawn:
spawn([env,] command... [,options]) => pid

env: hash
name => val : set the environment variable
name => nil : unset the environment variable
command...:
command : /bin/sh -c 'command'
cmdname, arg1, ... : exec argv (no shell)
[cmdname, argv0], arg1, ... : exec argv (no shell)
options: hash
clearing environment variables:
:unsetenv_others => true : clear environment vars not in env
:unsetenv_others => false : don't clear (default)
redirection:
key:
FD : single fd in child process
[FD, FD, ...] : multiple fd in child process
value:
FD : redirect to fd in parent process
:close : close the fd in child process
string : redir w/ open(string, "r" or "w")
[string] : redir w/ open(string, File::RDONLY)
[string, open_mode] : redir w/ open(string, open_mode, 0644)
[string, open_mode, perm] : redir w/ open(string, open_mode, perm)
FD is one of follows
:in : the fd 0 which is the standard input
:eek:ut : the fd 1 which is the standard output
:err : the fd 2 which is the standard error
integer : the fd of specified the integer
io : the fd specified as io.fileno
current directory:
:chdir => str

We have NOT yet implemented these options:
options: hash
process group:
:pgroup => true or 0 : make a new process group
:pgroup => pgid : join to specified process group
:pgroup => nil : don't change the process group (default)
resource limit: resourcename is core, cpu, data, etc.
:rlimit_resourcename => limit
:rlimit_resourcename => [cur_limit, max_limit]
umask:
:umask => int
redirection:
value:
[:child, FD] : redirect to the redirected fd
file descriptor inheritance: close non-redir non-standard fds > 3
:close_others => false : inherit fds (default for system and exec)
:close_others => true : no inherit (default for spawn and popen)

We have ideas for some of these :)pgroup, :umask, [:child, FD]) and
may implement them in future releases; others, like :rlimit, are not
supported by posix_spawn() and have no clear implementations strategy
outside of falling back to fork/exec when detected.

[0] https://github.com/rtomayko/posix-spawn
[1] http://www.ruby-doc.org/core-1.9/classes/Process.html#M002230
[2] http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html
[3] https://github.com/tmm1/rbtrace

Ryan Tomayko
Aman Gupta
 
Ad

Advertisements

C

Charles Oliver Nutter

Did you know about the "spoon" gem? It's a very simple binding of
posix_spawn via FFI that works fine in JRuby too. It would sure be
nice if this could be an FFI solution, so it would work without a C
extension.

<https://github.com/rtomayko/posix-spawn>
$ gem install posix-spawn

tmm1 and I are pleased to announce the initial release of posix-spawn,
a small extension library that implements a subset of Ruby 1.9's new
Process::spawn [1] in a way that takes advantage of fast process
spawning (IEEE Std 1003.1 posix_spawn(2) systems interfaces [2]) where
available and runs on all MRI Rubys >=3D 1.8.7.

=C2=A0- Fast, constant time process spawning across a variety of platform= s
=C2=A0- A largish compatible subset of Ruby 1.9's Process::spawn interfac= e
=C2=A0 as well as 1.9 enhancements to Kernel#system, Kernel#`, etc. under
=C2=A0 Ruby >=3D 1.8.7.
=C2=A0- High level and hopefully portable POSIX::Spawn::Child class for
=C2=A0 quick and dirty (but correct!) non-streaming IPC scenarios.

See the README for usage and graphs of benchmark results on Linux and
Darwin, or run them yourself:

=C2=A0 =C2=A0$ uname -a
=C2=A0 =C2=A0Linux aux1 2.6.26-2-xen-amd64 #1 SMP Thu Aug 20 2009 x86_64 = GNU/Linux
=C2=A0 =C2=A0$ ruby --version
=C2=A0 =C2=A0ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
=C2=A0 =C2=A0$ gem install posix-spawn
=C2=A0 =C2=A0$ posix-spawn-benchmark
=C2=A0 =C2=A0benchmarking fork/exec vs. posix_spawn over 1000 runs at 100= M res
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 user =C2=A0 =C2=A0 system =C2=A0 =C2=
=A0 =C2=A0total =C2=A0 =C2=A0 =C2=A0 =C2=A0real
=C2=A0 =C2=A0fspawn (fork/exec): =C2=A0 =C2=A0 0.080000 =C2=A014.920000 = =C2=A038.040000 ( 39.029493)
=C2=A0 =C2=A0pspawn (posix_spawn): =C2=A0 0.040000 =C2=A0 0.010000 =C2=A0= 0.560000 ( =C2=A00.939422)

Work on the library started when tmm1 found, through the use of his
brilliant rbtrace [3] program, a number of slow points in the GitHub
codebase where fork/exec is used heavily to spawn processes. In some
cases, a single fork() system call was using >30ms while in others
using only ~1ms. Our testsuite fork()'d especially slowly. Hmmm.

On Linux, fork(2) slows down as the parent process uses more memory
due to the need to copy page tables for COW. In many common uses of
fork(), where it is followed by one of the exec family of functions to
spawn child processes (Kernel#system, IO::popen, Process::spawn,
etc.), this overhead can be removed by using posix_spawn() or vfork()
instead.

After implementing a simple fast process spawner extension using
posix_spawn() and gaining some familiarity with the posix_spawn family
of C functions, we noticed that it could potentially be used to
implement a large subset of features provided by Ruby 1.9's
Process::spawn.

We love Process::spawn.

We love Process::spawn so much in fact that over the past few months,
even before surfacing any of the issues with Linux fork() slowness, an
effort had been underway at GitHub to move two key libraries (Grit,
the Ruby interface to Git, and Albino, a Ruby wrapper around the
excellent Pygments syntax highlighter) to use Process::spawn
compatible method invocations (implemented with fork/exec under
Ruby 1.8.7) so that we could take advantage of Process::spawn under
Ruby 1.9.

Once we had a basic Process::spawn interface implemented on top of
posix_spawn(), we were able to take some higher level utility classes
from this work on the Grit and Albino projects and include them in
posix-spawn as a nice POSIX::Spawn::Child class. It is:

=C2=A0- Simple, requiring little code for simple stream input and capture
=C2=A0- Internally non-blocking (uses select(2)), so it handles all pipe
=C2=A0 hang cases due to exceeding PIPE_BUF limits on one or more streams
=C2=A0- Potentially portable, due to the abstraction over lower-level
=C2=A0 process and stream management APIs

We hope to now remove large bodies of Ruby 1.8.7 spawn emulation code
and replace it with posix-spawn.

As the project continued to take shape, we noticed how much more
feature-rich the Kernel#system, IO.popen, etc. methods were in Ruby
1.9. Having been built on the foundation of the new Process::spawn,
they allow for setting up the child's environment, redirecting
arbitrary fds, and all the other great stuff in Process::spawn. We
were able to write Ruby 1.8.7 compatible subset implementations of
those as well and put them under the POSIX::Spawn module.

Now, about that subset. As of this initial release, we were able to
implement the following arguments and options to spawn:
spawn([env,] command... [,options]) =3D> pid

env: hash
=C2=A0 name =3D> val : set the environment variable
=C2=A0 name =3D> nil : unset the environment variable
command...:
=C2=A0 command =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 : /bin/sh -c 'command'
=C2=A0 cmdname, arg1, ... =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: exec argv = (no shell)
=C2=A0 [cmdname, argv0], arg1, ... : exec argv (no shell)
options: hash
=C2=A0 clearing environment variables:
=C2=A0 =C2=A0 :unsetenv_others =3D> true =C2=A0 : clear environment vars= not in env
=C2=A0 =C2=A0 :unsetenv_others =3D> false =C2=A0: don't clear (default)
=C2=A0 redirection:
=C2=A0 =C2=A0 key:
=C2=A0 =C2=A0 =C2=A0 FD =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= : single fd in child process
=C2=A0 =C2=A0 =C2=A0 [FD, FD, ...] =C2=A0 : multiple fd in child process
=C2=A0 =C2=A0 value:
=C2=A0 =C2=A0 =C2=A0 FD =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: redirect to fd in parent process
=C2=A0 =C2=A0 =C2=A0 :close =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0: close the fd in child process
=C2=A0 =C2=A0 =C2=A0 string =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0: redir w/ open(string, "r" or "w")
=C2=A0 =C2=A0 =C2=A0 [string] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0: redir w/ open(string, File::RDONLY)
=C2=A0 =C2=A0 =C2=A0 [string, open_mode] =C2=A0 =C2=A0 =C2=A0 : redir w/= open(string, open_mode, 0644)
=C2=A0 =C2=A0 =C2=A0 [string, open_mode, perm] : redir w/ open(string, o= pen_mode, perm)
=C2=A0 =C2=A0 FD is one of follows
=C2=A0 =C2=A0 =C2=A0 :in =C2=A0 =C2=A0 : the fd 0 which is the standard = input
=C2=A0 =C2=A0 =C2=A0 :eek:ut =C2=A0 =C2=A0: the fd 1 which is the standard = output
=C2=A0 =C2=A0 =C2=A0 :err =C2=A0 =C2=A0: the fd 2 which is the standard = error
=C2=A0 =C2=A0 =C2=A0 integer : the fd of specified the integer
=C2=A0 =C2=A0 =C2=A0 io =C2=A0 =C2=A0 =C2=A0: the fd specified as io.fil= eno
=C2=A0 current directory:
=C2=A0 =C2=A0 :chdir =3D> str

We have NOT yet implemented these options:
options: hash
=C2=A0 process group:
=C2=A0 =C2=A0 :pgroup =3D> true or 0 : make a new process group
=C2=A0 =C2=A0 :pgroup =3D> pgid =C2=A0 =C2=A0 =C2=A0: join to specified = process group
=C2=A0 =C2=A0 :pgroup =3D> nil =C2=A0 =C2=A0 =C2=A0 : don't change the p= rocess group (default)
=C2=A0 resource limit: resourcename is core, cpu, data, etc.
=C2=A0 =C2=A0 :rlimit_resourcename =3D> limit
=C2=A0 =C2=A0 :rlimit_resourcename =3D> [cur_limit, max_limit]
=C2=A0 umask:
=C2=A0 =C2=A0 :umask =3D> int
=C2=A0 redirection:
=C2=A0 =C2=A0 value:
=C2=A0 =C2=A0 =C2=A0 [:child, FD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0: redirect to the redirected fd
=C2=A0 file descriptor inheritance: close non-redir non-standard fds > 3
=C2=A0 =C2=A0 :close_others =3D> false : inherit fds (default for system= and exec)
=C2=A0 =C2=A0 :close_others =3D> true =C2=A0: no inherit (default for sp=
awn and popen)

We have ideas for some of these :)pgroup, :umask, [:child, FD]) and
may implement them in future releases; others, like :rlimit, are not
supported by posix_spawn() and have no clear implementations strategy
outside of falling back to fork/exec when detected.

[0] https://github.com/rtomayko/posix-spawn
[1] http://www.ruby-doc.org/core-1.9/classes/Process.html#M002230
[2] http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.= html
[3] https://github.com/tmm1/rbtrace

Ryan Tomayko
Aman Gupta
 
C

Charles Oliver Nutter

FWIW, https://github.com/headius/spoon

Did you know about the "spoon" gem? It's a very simple binding of
posix_spawn via FFI that works fine in JRuby too. It would sure be
nice if this could be an FFI solution, so it would work without a C
extension.

<https://github.com/rtomayko/posix-spawn>
$ gem install posix-spawn

tmm1 and I are pleased to announce the initial release of posix-spawn,
a small extension library that implements a subset of Ruby 1.9's new
Process::spawn [1] in a way that takes advantage of fast process
spawning (IEEE Std 1003.1 posix_spawn(2) systems interfaces [2]) where
available and runs on all MRI Rubys >=3D 1.8.7.

=C2=A0- Fast, constant time process spawning across a variety of platfor= ms
=C2=A0- A largish compatible subset of Ruby 1.9's Process::spawn interfa= ce
=C2=A0 as well as 1.9 enhancements to Kernel#system, Kernel#`, etc. unde= r
=C2=A0 Ruby >=3D 1.8.7.
=C2=A0- High level and hopefully portable POSIX::Spawn::Child class for
=C2=A0 quick and dirty (but correct!) non-streaming IPC scenarios.

See the README for usage and graphs of benchmark results on Linux and
Darwin, or run them yourself:

=C2=A0 =C2=A0$ uname -a
=C2=A0 =C2=A0Linux aux1 2.6.26-2-xen-amd64 #1 SMP Thu Aug 20 2009 x86_64= GNU/Linux
=C2=A0 =C2=A0$ ruby --version
=C2=A0 =C2=A0ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
=C2=A0 =C2=A0$ gem install posix-spawn
=C2=A0 =C2=A0$ posix-spawn-benchmark
=C2=A0 =C2=A0benchmarking fork/exec vs. posix_spawn over 1000 runs at 10= 0M res
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 user =C2=A0 =C2=A0 system =C2=A0 =
=C2=A0 =C2=A0total =C2=A0 =C2=A0 =C2=A0 =C2=A0real
=C2=A0 =C2=A0fspawn (fork/exec): =C2=A0 =C2=A0 0.080000 =C2=A014.920000 = =C2=A038.040000 ( 39.029493)
=C2=A0 =C2=A0pspawn (posix_spawn): =C2=A0 0.040000 =C2=A0 0.010000 =C2= =A0 0.560000 ( =C2=A00.939422)

Work on the library started when tmm1 found, through the use of his
brilliant rbtrace [3] program, a number of slow points in the GitHub
codebase where fork/exec is used heavily to spawn processes. In some
cases, a single fork() system call was using >30ms while in others
using only ~1ms. Our testsuite fork()'d especially slowly. Hmmm.

On Linux, fork(2) slows down as the parent process uses more memory
due to the need to copy page tables for COW. In many common uses of
fork(), where it is followed by one of the exec family of functions to
spawn child processes (Kernel#system, IO::popen, Process::spawn,
etc.), this overhead can be removed by using posix_spawn() or vfork()
instead.

After implementing a simple fast process spawner extension using
posix_spawn() and gaining some familiarity with the posix_spawn family
of C functions, we noticed that it could potentially be used to
implement a large subset of features provided by Ruby 1.9's
Process::spawn.

We love Process::spawn.

We love Process::spawn so much in fact that over the past few months,
even before surfacing any of the issues with Linux fork() slowness, an
effort had been underway at GitHub to move two key libraries (Grit,
the Ruby interface to Git, and Albino, a Ruby wrapper around the
excellent Pygments syntax highlighter) to use Process::spawn
compatible method invocations (implemented with fork/exec under
Ruby 1.8.7) so that we could take advantage of Process::spawn under
Ruby 1.9.

Once we had a basic Process::spawn interface implemented on top of
posix_spawn(), we were able to take some higher level utility classes
from this work on the Grit and Albino projects and include them in
posix-spawn as a nice POSIX::Spawn::Child class. It is:

=C2=A0- Simple, requiring little code for simple stream input and captur= e
=C2=A0- Internally non-blocking (uses select(2)), so it handles all pipe
=C2=A0 hang cases due to exceeding PIPE_BUF limits on one or more stream= s
=C2=A0- Potentially portable, due to the abstraction over lower-level
=C2=A0 process and stream management APIs

We hope to now remove large bodies of Ruby 1.8.7 spawn emulation code
and replace it with posix-spawn.

As the project continued to take shape, we noticed how much more
feature-rich the Kernel#system, IO.popen, etc. methods were in Ruby
1.9. Having been built on the foundation of the new Process::spawn,
they allow for setting up the child's environment, redirecting
arbitrary fds, and all the other great stuff in Process::spawn. We
were able to write Ruby 1.8.7 compatible subset implementations of
those as well and put them under the POSIX::Spawn module.

Now, about that subset. As of this initial release, we were able to
implement the following arguments and options to spawn:
spawn([env,] command... [,options]) =3D> pid

env: hash
=C2=A0 name =3D> val : set the environment variable
=C2=A0 name =3D> nil : unset the environment variable
command...:
=C2=A0 command =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 : /bin/sh -c 'command'
=C2=A0 cmdname, arg1, ... =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: exec argv= (no shell)
=C2=A0 [cmdname, argv0], arg1, ... : exec argv (no shell)
options: hash
=C2=A0 clearing environment variables:
=C2=A0 =C2=A0 :unsetenv_others =3D> true =C2=A0 : clear environment var= s not in env
=C2=A0 =C2=A0 :unsetenv_others =3D> false =C2=A0: don't clear (default)
=C2=A0 redirection:
=C2=A0 =C2=A0 key:
=C2=A0 =C2=A0 =C2=A0 FD =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0: single fd in child process
=C2=A0 =C2=A0 =C2=A0 [FD, FD, ...] =C2=A0 : multiple fd in child proces= s
=C2=A0 =C2=A0 value:
=C2=A0 =C2=A0 =C2=A0 FD =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: redirect to fd in parent process
=C2=A0 =C2=A0 =C2=A0 :close =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0: close the fd in child process
=C2=A0 =C2=A0 =C2=A0 string =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0: redir w/ open(string, "r" or "w")
=C2=A0 =C2=A0 =C2=A0 [string] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0: redir w/ open(string, File::RDONLY)
=C2=A0 =C2=A0 =C2=A0 [string, open_mode] =C2=A0 =C2=A0 =C2=A0 : redir w= / open(string, open_mode, 0644)
=C2=A0 =C2=A0 =C2=A0 [string, open_mode, perm] : redir w/ open(string, = open_mode, perm)
=C2=A0 =C2=A0 FD is one of follows
=C2=A0 =C2=A0 =C2=A0 :in =C2=A0 =C2=A0 : the fd 0 which is the standard= input
=C2=A0 =C2=A0 =C2=A0 :eek:ut =C2=A0 =C2=A0: the fd 1 which is the standard= output
=C2=A0 =C2=A0 =C2=A0 :err =C2=A0 =C2=A0: the fd 2 which is the standard= error
=C2=A0 =C2=A0 =C2=A0 integer : the fd of specified the integer
=C2=A0 =C2=A0 =C2=A0 io =C2=A0 =C2=A0 =C2=A0: the fd specified as io.fi= leno
=C2=A0 current directory:
=C2=A0 =C2=A0 :chdir =3D> str

We have NOT yet implemented these options:
options: hash
=C2=A0 process group:
=C2=A0 =C2=A0 :pgroup =3D> true or 0 : make a new process group
=C2=A0 =C2=A0 :pgroup =3D> pgid =C2=A0 =C2=A0 =C2=A0: join to specified= process group
=C2=A0 =C2=A0 :pgroup =3D> nil =C2=A0 =C2=A0 =C2=A0 : don't change the = process group (default)
=C2=A0 resource limit: resourcename is core, cpu, data, etc.
=C2=A0 =C2=A0 :rlimit_resourcename =3D> limit
=C2=A0 =C2=A0 :rlimit_resourcename =3D> [cur_limit, max_limit]
=C2=A0 umask:
=C2=A0 =C2=A0 :umask =3D> int
=C2=A0 redirection:
=C2=A0 =C2=A0 value:
=C2=A0 =C2=A0 =C2=A0 [:child, FD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0: redirect to the redirected fd
=C2=A0 file descriptor inheritance: close non-redir non-standard fds > = 3
=C2=A0 =C2=A0 :close_others =3D> false : inherit fds (default for syste= m and exec)
=C2=A0 =C2=A0 :close_others =3D> true =C2=A0: no inherit (default for s=
pawn and popen)

We have ideas for some of these :)pgroup, :umask, [:child, FD]) and
may implement them in future releases; others, like :rlimit, are not
supported by posix_spawn() and have no clear implementations strategy
outside of falling back to fork/exec when detected.

[0] https://github.com/rtomayko/posix-spawn
[1] http://www.ruby-doc.org/core-1.9/classes/Process.html#M002230
[2] http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn= html
[3] https://github.com/tmm1/rbtrace

Ryan Tomayko
Aman Gupta
 
R

Ryan Tomayko

Did you know about the "spoon" gem? It's a very simple binding of
posix_spawn via FFI that works fine in JRuby too. It would sure be
nice if this could be an FFI solution, so it would work without a C
extension.

Yes. I experimented with using spoon for JRuby support in Grit some
time ago. Unfortunately, a large number of posix_spawn() features
require real OS file descriptors and there was no standard/supported
way of retrieving them for most standard Java stream types.

That issue aside, I'm not opposed to using FFI so long as the
performance profile is on par with the C extension. We certainly plan
on supporting JRuby in some way down the road. I doubt the
Process::spawn interface can be fully implemented without real fds,
but we'd like the higher level POSIX::Spawn::Child class to be JRuby
compatible.

Ryan
 
R

Ryan Tomayko

Yes. I experimented with using spoon for JRuby support in Grit some
time ago. Unfortunately, a large number of posix_spawn() features
require real OS file descriptors and there was no standard/supported
way of retrieving them for most standard Java stream types.

Turns out I had some of that work laying around:

https://github.com/rtomayko/spoon/compare/spawn_file_actions

I was able to make posix_spawn_file_actions_adddup2() and
posix_spawn_file_actions_addclose() calls but running the test fails
with:

$ ruby lib/spoon.rb
_posix_spawn_file_actions_init => 0
/bin/echo: write: Bad file descriptor
read:

Under Ruby 1.9.2 with the ffi gem, I get:

ruby lib/spoon.rb
_posix_spawn_file_actions_init => 0
read: hello world

If you can describe a method for retrieving real fds under JRuby
(especially IO objects returned from open() and IO.pipe()), I don't
see any reason why we couldn't have an FFI based implementation with
only a little more work.

Ryan
 
Ad

Advertisements

C

Charles Oliver Nutter

Turns out I had some of that work laying around:

https://github.com/rtomayko/spoon/compare/spawn_file_actions

I was able to make posix_spawn_file_actions_adddup2() and
posix_spawn_file_actions_addclose() calls but running the test fails
with:

=C2=A0 =C2=A0$ ruby lib/spoon.rb
=C2=A0 =C2=A0_posix_spawn_file_actions_init =3D> 0
=C2=A0 =C2=A0/bin/echo: write: Bad file descriptor
=C2=A0 =C2=A0read: ...
If you can describe a method for retrieving real fds under JRuby
(especially IO objects returned from open() and IO.pipe()), I don't
see any reason why we couldn't have an FFI based implementation with
only a little more work.

As you discovered, in JRuby IO.fileno is simulated. This is mostly
because the JVM does not provide us a way to work with file
descriptors directly; everything is wrapped in a Channel or an
Input/OutputStream, and determining the real fd (if one exists) is
tricky.

However, this is a feature many people have asked for, since many
native calls need a real file descriptor. I'll look into it and see if
there's a way we can provide real file descriptor values somehow.

- Charlie
 
Ad

Advertisements


Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top