File descriptors

[Antonio Moreno]

Hi,

I'm trying to code a script for replacing an application which takes
it's input from file descriptors 0 (stdin) and 1 (stdout). It must also
be able to open another application which in turn also reads from the
above mentioned file descriptors.

I've managed to take stdin and put it into stdin of the second app,
though couldn't find the way of opening stdout for reading.

In http://www.rubycentral.com/book/ref_m_kernel.html#Kernel.open, for
example, states pretty clear that "...The returned IO object may be used
to write to the standard input and read from the standard output of this
subprocess..."

Does anybody have a clue about this? I'm not very skilled in Ruby, so
any help will be very appreciated.

Regards.

 

[Robert Klemme]

I'm trying to code a script for replacing an application which takes
it's input from file descriptors 0 (stdin) and 1 (stdout).

This is not possible. You can only read from stdin and not stdout.

It must also
be able to open another application which in turn also reads from the
above mentioned file descriptors.

This is not clear to me: does the other app read from the same fd's
(i.e. reading continues in the sub process) or do you want to connect
them via a pipe? (The latter is done with IO.popen() or Kernel open as
you discovered already).

I've managed to take stdin and put it into stdin of the second app,
though couldn't find the way of opening stdout for reading.

In http://www.rubycentral.com/book/ref_m_kernel.html#Kernel.open, for
example, states pretty clear that "...The returned IO object may be used
to write to the standard input and read from the standard output of this
subprocess..."

Does anybody have a clue about this? I'm not very skilled in Ruby, so
any help will be very appreciated.

What exactly is your problem? Did you try it and it did not work?
Here's a simply sample that works for me

irb(main):010:0> open("|cat", "r+") {|io| Thread.new {5.times {|i|
io.puts i}; io.flush; io.close_write}; p io.readlines}
["0\n", "1\n", "2\n", "3\n", "4\n"]
=> nil

Note the mode flag.

Kind regards

robert

 

[Antonio Moreno]

This is not possible. You can only read from stdin and not stdout.
(snif)


This is not clear to me: does the other app read from the same fd's
(i.e. reading continues in the sub process) or do you want to connect
them via a pipe? (The latter is done with IO.popen() or Kernel open as
you discovered already).


What exactly is your problem? Did you try it and it did not work?
Here's a simply sample that works for me

I'll tell you exactly what am I trying to do. I have qmail and I want to
replace qmail-queue
(http://www.qmail.org/qmail-manual-html/man8/qmail-queue.html) with my
ruby script in order to make some mangling of the email, and then call
qmail-queue to let the email go on.

irb(main):010:0> open("|cat", "r+") {|io| Thread.new {5.times {|i|
io.puts i}; io.flush; io.close_write}; p io.readlines}
["0\n", "1\n", "2\n", "3\n", "4\n"]
=> nil

Note the mode flag.

Kind regards

robert

Thanks, Robert,

Regards.

 

[Vidar Hokstad]

(snif)

Robert is half right and half wrong (on POSIX compatible platforms
anyway):

readable_fd1 = IO.for_fd(1,"r")
STDERR.puts "FROM STDIN (fd 0): #{STDIN.read}"
STDERR.puts "FROM STDOUT (fd 1): #{readable_fd1.read}"

You might argue that if someone provides a file on fd 1 it isn't really
stdout, but you can certainly open fd 1 (or any fd) both for reading or
writing if whatever is attached to them supports the mode you want - fd
0, 1 and 2 aren't special in any way to the OS.

You can test the above like this:
ruby in.rb 1<bar.txt 0<foo.txt

The 1<filename and 0<filename opens the files for input at the file
descriptors specified.

Vidar

 

[Robert Klemme]

Robert is half right and half wrong (on POSIX compatible platforms
anyway):

readable_fd1 = IO.for_fd(1,"r")
STDERR.puts "FROM STDIN (fd 0): #{STDIN.read}"
STDERR.puts "FROM STDOUT (fd 1): #{readable_fd1.read}"

You might argue that if someone provides a file on fd 1 it isn't really
stdout, but you can certainly open fd 1 (or any fd) both for reading or
writing if whatever is attached to them supports the mode you want - fd
0, 1 and 2 aren't special in any way to the OS.

You can test the above like this:
ruby in.rb 1<bar.txt 0<foo.txt

The 1<filename and 0<filename opens the files for input at the file
descriptors specified.

Thanks for elaborating! Actually I was not aware that you can do this.
However, even if it is possible I would not do it as the general
convention is that 1 should be written to and 0 be read from only.
After all, what do you gain by reading from FD 1 if nothing writes to
it?

But, as we have seen, the issue at hand was about connecting parent and
child process through pipes.

Kind regards

robert

 

[Vidar Hokstad]

Thanks for elaborating! Actually I was not aware that you can do this.
However, even if it is possible I would not do it as the general
convention is that 1 should be written to and 0 be read from only.

I'd restrict that to say that the convention is to do this if your app
is meant to be executed from a shell or shell script. When you're
writing code meant to plug into another application, all bets are off.

Particularly with Qmail as Antonio was looking to do - Qmail has some
very weird conventions and most of the different pieces talk to each
other via different set of predefined file descriptors and you don't
really have any choice.

Vidar

 

[Antonio Moreno]

Robert is half right and half wrong (on POSIX compatible platforms
anyway):

readable_fd1 = IO.for_fd(1,"r")
STDERR.puts "FROM STDIN (fd 0): #{STDIN.read}"
STDERR.puts "FROM STDOUT (fd 1): #{readable_fd1.read}"

You might argue that if someone provides a file on fd 1 it isn't really
stdout, but you can certainly open fd 1 (or any fd) both for reading or
writing if whatever is attached to them supports the mode you want - fd
0, 1 and 2 aren't special in any way to the OS.

You can test the above like this:
ruby in.rb 1<bar.txt 0<foo.txt

The 1<filename and 0<filename opens the files for input at the file
descriptors specified.

Vidar

Thanks, Vidar! You solved half my problem. Now all I'd need to know is
how to write on fd 1 of a process spawned from ruby with IO.popen.

I tried the following without success:

f = open("|-", "w+")
if f == nil
puts "This is Child"
r_fd1 = IO.for_fd(1,"r+")
STDERR.puts "FROM STDOUT (fd 1): #{r_fd1.read}"
STDERR.puts "Hijo: #{STDIN.gets}"
exit 69
else
STDERR.puts "Running parent."
f.puts "From parent\n"
STDERR.puts "Got: #{f.gets}"
STDIN.puts "To fd 1 of the child."
pid = Process.wait
STDERR.puts "Child terminated, pid = #{pid}, exit code = #{$? >>
8}"
end

If I commmented the lines with "readable_fd1", this piece of code would
run. What I get is:

fork.rb:4:in `for_fd': Invalid argument (Errno::EINVAL)
from fork.rb:4
Running parent.
fork.rb:10:in `write': Broken pipe (Errno::EPIPE)
from fork.rb:10



Thanks again.

 

[Vidar Hokstad]

Thanks, Vidar! You solved half my problem. Now all I'd need to know is
how to write on fd 1 of a process spawned from ruby with IO.popen.

That's trickier... In C you'd do this as follows:
1. Create a pair of pipe's
2. Fork
3. Close fd 0 and 1 in the child process (as they are the fd's the
parent is reading from)
4. Call dup2() to duplicate the read end's of the two pipe's you
created to fd 0 and 1.
5. Close the original fd's (both read and write sides) for the pipe's
in the child.
6. exec the original qmail-queue.

Step 1. you can achieve with IO#pipe. Steps 2,3,5. and 6. are
reasonably straightforward. The problem is 4. I haven't found a way of
calling dup2() in Ruby (I grep'ed through the 1.8.4 sources even).

An alternative that works on some systems (including Linux), would be
to use IO#fcntl and F_DUPFD (do a "man fcntl" to read about how it
works on Linux), but I haven't tested that. Lacking dup2() is a pretty
serious limitation for doing serious work on POSIX platforms so I hope
I'm mistaken or that it's fixed in more recent versions...

Vidar

 

[Antonio Moreno]

Thanks to both of you, for your interest and for your help.