File descriptors

Discussion in 'Ruby' started by Antonio Moreno, Nov 20, 2006.

  1. Hi,

    I'm trying to code a script for replacing an application which takes
    it's input from file descriptors 0 (stdin) and 1 (stdout). It must also
    be able to open another application which in turn also reads from the
    above mentioned file descriptors.

    I've managed to take stdin and put it into stdin of the second app,
    though couldn't find the way of opening stdout for reading.

    In http://www.rubycentral.com/book/ref_m_kernel.html#Kernel.open, for
    example, states pretty clear that "...The returned IO object may be used
    to write to the standard input and read from the standard output of this
    subprocess..."

    Does anybody have a clue about this? I'm not very skilled in Ruby, so
    any help will be very appreciated.

    Regards.

    --
    Posted via http://www.ruby-forum.com/.
     
    Antonio Moreno, Nov 20, 2006
    #1
    1. Advertising

  2. On 20.11.2006 20:24, Antonio Moreno wrote:
    > I'm trying to code a script for replacing an application which takes
    > it's input from file descriptors 0 (stdin) and 1 (stdout).


    This is not possible. You can only read from stdin and not stdout.

    > It must also
    > be able to open another application which in turn also reads from the
    > above mentioned file descriptors.


    This is not clear to me: does the other app read from the same fd's
    (i.e. reading continues in the sub process) or do you want to connect
    them via a pipe? (The latter is done with IO.popen() or Kernel open as
    you discovered already).

    > I've managed to take stdin and put it into stdin of the second app,
    > though couldn't find the way of opening stdout for reading.
    >
    > In http://www.rubycentral.com/book/ref_m_kernel.html#Kernel.open, for
    > example, states pretty clear that "...The returned IO object may be used
    > to write to the standard input and read from the standard output of this
    > subprocess..."
    >
    > Does anybody have a clue about this? I'm not very skilled in Ruby, so
    > any help will be very appreciated.


    What exactly is your problem? Did you try it and it did not work?
    Here's a simply sample that works for me

    irb(main):010:0> open("|cat", "r+") {|io| Thread.new {5.times {|i|
    io.puts i}; io.flush; io.close_write}; p io.readlines}
    ["0\n", "1\n", "2\n", "3\n", "4\n"]
    => nil

    Note the mode flag.

    Kind regards

    robert
     
    Robert Klemme, Nov 20, 2006
    #2
    1. Advertising

  3. Robert Klemme wrote:
    > On 20.11.2006 20:24, Antonio Moreno wrote:
    >> I'm trying to code a script for replacing an application which takes
    >> it's input from file descriptors 0 (stdin) and 1 (stdout).

    >
    > This is not possible. You can only read from stdin and not stdout.
    >

    (snif)

    > > It must also
    >> be able to open another application which in turn also reads from the
    >> above mentioned file descriptors.

    >
    > This is not clear to me: does the other app read from the same fd's
    > (i.e. reading continues in the sub process) or do you want to connect
    > them via a pipe? (The latter is done with IO.popen() or Kernel open as
    > you discovered already).
    >
    >> I've managed to take stdin and put it into stdin of the second app,
    >> though couldn't find the way of opening stdout for reading.
    >>
    >> In http://www.rubycentral.com/book/ref_m_kernel.html#Kernel.open, for
    >> example, states pretty clear that "...The returned IO object may be used
    >> to write to the standard input and read from the standard output of this
    >> subprocess..."
    >>
    >> Does anybody have a clue about this? I'm not very skilled in Ruby, so
    >> any help will be very appreciated.

    >
    > What exactly is your problem? Did you try it and it did not work?
    > Here's a simply sample that works for me
    >

    I'll tell you exactly what am I trying to do. I have qmail and I want to
    replace qmail-queue
    (http://www.qmail.org/qmail-manual-html/man8/qmail-queue.html) with my
    ruby script in order to make some mangling of the email, and then call
    qmail-queue to let the email go on.

    > irb(main):010:0> open("|cat", "r+") {|io| Thread.new {5.times {|i|
    > io.puts i}; io.flush; io.close_write}; p io.readlines}
    > ["0\n", "1\n", "2\n", "3\n", "4\n"]
    > => nil
    >
    > Note the mode flag.
    >
    > Kind regards
    >
    > robert


    Thanks, Robert,

    Regards.

    --
    Posted via http://www.ruby-forum.com/.
     
    Antonio Moreno, Nov 20, 2006
    #3
  4. Antonio Moreno wrote:
    > Robert Klemme wrote:
    > > On 20.11.2006 20:24, Antonio Moreno wrote:
    > >> I'm trying to code a script for replacing an application which takes
    > >> it's input from file descriptors 0 (stdin) and 1 (stdout).

    > >
    > > This is not possible. You can only read from stdin and not stdout.
    > >

    > (snif)


    Robert is half right and half wrong (on POSIX compatible platforms
    anyway):

    readable_fd1 = IO.for_fd(1,"r")
    STDERR.puts "FROM STDIN (fd 0): #{STDIN.read}"
    STDERR.puts "FROM STDOUT (fd 1): #{readable_fd1.read}"

    You might argue that if someone provides a file on fd 1 it isn't really
    stdout, but you can certainly open fd 1 (or any fd) both for reading or
    writing if whatever is attached to them supports the mode you want - fd
    0, 1 and 2 aren't special in any way to the OS.

    You can test the above like this:
    ruby in.rb 1<bar.txt 0<foo.txt

    The 1<filename and 0<filename opens the files for input at the file
    descriptors specified.

    Vidar
     
    Vidar Hokstad, Nov 21, 2006
    #4
  5. On 21.11.2006 01:21, Vidar Hokstad wrote:
    > Antonio Moreno wrote:
    >> Robert Klemme wrote:
    >>> On 20.11.2006 20:24, Antonio Moreno wrote:
    >>>> I'm trying to code a script for replacing an application which takes
    >>>> it's input from file descriptors 0 (stdin) and 1 (stdout).
    >>> This is not possible. You can only read from stdin and not stdout.
    >>>

    >> (snif)

    >
    > Robert is half right and half wrong (on POSIX compatible platforms
    > anyway):
    >
    > readable_fd1 = IO.for_fd(1,"r")
    > STDERR.puts "FROM STDIN (fd 0): #{STDIN.read}"
    > STDERR.puts "FROM STDOUT (fd 1): #{readable_fd1.read}"
    >
    > You might argue that if someone provides a file on fd 1 it isn't really
    > stdout, but you can certainly open fd 1 (or any fd) both for reading or
    > writing if whatever is attached to them supports the mode you want - fd
    > 0, 1 and 2 aren't special in any way to the OS.
    >
    > You can test the above like this:
    > ruby in.rb 1<bar.txt 0<foo.txt
    >
    > The 1<filename and 0<filename opens the files for input at the file
    > descriptors specified.


    Thanks for elaborating! Actually I was not aware that you can do this.
    However, even if it is possible I would not do it as the general
    convention is that 1 should be written to and 0 be read from only.
    After all, what do you gain by reading from FD 1 if nothing writes to
    it? :)

    But, as we have seen, the issue at hand was about connecting parent and
    child process through pipes.

    Kind regards

    robert
     
    Robert Klemme, Nov 21, 2006
    #5
  6. Robert Klemme wrote:
    > Thanks for elaborating! Actually I was not aware that you can do this.
    > However, even if it is possible I would not do it as the general
    > convention is that 1 should be written to and 0 be read from only.


    I'd restrict that to say that the convention is to do this if your app
    is meant to be executed from a shell or shell script. When you're
    writing code meant to plug into another application, all bets are off.

    Particularly with Qmail as Antonio was looking to do - Qmail has some
    very weird conventions and most of the different pieces talk to each
    other via different set of predefined file descriptors and you don't
    really have any choice.

    Vidar
     
    Vidar Hokstad, Nov 21, 2006
    #6
  7. Vidar Hokstad wrote:
    >
    >
    > Robert is half right and half wrong (on POSIX compatible platforms
    > anyway):
    >
    > readable_fd1 = IO.for_fd(1,"r")
    > STDERR.puts "FROM STDIN (fd 0): #{STDIN.read}"
    > STDERR.puts "FROM STDOUT (fd 1): #{readable_fd1.read}"
    >
    > You might argue that if someone provides a file on fd 1 it isn't really
    > stdout, but you can certainly open fd 1 (or any fd) both for reading or
    > writing if whatever is attached to them supports the mode you want - fd
    > 0, 1 and 2 aren't special in any way to the OS.
    >
    > You can test the above like this:
    > ruby in.rb 1<bar.txt 0<foo.txt
    >
    > The 1<filename and 0<filename opens the files for input at the file
    > descriptors specified.
    >
    > Vidar


    Thanks, Vidar! You solved half my problem. Now all I'd need to know is
    how to write on fd 1 of a process spawned from ruby with IO.popen.

    I tried the following without success:

    f = open("|-", "w+")
    if f == nil
    puts "This is Child"
    r_fd1 = IO.for_fd(1,"r+")
    STDERR.puts "FROM STDOUT (fd 1): #{r_fd1.read}"
    STDERR.puts "Hijo: #{STDIN.gets}"
    exit 69
    else
    STDERR.puts "Running parent."
    f.puts "From parent\n"
    STDERR.puts "Got: #{f.gets}"
    STDIN.puts "To fd 1 of the child."
    pid = Process.wait
    STDERR.puts "Child terminated, pid = #{pid}, exit code = #{$? >>
    8}"
    end

    If I commmented the lines with "readable_fd1", this piece of code would
    run. What I get is:

    fork.rb:4:in `for_fd': Invalid argument (Errno::EINVAL)
    from fork.rb:4
    Running parent.
    fork.rb:10:in `write': Broken pipe (Errno::EPIPE)
    from fork.rb:10



    Thanks again.

    --
    Posted via http://www.ruby-forum.com/.
     
    Antonio Moreno, Nov 21, 2006
    #7
  8. Antonio Moreno wrote:

    > Thanks, Vidar! You solved half my problem. Now all I'd need to know is
    > how to write on fd 1 of a process spawned from ruby with IO.popen.


    That's trickier... In C you'd do this as follows:
    1. Create a pair of pipe's
    2. Fork
    3. Close fd 0 and 1 in the child process (as they are the fd's the
    parent is reading from)
    4. Call dup2() to duplicate the read end's of the two pipe's you
    created to fd 0 and 1.
    5. Close the original fd's (both read and write sides) for the pipe's
    in the child.
    6. exec the original qmail-queue.

    Step 1. you can achieve with IO#pipe. Steps 2,3,5. and 6. are
    reasonably straightforward. The problem is 4. I haven't found a way of
    calling dup2() in Ruby (I grep'ed through the 1.8.4 sources even).

    An alternative that works on some systems (including Linux), would be
    to use IO#fcntl and F_DUPFD (do a "man fcntl" to read about how it
    works on Linux), but I haven't tested that. Lacking dup2() is a pretty
    serious limitation for doing serious work on POSIX platforms so I hope
    I'm mistaken or that it's fixed in more recent versions...

    Vidar
     
    Vidar Hokstad, Nov 21, 2006
    #8
  9. Thanks to both of you, for your interest and for your help.


    ----------------------
    Please, don't send any more email to since the
    account has been disabled.
    ----------------------

    --
    Posted via http://www.ruby-forum.com/.
     
    Antonio Moreno, Nov 27, 2006
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gordon Beaton

    Re: Running out of file descriptors

    Gordon Beaton, Jul 24, 2003, in forum: Java
    Replies:
    0
    Views:
    393
    Gordon Beaton
    Jul 24, 2003
  2. Oz Levanon

    File Descriptors Leak

    Oz Levanon, Nov 25, 2003, in forum: Java
    Replies:
    3
    Views:
    580
    Robert Olofsson
    Nov 25, 2003
  3. JG
    Replies:
    5
    Views:
    432
    Lawrence Kirby
    Feb 8, 2005
  4. DJ Dharme
    Replies:
    2
    Views:
    272
    Maxim Yegorushkin
    Oct 20, 2008
  5. Stephan Beal
    Replies:
    11
    Views:
    2,136
    CBFalconer
    Jan 8, 2009
Loading...

Share This Page