running external process?

Discussion in 'Ruby' started by Ferenc Engard, Jan 18, 2004.

  1. Hi all,

    Is there a simple way to spawn an external program, feed its stdin, and
    get its stdout?

    The problem with popen is, if I want to feed a few MB's of input to it,
    then it hangs (I suspect that its stdout IO buffer is full) before I
    could read out its stdout on the next line. So, the following do not
    work:

    io=IO.popen("externalfilter")
    io.write(verybigstring)
    result=io.read

    Thanks,
    Ferenc
     
    Ferenc Engard, Jan 18, 2004
    #1
    1. Advertising

  2. --Boundary-02=_WWqCA6VXupBxfHq
    Content-Type: text/plain;
    charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
    Content-Description: signed data
    Content-Disposition: inline

    On Sunday 18 January 2004 5:39 am, Ferenc Engard wrote:
    > Hi all,
    >
    > Is there a simple way to spawn an external program, feed its stdin,
    > and get its stdout?
    >
    > The problem with popen is, if I want to feed a few MB's of input to
    > it, then it hangs (I suspect that its stdout IO buffer is full)
    > before I could read out its stdout on the next line. So, the
    > following do not work:
    >
    > io=3DIO.popen("externalfilter")
    > io.write(verybigstring)
    > result=3Dio.read


    require 'open3'
    Open3.popen3("externalfilter") { |sin,sout,serr|
    sin.write(verybigstring)
    result =3D sout.read
    }

    =2D-=20
    Wesley J. Landaker -
    OpenPGP FP: 4135 2A3B 4726 ACC5 9094 0097 F0A9 8A4C 4CD6 E3D2


    --Boundary-02=_WWqCA6VXupBxfHq
    Content-Type: application/pgp-signature
    Content-Description: signature

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.4 (GNU/Linux)

    iD8DBQBACqWW8KmKTEzW49IRAnIkAJ0Vd2dbU6GPjP0O42JlmsGNTF9W7gCfchXd
    OcpyAmJSdkzdgbIEsG4jdsM=
    =YLkY
    -----END PGP SIGNATURE-----

    --Boundary-02=_WWqCA6VXupBxfHq--
     
    Wesley J Landaker, Jan 18, 2004
    #2
    1. Advertising

  3. Ferenc Engard

    Ara.T.Howard Guest

    On Sun, 18 Jan 2004, Ferenc Engard wrote:

    > Date: Sun, 18 Jan 2004 21:39:41 +0900
    > From: Ferenc Engard <>
    > Newsgroups: comp.lang.ruby
    > Subject: running external process?
    >
    > Hi all,
    >
    > Is there a simple way to spawn an external program, feed its stdin, and
    > get its stdout?
    >
    > The problem with popen is, if I want to feed a few MB's of input to it,
    > then it hangs (I suspect that its stdout IO buffer is full) before I
    > could read out its stdout on the next line. So, the following do not
    > work:
    >
    > io=IO.popen("externalfilter")
    > io.write(verybigstring)

    io.close # the program is still reading it's stding here
    > result=io.read
    >
    > Thanks,
    > Ferenc


    -a
    --

    ATTN: please update your address books with address below!

    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | STP :: http://www.ngdc.noaa.gov/stp/
    | NGDC :: http://www.ngdc.noaa.gov/
    | NESDIS :: http://www.nesdis.noaa.gov/
    | NOAA :: http://www.noaa.gov/
    | US DOC :: http://www.commerce.gov/
    |
    | The difference between art and science is that science is what we
    | understand well enough to explain to a computer.
    | Art is everything else.
    | -- Donald Knuth, "Discover"
    |
    | /bin/sh -c 'for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done'
    ===============================================================================
     
    Ara.T.Howard, Jan 18, 2004
    #3
  4. --Boundary-02=_gZqCAPrbmqy6PrH
    Content-Type: text/plain;
    charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
    Content-Description: signed data
    Content-Disposition: inline

    On Sunday 18 January 2004 8:26 am, Wesley J Landaker wrote:
    > On Sunday 18 January 2004 5:39 am, Ferenc Engard wrote:
    > > Hi all,
    > >
    > > Is there a simple way to spawn an external program, feed its stdin,
    > > and get its stdout?
    > >
    > > The problem with popen is, if I want to feed a few MB's of input to
    > > it, then it hangs (I suspect that its stdout IO buffer is full)
    > > before I could read out its stdout on the next line. So, the
    > > following do not work:
    > >
    > > io=3DIO.popen("externalfilter")
    > > io.write(verybigstring)
    > > result=3Dio.read

    >
    > require 'open3'
    > Open3.popen3("externalfilter") { |sin,sout,serr|
    > sin.write(verybigstring)
    > result =3D sout.read
    > }


    Or (should have mentioned this in my first e-mail) if that still hangs,=20
    you might try using Threads:

    require 'open3'
    Open3.popen3("externalfilter") { |sin,sout,serr|
    threads =3D []
    threads << Thread.new {
    sin.write(verybigstring)
    }
    threads << Thread.new {
    result =3D sout.read
    }
    threads.each {|t| t.join }
    }

    =2D-=20
    Wesley J. Landaker -
    OpenPGP FP: 4135 2A3B 4726 ACC5 9094 0097 F0A9 8A4C 4CD6 E3D2


    --Boundary-02=_gZqCAPrbmqy6PrH
    Content-Type: application/pgp-signature
    Content-Description: signature

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.4 (GNU/Linux)

    iD8DBQBACqZg8KmKTEzW49IRAig3AJ47xcktxRif46Koj5jfzeTPa2rUDACfTLNr
    OrQEmw9R0/vWZq8DcaAS9DM=
    =Xy10
    -----END PGP SIGNATURE-----

    --Boundary-02=_gZqCAPrbmqy6PrH--
     
    Wesley J Landaker, Jan 18, 2004
    #4
  5. "Ferenc Engard" <> schrieb im Newsbeitrag
    news:...
    > Hi all,
    >
    > Is there a simple way to spawn an external program, feed its stdin, and
    > get its stdout?
    >
    > The problem with popen is, if I want to feed a few MB's of input to it,
    > then it hangs (I suspect that its stdout IO buffer is full) before I
    > could read out its stdout on the next line. So, the following do not
    > work:
    >
    > io=IO.popen("externalfilter")
    > io.write(verybigstring)
    > result=io.read


    You need at least two threads or non blocking IO. How about:

    IO.popen( "cat", "w+" ) do |io|
    r = Thread.new(io) do |reader|
    while ( line = reader.gets )
    line.chomp!
    $stdout.puts line
    end
    end

    (1..100000).each {|i| io.puts i}
    r.join
    end

    Regards

    robert
     
    Robert Klemme, Jan 18, 2004
    #5
  6. > > The problem with popen is, if I want to feed a few MB's of input to it,
    > > then it hangs (I suspect that its stdout IO buffer is full) before I
    > > could read out its stdout on the next line. So, the following do not
    > > work:
    > >
    > > io=IO.popen("externalfilter")
    > > io.write(verybigstring)
    > > result=io.read

    >
    > You need at least two threads or non blocking IO. How about:
    >
    > IO.popen( "cat", "w+" ) do |io|
    > r = Thread.new(io) do |reader|
    > while ( line = reader.gets )
    > line.chomp!
    > $stdout.puts line
    > end
    > end
    >
    > (1..100000).each {|i| io.puts i}
    > r.join
    > end


    Thank you, this is what I need. Since I program in ruby, a 8-9 rows of
    code seems too difficult to write by myself. :-/

    Ferenc
     
    Ferenc Engard, Jan 18, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jona
    Replies:
    2
    Views:
    835
  2. Brian Elmegaard
    Replies:
    2
    Views:
    1,005
    Robert Amesz
    Sep 10, 2003
  3. ToMasz
    Replies:
    9
    Views:
    460
    Dennis Lee Bieber
    Jan 24, 2006
  4. bingbang
    Replies:
    1
    Views:
    131
    Nobody
    Sep 30, 2011
  5. RBCS
    Replies:
    1
    Views:
    147
    A. Sinan Unur
    Feb 14, 2006
Loading...

Share This Page