Two way communication with the command shell (IO.popen?)

[James Smith]

Hi,

I would basically like to be able to run a ruby program from my rails
application and be able to read from it and write to it accordingly.

OK, i am currently using:

IO.popen("ruby ruby_program.rb", "r+") do |f|
@read = f.read
@write = f.write #this is where i need help
end

say if the external ruby program was as follows:

0 def readWrite
1 puts "printing.."
2 @string = gets
3 end
4 readWrite

I can currently read (in example) "printing..", however when it reaches
line 2 of the ruby program, i can't seem to write to it, and my rails
program just waits. I need to use something that says "read from program
until it is waiting for input, and then write to it" (also, when writing
to the program, how do i emulate the user pressing enter - \n? )

Thanks

 

[ara.t.howard]

Hi,

I would basically like to be able to run a ruby program from my rails
application and be able to read from it and write to it accordingly.

OK, i am currently using:

IO.popen("ruby ruby_program.rb", "r+") do |f|
@read = f.read
@write = f.write #this is where i need help
end

say if the external ruby program was as follows:

0 def readWrite
1 puts "printing.."
2 @string = gets
3 end
4 readWrite

I can currently read (in example) "printing..", however when it reaches
line 2 of the ruby program, i can't seem to write to it, and my rails
program just waits. I need to use something that says "read from program
until it is waiting for input, and then write to it" (also, when writing
to the program, how do i emulate the user pressing enter - \n? )

Thanks

this is a terrifically bad idea, but here's how you'd do it


harp:~ > cat a.rb
IO.popen("ruby ruby_program.rb", "r+") do |f|
@read = f.gets
p @read
@write = f.puts 42
end

harp:~ > cat ruby_program.rb
STDOUT.sync = true

def readWrite
puts "printing.."
@string = gets
end

readWrite


harp:~ > ruby a.rb
"printing..\n"


you were calling read, which reads till the end of input and that's not going
to occur until the program exits.

i highly reccomend post exatly what you are trying to accomplish so we can try
to come up with a safer solution: spawning processes from a web-app like this
will cause problems unless you are __very__ comfortable with ipc.

cheers.

-a

 

[James Smith]

this is a terrifically bad idea, but here's how you'd do it


harp:~ > cat a.rb
IO.popen("ruby ruby_program.rb", "r+") do |f|
@read = f.gets
p @read
@write = f.puts 42
end

harp:~ > cat ruby_program.rb
STDOUT.sync = true

def readWrite
puts "printing.."
@string = gets
end

readWrite


harp:~ > ruby a.rb
"printing..\n"


you were calling read, which reads till the end of input and that's not
going
to occur until the program exits.

i highly reccomend post exatly what you are trying to accomplish so we
can try
to come up with a safer solution: spawning processes from a web-app like
this
will cause problems unless you are __very__ comfortable with ipc.

cheers.

-a

Thank you very much for your comments. I am basically writing a web
application where users can write and save ruby programs; then run them
on the server. I am at the stage where the user can save their programs
onto the server, however i now need to be able to input and recieve
output from them, as well as error messages. I am fairly certain i will
need to create a new thread for each of these processes, and probably
have a timeout on each of them.

Any help on this would be very much appreciated. Are there any sources
that you'd recommend me looking at?

 

[ara.t.howard]

Thank you very much for your comments. I am basically writing a web
application where users can write and save ruby programs; then run them on
the server. I am at the stage where the user can save their programs onto
the server, however i now need to be able to input and recieve output from
them, as well as error messages. I am fairly certain i will need to create a
new thread for each of these processes, and probably have a timeout on each
of them.

ah. you really need to do be able to do this then

Any help on this would be very much appreciated. Are there any sources that
you'd recommend me looking at?

sure, i think this already does everything you need:

harp:~ > cat a.rb
require 'open4'

def gen_ruby_program path = 'b.rb'
open path, 'w' do |f|
f.write <<-src
puts "what's your name?"
name = gets
puts("hi %s!" % name)
src
end
path
end

program = gen_ruby_program

stdin, stdout, stderr = "zaphod", "", ""
timeout = 42

status =
open4.spawn "ruby #{ program }", :stdin=>stdin, :stdout=>stdout, :stderr=>stderr, :timeout=>timeout

puts "status: #{ status.inspect }"
puts "stdout: #{ stdout.inspect }"
puts "stderr: #{ stderr.inspect }"
harp:~ > ruby a.rb
status: #<Process::Status: pid=8456,exited(0)>
stdout: "what's your name?\nhi zaphod!\n"
stderr: ""


if you are on windows you'll want to use systemu instead of open4.

-a

 

[James Smith]

ah. you really need to do be able to do this then


sure, i think this already does everything you need:

harp:~ > cat a.rb
require 'open4'

def gen_ruby_program path = 'b.rb'
open path, 'w' do |f|
f.write <<-src
puts "what's your name?"
name = gets
puts("hi %s!" % name)
src
end
path
end

program = gen_ruby_program

stdin, stdout, stderr = "zaphod", "", ""
timeout = 42

status =
open4.spawn "ruby #{ program }", :stdin=>stdin, :stdout=>stdout,
:stderr=>stderr, :timeout=>timeout

puts "status: #{ status.inspect }"
puts "stdout: #{ stdout.inspect }"
puts "stderr: #{ stderr.inspect }"
harp:~ > ruby a.rb
status: #<Process::Status: pid=8456,exited(0)>
stdout: "what's your name?\nhi zaphod!\n"
stderr: ""


if you are on windows you'll want to use systemu instead of open4.

-a

Hello again,

Thanks once again for your comments. OK i started off by installing
'systemu' (i am on windows). But i am struggling to understand your
code, and i can't work out if the external ruby program is included or
not. What i would like is for the user program to be completely seperate
from the rails, so the user doesn't know anything about the rails
program. Obviously when writing the users ruby program to file i can add
some lines of code if necessary. Then i would just like the rails
program to execute the users program (using its own thread??). This will
mean it reading any output from the users program and if it is waiting
for input, notify the rails program (eventually i will now display a
dialog allowing the user to input) and the rails program writes to it.
Then exit the thread after a timeout (if necessary) and print any
errors. So i just it to behave as though i was running the program on
the command line.
If this is wat the code above does, then could you please just explain
it to me please. Thank you once again.

jamie

 

[James Smith]

OK, keeping it simple I am basically using the following code:

IO.popen("ruby my_program.rb", "r+") do |f|

while(process hasn't finished) # need ruby code for this

read = f.gets
while(read != nil)
print read
read = f.gets
end

while(waiting for input) #need ruby code for this
write "xyz"
end

end
end


- I need to be able to test whether or not the ruby program is waiting
for input
- I would also like to be able to test whether or not the program has
finished running

Can anybody help me please.

 

[Patrick Hurley]

OK, keeping it simple I am basically using the following code:

IO.popen("ruby my_program.rb", "r+") do |f|

while(process hasn't finished) # need ruby code for this

read = f.gets
while(read != nil)
print read
read = f.gets
end

while(waiting for input) #need ruby code for this
write "xyz"
end

end
end


- I need to be able to test whether or not the ruby program is waiting
for input
- I would also like to be able to test whether or not the program has
finished running

Can anybody help me please.

This varies by platform a little, what os are you using?

pth

 

[James Smith]

This varies by platform a little, what os are you using?

pth

Windows. Any help would be great.. any ideas?

 

[James Smith]

Whats happened to my helpers?

 

[Matthew Williams]

- I need to be able to test whether or not the ruby program is waiting

for input
- I would also like to be able to test whether or not the program has
finished running

I'm not sure how to do this directly, or if this is even possible in
Ruby, however what you could do is to insert wrapper code into your user
programs. This code could override existing methods and take over the
input and output for the user programs. When the programs were then
waiting for input from the user, they would then call the wrapper code
and this could then indicate to the rails program that the users Ruby
program was waiting on input.

hope this helps, matt

 

[Matthew Williams]

I'm not sure how to do this directly, or if this is even possible in
Ruby,

I refuse to accept that this is not possible in Ruby (although I still
am unable to do this neatly). Can someone PLEASE shed some light, this
has caused me soo much anguish..

..in case you don't want to read though the posts then i would like to
know the following:

- I am running a Ruby program from my Rails application using:

thread = Thread.new do
IO.popen("ruby my_program.rb", "r+") do |pipe|
while !pipe.eof
NOW - I NEED TO KNOW HOW TO CHECK IF THE RUBY PROGRAM IS WAITING FOR INPUT

..i can read from it by reading and then checking if this returns 'nil';
if it is not equal to 'nil' then i will print it. i.e. i am trying to
simulate the behaviour of a standard command prompt. BUT i cannot check
if i need to write something to the program.

NOTE: all Ruby programs have "STDOUT.sync = true" in them.

Any ideas?

 

[Hugh Sasse]

I refuse to accept that this is not possible in Ruby (although I still

Is what you want possible in another language? It might help
clarify what you need...

am unable to do this neatly). Can someone PLEASE shed some light, this
has caused me soo much anguish.. [...]
- I am running a Ruby program from my Rails application using:

thread = Thread.new do
IO.popen("ruby my_program.rb", "r+") do |pipe|
while !pipe.eof
NOW - I NEED TO KNOW HOW TO CHECK IF THE RUBY PROGRAM IS WAITING FOR INPUT

..i can read from it by reading and then checking if this returns 'nil';

You mean this thread reading from pipe? Having two programs here
confuses the issue somewhat.

if it is not equal to 'nil' then i will print it. i.e. i am trying to
simulate the behaviour of a standard command prompt. BUT i cannot check
if i need to write something to the program.

Well, I think that's the same as interactive use: how do you know a
program needs input, and isn't busy calculating The Answer? If it
is running as a separate process, I don't see how you could interrogate
it from this process. It would have to prompt you for input, use
some other channel (sockets?) to tell you, etc.
Because you have done "ruby my_program.rb", that's in a shell running
another copy of ruby. You can't really get at the internals even from
the shell. Even Expect scripts rely on prompting.

If the read from my_program.rb is nonblocking, when there's no data
it will not be reading any more.

 

[Matthew Williams]

Is what you want possible in another language? It might help
clarify what you need...

I'm not sure, have never had to do this before

You mean this thread reading from pipe? Having two programs here
confuses the issue somewhat.

yes - Thread created in Rails app is running external Roby program - and
needs to display output and know when its waiting for input

If it
is running as a separate process, I don't see how you could interrogate
it from this process. It would have to prompt you for input, use
some other channel (sockets?) to tell you, etc.
Because you have done "ruby my_program.rb", that's in a shell running
another copy of ruby. You can't really get at the internals even from
the shell. Even Expect scripts rely on prompting.

So your saying that, when running a program from the command prompt, it
isn't possible to find out if the program is waiting for input from the
user or is just executing part of the program?

So how would you advise i go about trying to determine whether or not
the program is waiting for input? Should i wait for a certain time and
then try and write to the program?

 

[Hugh Sasse]

I'm not sure, have never had to do this before

Oh well, it was worth a try...

yes - Thread created in Rails app is running external Roby program - and
needs to display output and know when its waiting for input

OK, with you so far then...

So your saying that, when running a program from the command prompt, it
isn't possible to find out if the program is waiting for input from the
user or is just executing part of the program?

As a human, no, one cannot. Not in my experience, anyway. I suppose
one might look at the output of ps and see if it talks about IO, but
then you don't know if it is scribbling exciting things on the disk
or doing the IO that connects to you.

So how would you advise i go about trying to determine whether or not
the program is waiting for input? Should i wait for a certain time and
then try and write to the program?

Well, if you are invoking it as above, you have access to the source
so you can get it to prompt you for input.

There are exits North, East, and West.
OK>

But you have to *know* that the "OK> " is a prompt.
require 'semiotics' #
Actually, apart from Tcl escaping, it's mostly the interaction
that is the time consuming part of writing Expect scripts.Hugh

 

[ara.t.howard]

So your saying that, when running a program from the command prompt, it
isn't possible to find out if the program is waiting for input from the
user or is just executing part of the program?

yes. it's impossible on every platform in every language to do in a reliable
and generic way.

So how would you advise i go about trying to determine whether or not the
program is waiting for input? Should i wait for a certain time and then try
and write to the program?

can you state __exactly__ what you are trying to accomplish please: i think
there is a fundemental problem with your design which you are not considering:

1) it might always be possible to send __all__ input up front. in fact,
this is the normal case, in general it's not needed to know __when__ a
program is waiting for input, only that it will be at some point and you
then simply send it all up from. eg

IO.popen( cmd, 'r+' ) do |pipe|
pipe.write the_precalculated_stdin
pipe.close_write
pipe.read
end

this is, by far, the normal means of bi-directional communication with
proccess.


2) the input you send depends on the output of the program, in that case you
will have this sort of logic

IO.popen( cmd, 'r+' ) do |pipe|
pipe.each do |stdout|
case stdout
when /foo/
pipe.write 'you need foo'
when /bar
pipe.write 'you need bar'
else
pipe.write 'you need foobar'
end
end
end

it seems like this problems your are having result from not realizing that
input into any program, in any platform, is buffered by the os and your
program can merely send it when it finds it convenient - maybe i'm wrong, but
it's extrememly hard for me to imagine a scenario where one would __need__ to
know when a program required input - especially if one considers
multi-threaded or forking programs which may require multiple inputs at once.

regards.

-a

 

[Matthew Williams]

I suppose
one might look at the output of ps and see if it talks about IO, but
then you don't know if it is scribbling exciting things on the disk
or doing the IO that connects to you.

I just thought this must be possible because the command prompt knows
that input is expected because it allows characters to be entered from
the keyboard, and when it is executing it blocks all input..

Well, if you are invoking it as above, you have access to the source
so you can get it to prompt you for input.
OK

There are exits North, East, and West.
OK>

But you have to *know* that the "OK> " is a prompt.
require 'semiotics' #
Actually, apart from Tcl escaping, it's mostly the interaction
that is the time consuming part of writing Expect scripts.

You've lost me here.. i'm not a very experienced user of Ruby and am not
sure what 'Tcl escaping' or 'expect scripts' are. Could you please
explain?

Couldn't I just override the 'gets' method in the Ruby program and get
the new method to output something like 'WRITE', and then the Rails
program could listen for this command, and when it gets it write to the
program? Is 'gets' the only method that prompts input from the user?

 

[Hugh Sasse]

I just thought this must be possible because the command prompt knows
that input is expected because it allows characters to be entered from
the keyboard, and when it is executing it blocks all input..

Are you sure it is not actually held in a buffer? What platform
are you on, anyway?

You've lost me here.. i'm not a very experienced user of Ruby and am not
sure what 'Tcl escaping' or 'expect scripts' are. Could you please
explain?

Tcl is another language and is the basis for Don Libes' Expect package,
which handles this interaction pretty well. Tcl has a system of quoting
things with [ something ] or { something } for immediate or delayed
interpretation, which confuses me just too often! Ruby has Expect
facilities, but I've not re-examined them very recently so am not up
to speed as to how well they compare with Expect's facilities now.

Should you wish to explore those other programs, which are not Ruby, then
http://www.tcl.tk/
and
http://expect.nist.gov/
are good starting places.

Couldn't I just override the 'gets' method in the Ruby program and get
the new method to output something like 'WRITE', and then the Rails
program could listen for this command, and when it gets it write to the
program? Is 'gets' the only method that prompts input from the user?

Yes, but as Ara said, buffering will do horrible things. You won't just
get the word WRITE on it's own, almost certainly. If WRITE is
all you need, then you don't need to know when to send the data to
the other program. See his reply for more about that.HTH
Hugh

 

[Matthew Williams]

can you state __exactly__ what you are trying to accomplish please: i
think there is a fundemental problem with your design

yes.
I am basically writing a web application in Rails where users can write
and save ruby programs (like a Ruby IDE); then run them on the server.
Obviously these programs are going to be random. I am at the stage where
the user can write and save their programs onto the server, I can then
run the program using:

IO.popen("ruby ruby_program.rb", "r+") do |pipe| while !pipe.eof

If there is something to read from ruby_program then it can be read
using:

read = pipe.gets
while read != nil
puts read <------------eventually display this in a console, print for
now read = pipe.gets end

However the next stage is to detect if the program is waiting for input.
I understand this is not possible, but am not sure how to go about it..
I have access to the users programs, so could insert some code into them
overriding the 'gets' method, and could handle this by outputting some
string i.e. 'WRITE', which the rails app could read and know then to
prompt the user for input..

Example: I insert into every user program the following:

STDOUT.sync = true

def gets
write = puts "WAIT" #not sure how this would get written input from
puts "written in: #{write}" #rails program
end

which is appended to this example user program, for example:

def something
puts "output"
input = gets
end
something

Plausible? Are there any other methods that prompt input from the user
besides 'gets' that I would need to override, or is there any other way
of calling 'gets' using its class name that would bypass the overridden
method?

I also need to get any errors that occur and display them to the user..

I'm on a windows platform, and any help is very much appreciated.. Thank
you

 

[ara.t.howard]

I'm on a windows platform, and any help is very much appreciated.. Thank
you

ah - you cannot possible do it then. threads and io and webservers will 100%
sure hang your web server. if you __really__ want to do this you'll have to
move to *nix.

sorry!

-a