How can I know how much to read from a subprocess

G

Guest

Hello,

I want to write a terminal program in pygtk. It will run a subprocess,
display everything it writes in its standard output and standard
error, and let the user write text into its standard input.

The question is, how can I know if the process wrote something to its
output, and how much it wrote? I can't just call read(), since it will
block my process.

Thanks,
Noam
 
G

Guest

Ok, I could have researched this before posting, but here's an
explanation how to do it with twisted:

http://unpythonic.blogspot.com/2007/08/spawning-subprocess-with-pygtk-using.html

It seems that another solution is gobject.io_add_watch, but I don't
see how it tells me how much I can read from the file - if I don't
know that, I won't know the argument to give to the read() method in
order to get all the data:

http://www.pygtk.org/docs/pygobject/gobject-functions.html#function-gobject--io-add-watch

Noam
 
A

A.T.Hofkamp

It seems that another solution is gobject.io_add_watch, but I don't
see how it tells me how much I can read from the file - if I don't
know that, I won't know the argument to give to the read() method in
order to get all the data:

http://www.pygtk.org/docs/pygobject/gobject-functions.html#function-gobject--io-add-watch

Usually, gobject only tells you that data is there (that is all it knows).
Therefore a read(1) should be safe.
If that is too slow, consider os.read() which reads all data available (afaik,
never tried it myself).

Albert
 
G

Guest

Usually, gobject only tells you that data is there (that is all it knows).
Therefore a read(1) should be safe.

But even if it's fast enough, how do you know how many times you
should call read(1)? If you do it too much, you'll be blocked until
more output is available.
If that is too slow, consider os.read() which reads all data available (afaik,
never tried it myself).
I tried it now, and it blocks just like the normal file.read().

Thanks,
Noam
 
G

Grant Edwards

But even if it's fast enough, how do you know how many times you
should call read(1)? If you do it too much, you'll be blocked until
more output is available.

You don't know. That's why you use non-blocking mode.
 
K

Karthik Gurusamy

Hello,

I want to write a terminal program in pygtk. It will run a subprocess,
display everything it writes in its standard output and standard
error, and let the user write text into its standard input.

The question is, how can I know if the process wrote something to its
output, and how much it wrote? I can't just call read(), since it will
block my process.

The solution is not simple. You may try a separate thread to do the
reading (so that you can afford to block that single thread). You can
also run into other I/O deadlocks with a simple solution (ie if you
try to feed the whole stdin before reading any stdout, your stdin
feeding can block because the process's stdout is full -- it's a bit
confusing but for large stdin/stdout, a deadlock is very much
possible).

Did you try subprocess.Popen.communicate() ? Under the covers it
should take care of th ese deadlock issues. It may read and buffer the
whole stdout/stderr, so don't use it if the process generates infinite/
very-large stdout/stderr.

If the process you launch needs two way communication, like an ssh/ftp
session (where the stdin depends on process's prior stdout), the only
viable solution is to simulate a human (e.g. pexpect module)

Karthik
 
A

A.T.Hofkamp

But even if it's fast enough, how do you know how many times you
should call read(1)? If you do it too much, you'll be blocked until
more output is available.

after reading 1 byte, wait for gobject again.

In other words, when gobject sends a notice, read 1 byte.
When there is more, gobject will tell you (probably very often :) ).

I tried it now, and it blocks just like the normal file.read().

So "os.read(handle.fileno(), 1000000)" blocks after gobject tells you there is data?
That is different than I heard, but you are probably correct, I cannot easily test this.


Albert
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,898
Latest member
BlairH7607

Latest Threads

Top