popen2 trouble

[Diez B. Roggisch]

Hi,

I'm using the popen2 module to communicate with the crm114 text
classificator. First I used to create a subprocess, feed the text to
classify to it, close the crm's stdin file and read the results. Easy
enough.

However, the crm114 is a slow beast, and I try to increase its performanc by
using it in the "window"-mode. That means that the text fed to the
subprocess looks like this:

---START---
including bfast
---STOP---

The start/stop is detected by the crm114, and everything inbetween is
subject to classification. This simple protocol allows multiple
classifications without terminating the crm114 subprocess in between.

When performing this in my console window, everything works as expected. I
can paste several blocks of text that look like the above one, and the
crm114 constantly outputs classification results.

But when I try it from python like this


out, inn, err = _.get_crminstance(path, _.classify_program)

t = '%s\n%s\n%s\n' % (_.START, text, _.STOP)
logger.debug(t)
inn.write(t)
inn.flush()
#inn.close()

res = out.read().strip()
e = err.read().strip()

my program hangs. Uncommenting the inn.close() helps somewhat as then the
out.read() returns, but only after a minute or so. And I don't want to
close it anyway, because I want to use it for the next classification.

Any suggestions? I know that there are subtle differences between files and
tenminals, but I'm no unix expert, so I'm somewtaht lost.

 

[Armin Rigo]

Hello Diez,

When performing this in my console window, everything works as expected. I
can paste several blocks of text that look like the above one, and the
crm114 constantly outputs classification results.

Try using the 'bufsize' optional argument in the call to popen2. Set it
to 0 to disable all buffering, if you are writing in large chunks (e.g.
calling inn.write(bigstring)).


Armin

 

[Diez B. Roggisch]

Try using the 'bufsize' optional argument in the call to popen2. Set it

to 0 to disable all buffering, if you are writing in large chunks (e.g.
calling inn.write(bigstring)).

That didn't work - now I'm using pexpect, which simulates a tty.