Is there a limit to os.popen()?

[Carl J. Van Arsdall]

I'm not sure the proper way to phrase the question, but let me try.

Basically, I'm working with a script where someone wrote:

kr = string.strip(os.popen('make kernelrelease').read())


And then searches kr to match a regular expression.

This seems to have been working, however lately when this line executes
I get a number of messages to stderr after several minutes of execution:

cat: write error: Broken pipe
cat: write error: Broken pipe
cat: write error: Broken pipe


I know the output from this make has been growing (make applies some
patches and the patch list is growing). Does os.popen() have some kind
of read buffer limit that i'm hitting which is causing things to break?




--

Carl J. Van Arsdall
(e-mail address removed)
Build and Release
MontaVista Software

 

[cdecarlo]

Hello,

I'm not 100% sure on this but to me it looks like there is a problem in
your make file. I would look in there first, see where 'cat' is
executed, I bet your problem will be around there.

Hope this helps,

Colin

 

[Carl J. Van Arsdall]

Hello,

I'm not 100% sure on this but to me it looks like there is a problem in
your make file. I would look in there first, see where 'cat' is
executed, I bet your problem will be around there.

Hope this helps,

Colin

Well, running the make on the command line seems to work just fine, no
errors at all. I, in fact, get the results I expect with no error
messages printed out. I thought maybe something might have used cat as
a utility, thanks, i'll keep on it.

-c



--

Carl J. Van Arsdall
(e-mail address removed)
Build and Release
MontaVista Software

 

[Lawrence D'Oliveiro]

Well, running the make on the command line seems to work just fine, no
errors at all.

What about running it as

make kernelrelease | cat

This way the output goes to a pipe, which is what happens when it's
called from your script. Do you see those "broken pipe" messages after a
few minutes in this case?

 

[Carl J. Van Arsdall]

What about running it as

make kernelrelease | cat

This way the output goes to a pipe, which is what happens when it's
called from your script. Do you see those "broken pipe" messages after a
few minutes in this case?

Alright, so I tried that line in the interpreter and got the same
problem. So let me quickly state what I know, and what I'm starting to
infer.

Anyhow, running make on the command line gives me no problems. There
are cats all over the makefile, I checked, but I don't seem to have any
problems on the command line. Now, when attempting to run this make in
python things seem to go alright, except I get tons of errors dumped to
the screen, its very confusing. I don't know too much about pipes
though or how exactly to go about debugging this. Could it be something
like, make's output is piped correctly but pipes used by cat within the
makefile get screwed up inside python somehow?

-c




--

Carl J. Van Arsdall
(e-mail address removed)
Build and Release
MontaVista Software

 

[sreekant]

kr = string.strip(os.popen('make kernelrelease').read())

If make is being executed, I presume that the python script has rw
access to the location. How about

x=os.system("make kernelrelease > my_report 2>&1 ")
kr=open("my_report").read()


Just a blind throw.

Good luck
sree

 

[Dennis Lee Bieber]

Less than a hypothesis, merely a conjecture...

kr = string.strip(os.popen('make kernelrelease').read())

What happens if the SINGLE read operation only returns a single
buffer of data, and more is generated via "make" after... I've not used
popen/read enough to know if it is possible to need more than one read()
call to get all the data. Or could it be catching/returning when one of
those "cat" calls completes (some sort of EOF getting passed through
make into the read(), such that the Python program closes the pipe and
continues -- then the make tries more output)
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Lawrence D'Oliveiro]

Alright, so I tried that line in the interpreter and got the same
problem.

No, I wanted you to try it from a bash command prompt, to try to
simulate the environment when the make command on its own is invoked via
popen.