non blocking "i o", The deep freeze.

[guy.flowers]

Hi

Have a problem, Ill give some history to the problem and add a little
example code to start with
to see if anybody can help or if I am correct in what the problem is.

I have been looking on the newsgroups and have found lots of stuff on
the problem
but no solutions as of yet, will keep looking here to see if I can find
one that fits.

I created a script which runs a whole suite of regression tests, the
test exe which is
built daily, is created from c code, this changes from day to day as
stuff is added, and fixed, the exe is then used to run text files which
have lots of commands and data in, which produce lots of pretty
pictures on the screen, errors and warning output is also produced from
this, which I collect and create logs with. Up until recently I havn't
had any problems with my script, the main tests were run on a local
win32 machine with all the logs and test files all local, with
occasional runs on various different unix's and win64 platforms. We
have just recently had installed a big test and build rack full of
win32 (winxp) machines 3 of these are automated test machines which run
my script in a loop (it also builds the test exe and does one or two
other things), the test files are now held on a big central disk, which
was ment to save me coping data down from the network. The whole thing
works pretty well, a part from every so offen completely randomly the 3
machines seem to freeze (or to put in the terms of our sys admin it
trashes them).
The machines needs to be reset and the connection between the machines
and the big central disk is lost and needs to be reset as well.

According to the sys admin I need to be using, non blocking "i o" for
my file control, the code below shows what Im doing at the moment. Has
any one got any suggestions on what to do here, I kept everything
really simple so it would work across all platform and I thought I was
there,
but like with most things, its not and the code will probably get
really complex.
The code below is only a small snipet and I've made it simple for this
example but its the part I think where the freeze is happening.


output, input = popen2("cmd.exe")

input.write("set LI_DIR=" + MW_OPTS.GetOption("MainOpts","LI_DIR") +
"\n")
input.write("set LI\n")

# Writes commands to it.
input.write(mwtest_exe + mwtest_args + os.path.dirname(f_testlwc_str)
+ testlwc + "\n")
input.write("exit\n")

while 1:
text = output.readline()
if text:
OutputCount += 1
# Searches the line for errors, when found stops doing this.
f_Error_Code = FileLineQuickStat(text , f_Possible_Errs_dict ,
f_Error_Code)
# Records the line in a log.

LW_FLOG.WriteLogEntry("TEST_LOG",str(OutputCount),string.replace(text,"\n",""),1)
else:
LW_FLOG.FinishLog("TEST_LOG","")
# Returns the error code.
return f_Error_Code
break

TIA
Guy

 

[Fredrik Lundh]

The whole thing works pretty well, a part from every so offen completely
randomly the 3 machines seem to freeze (or to put in the terms of our sys
admin it trashes them).

thrashing?

that usually means that a process uses too much memory, thus causing the
system to spend too much time swapping to be able to do anything useful.

have you checked the process size during normal operations? is it stable, or
does it grow a little all the time?

According to the sys admin I need to be using, non blocking "i o" for
my file control

if the problem is thrashing, I'm not sure I see how that would help.

</F>