A little help with child processes.

[Rech]

Hi,
I need a little help here managing child processes in Python. I'm not so
skilled in system programming so I hope you can give me some good
suggestions.

I have a very CPU and memory intensive task that has to be repeated many
times (with different input parameters). So I've thought to write a
Python script that create a child process, wait for it to finish and
then starts again another child with different parameters.

The children processes will save all the results to the disk (using
cPickle module), so the parent process has to wait each child only and
then starts the next one. The problem is that I can't work out a
solution. Sorry, but system programming is not my job.

Any suggestions about how to realize that? A skeleton of the script will
just suffice.


Thanks in advance,
Andrea.

 

[Larry Bates]

Unless you are going to start more than one child
process in parallel and there is enough I/O to
make it worthwhile, there's no reason for using
child processes at all. Just program the application
as a single loop. You can't speed up CPU bound
applications with child processes.

HTH,
Larry Bates
Syscon, Inc.

 

[Rech]

Yes I know, but the CPU-intensive tasks eat a lot of memory and for
weird reasons they don't free it (I know bad programming, but re-writing
all the code is impratical at this point). Running them in child
processes solves the problem because when a child dies it frees all the
allocated memory.

Andrea.