N
Neal Becker
I'm using multiprocessing as a poor man's batch system. It is working OK,
except that it doesn't seem to do load balancing quite right.
I have a 8-cpu machine. If I start, using multiprocessing pool, calling map
with more than 8 elements, it will start 8 processes. It seems, though,
that when some of them finish, it does not start others right away. The
load average may be a lot less than 8 but still additional processes are not
active.
Is there a way to use multiprocessing to achieve the effect I want? (I
really want a simple batch system, but without complex setup)
except that it doesn't seem to do load balancing quite right.
I have a 8-cpu machine. If I start, using multiprocessing pool, calling map
with more than 8 elements, it will start 8 processes. It seems, though,
that when some of them finish, it does not start others right away. The
load average may be a lot less than 8 but still additional processes are not
active.
Is there a way to use multiprocessing to achieve the effect I want? (I
really want a simple batch system, but without complex setup)