launching python in parallel

[Toon Knapen]

Dear all,

I'm looking into launching python in parallel using MPI. There are many
projects already doing this but I would like to understand how this can
be done in a portable way.

For instance, is it possible to launch myscript.py (which calls MPI_Init
through an extension module) like:

mpirun -np 2 /path/to/python myscript.py

Using mpichgm this works, using stock mpich-1.2.5.2 (ch_p4) however,
this does not work.

I have been looking into the implemenation of p4 and the behavious (as
described above) is understandable. The reason is that p4 creates the
remote processes _in_ the MPI_Init of the master. The MPI_Init performs
and execlp specifying the name of the executable only. Once the 'slave'
is launched and also calls the MPI_Init, the command-line arguments are
passed on to the slave. But the slave will thus only launch python, not
the myscript.py. Therefore it will never call MPI_Init.

I would like to know how other parallel python projects handle this.

Thanks in advance,

Toon Knapen