K
Kyle
I wanted to share a Python based project which I've been working on.
Python Integrated Parallel Pipeline EnviRonment (PIPPER) is an MPI
based programming environment that works much like an OpenMP on Python
code. It is designed to create a python programming environment where
parallel computations on a set of distributed memory processors (you
may call it a cluster, or a Beowulf cluster) is easy to accomplish.
The idea is to make writing parallel code easy in order to promote
rapid development of script based distributed calculations.
There are tools, such as MPI or PVM that help with communicating
between processes running on different machines, but most people are
quickly scared off by the additional complexity in programming.
PIPPER eliminates this barrier to entry by automating the process of
data passing and job scheduling.
Most importantly is that there is no code 'lock-in'. PIPPER works as
a pre-parser and is designed to be completely backward compatible with
a single CPU python environment. If you write a script for PIPPER, it
will still work on systems that don't have PIPPER installed.
You can find the source code and documentation at http://pipper.sourceforge.net
A 'Hello Work' example of PIPPER code:
#!/usr/bin/python
import sys
import os
def do_call(x,y):
print "Hello World", x, y, os.getpid()
if __name__ == '__pipper_main__':
a_range = range( int(sys.argv[1]) )
#pragma pipper_start
for a in a_range :
for b in a_range :
do_call(a,b)
#pragma pipper_end
Python Integrated Parallel Pipeline EnviRonment (PIPPER) is an MPI
based programming environment that works much like an OpenMP on Python
code. It is designed to create a python programming environment where
parallel computations on a set of distributed memory processors (you
may call it a cluster, or a Beowulf cluster) is easy to accomplish.
The idea is to make writing parallel code easy in order to promote
rapid development of script based distributed calculations.
There are tools, such as MPI or PVM that help with communicating
between processes running on different machines, but most people are
quickly scared off by the additional complexity in programming.
PIPPER eliminates this barrier to entry by automating the process of
data passing and job scheduling.
Most importantly is that there is no code 'lock-in'. PIPPER works as
a pre-parser and is designed to be completely backward compatible with
a single CPU python environment. If you write a script for PIPPER, it
will still work on systems that don't have PIPPER installed.
You can find the source code and documentation at http://pipper.sourceforge.net
A 'Hello Work' example of PIPPER code:
#!/usr/bin/python
import sys
import os
def do_call(x,y):
print "Hello World", x, y, os.getpid()
if __name__ == '__pipper_main__':
a_range = range( int(sys.argv[1]) )
#pragma pipper_start
for a in a_range :
for b in a_range :
do_call(a,b)
#pragma pipper_end