No speedup on multi-processor machine?

[danfan1981]

Hi,
I am using Python Thread library for my parallel processing course
project. I am doing matrix convolution on a multi-processor machine
running Solaris. I just found out that no speed-up is obtained with
threading. It is probably because of something called GIL in Python.
How can I get around
that GIL and get speed-up?
Thanks in advance.
Daniel

 

[Dennis Lee Bieber]

Hi,
I am using Python Thread library for my parallel processing course
project. I am doing matrix convolution on a multi-processor machine
running Solaris. I just found out that no speed-up is obtained with
threading. It is probably because of something called GIL in Python.
How can I get around
that GIL and get speed-up?

Threading in Python is optimized for I/O bound processing, wherein
the threads spend most of their lives sleeping (blocked waiting for some
I/O to complete, or some lock/event/condition to change state). It is
not optimized for parallel number crunching.

Options:

Don't use the common "CPython" (eg, the Python built from C-language
source using the C-runtime library). Jython (a version that runs on the
JVM, using Java libraries) may not be afflicted with the GIL.

I don't recall if "Stackless" has the GIL...

Convert the deep math operations (ie, the math work done with each
cell and neighbors) to a compiled C-extension module and make sure it
releases the GIL on entry, and reclaims the GIL on exit. This would
allow the Python interpreter to get control back, while one thread is
number crunching, and start another thread. Otherwise you not only have
the GIL-serialized number crunching time, but you've also added the
overhead of thread context switches (that is, the threaded version runs
SLOWER than straight nested for loops).
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Fuzzyman]

Threading in Python is optimized for I/O bound processing, wherein
the threads spend most of their lives sleeping (blocked waiting for some
I/O to complete, or some lock/event/condition to change state). It is
not optimized for parallel number crunching.

Options:

Don't use the common "CPython" (eg, the Python built from C-language
source using the C-runtime library). Jython (a version that runs on the
JVM, using Java libraries) may not be afflicted with the GIL.

IronPython is *definitely* not restricted by the GIL.

Fuzzyman
http://www.voidspace.org.uk/ironpython/index.shtml

 

[Caleb Hattingh]

Hi,
I am using Python Thread library for my parallel processing course
project. I am doing matrix convolution on a multi-processor machine
running Solaris. I just found out that no speed-up is obtained with
threading. It is probably because of something called GIL in Python.
How can I get around
that GIL and get speed-up?
Thanks in advance.
Daniel

Perhaps try

http://www.parallelpython.com/

or

http://www.its.caltech.edu/~astraw/seppo.html

Caleb

 

[John Nagle]

If you're actually doing the convolution in Python, you need
optimization before you need more CPUs. There's a numerics library
for Python called NumPy, but it doesn't have a convolution function,
although it has an FFT, which may be useful.

But this is just homework. Do something reasonable and turn it
in. A high performance solution to this problem is probably more
work than it's worth.

John Nagle

 

[Neil Hodgson]

Fuzzyman:

IronPython is *definitely* not restricted by the GIL.

IronPython is currently mostly slower than CPython although the
particular problem should be tested to see if IronPython helps.

Some recent benchmarks between IronPython and CPython:
http://sparcs.kaist.ac.kr/~tinuviel/pybench/

Neil

 

[Fuzzyman]

Fuzzyman:


IronPython is currently mostly slower than CPython although the
particular problem should be tested to see if IronPython helps.

Some recent benchmarks between IronPython and CPython:http://sparcs.kaist.ac.kr/~tinuviel/pybench/

Yep, I've seen that.

http://www.voidspace.org.uk/python/weblog/arch_d7_2007_04_21.shtml#e687

It's not entirely slower, but mainly slower.

OTOH, if you split a job into two threads on a twin-core machine,
IronPython performance will increase dramatically whilst CPython will
go down...

Additionally, extending IronPython from C# is orders of magnitude
easier than extending CPython from C.

Fuzzyman
http://www.voidspace.org.uk/ironpython/index.shtml

 

[Grant Edwards]

I am using Python Thread library for my parallel processing
course project. I am doing matrix convolution on a
multi-processor machine running Solaris. I just found out that
no speed-up is obtained with threading. It is probably because
of something called GIL in Python. How can I get around that
GIL and get speed-up?

Not much of a parallel processing course. It appears they
haven't taught you anything about parallel processing.

http://wiki.python.org/moin/ParallelProcessing

 

[Robert Kern]

There's a numerics library
for Python called NumPy, but it doesn't have a convolution function,
although it has an FFT, which may be useful.

In [1]: from numpy import *

In [2]: convolve?
Type: function
Base Class: <type 'function'>
Namespace: Interactive
File:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy-1.0.3.dev3714-py2.5-macosx-10.3-fat.egg/numpy/core/numeric.py
Definition: convolve(a, v, mode='full')
Docstring:
Returns the discrete, linear convolution of 1-D sequences a and v; mode
can be 'valid', 'same', or 'full' to specify size of the resulting sequence.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[danfan1981]

Thanks guys. But I think IronPython only works on Windows machine, but
I am using a Sun machine. I was suggested to use Jython, which can run
on Sun. But I need to use Numpy for matrix operations, which is only
available to CPython.

 

[Fuzzyman]

Fuzzyman:


IronPython is currently mostly slower than CPython although the
particular problem should be tested to see if IronPython helps.

Some recent benchmarks between IronPython and CPython:http://sparcs.kaist.ac.kr/~tinuviel/pybench/

Not when running on the .NET platform, there is not a lot of
difference overall:

http://www.voidspace.org.uk/python/weblog/arch_d7_2007_04_21.shtml#e688

Fuzzyman
http://www.voidspace.org.uk/ironpython/index.shtml

 

[Cameron Laird]

If you're actually doing the convolution in Python, you need
optimization before you need more CPUs. There's a numerics library
for Python called NumPy, but it doesn't have a convolution function,
although it has an FFT, which may be useful.

But this is just homework. Do something reasonable and turn it
in. A high performance solution to this problem is probably more
work than it's worth.

John Nagle

Along with the excellent advice given by Dennis, John, and the
rest, please be aware that *process*-level parallelization of
a problem sometimes is a benefit. As already recommended, <URL:
http://wiki.python.org/moin/ParallelProcessing > touches on most
of the pertinent concepts.

 

[Stefan Behnel]

Thanks guys. But I think IronPython only works on Windows machine, but
I am using a Sun machine.

Isn't there a mono port for Sun?

Stefan

 

[Klaas]

Additionally, extending IronPython from C# is orders of magnitude
easier than extending CPython from C.

Given the existence of Pyrex, that statement is pretty difficult to
substantiate.

-Mike

 

[Fuzzyman]

Given the existence of Pyrex, that statement is pretty difficult to
substantiate.

With Pyrex you still need to do memory management for non-Python
types. Additionally compiling C# from within IronPython is *very*easy.

C# is a much easier language to use than C - even with the help of
Pyrex. You use types defined in C# *natively* within IronPython.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml

 

[Jorgen Grahn]

Given the existence of Pyrex, that statement is pretty difficult to
substantiate.

IMHO, you don't even need Pyrex. When you've done it manually once,
you have the blueprint for your other extension modules. Most of the
work should be normal "fake OOP in C" programming.

Doing it for the first time is another matter, however.

/Jorgen