SIMD powered Python

[Bytter]

Hi!

Is there any I&D ongoing about using SIMD [1] instructions, like SSE
[2], to speed up Python, especially regarding functional features,
like list comprehension, map and reduce, etc.. ?

Best regards,

Hugo Ferreira

 

[Marc 'BlackJack' Rintsch]

Is there any I&D ongoing about using SIMD [1] instructions, like SSE
[2], to speed up Python, especially regarding functional features,
like list comprehension, map and reduce, etc.. ?

SIMD instruction sets know about "low level" data types, Python is about
objects. `map()`, `reduce()`, list comprehension work on arbitrary
iterables so how do you expect SIMD instructions handle this? Even simple
lists contain objects and those don't have to be of the same type.

Ciao,
Marc 'BlackJack' Rintsch

 

[Bytter]

Hi...

True... But maybe in NumPy arrays that would be more feasible...?

Cheers.

Hugo Ferreira

Marc 'BlackJack' Rintsch escreveu:

Is there any I&D ongoing about using SIMD [1] instructions, like SSE
[2], to speed up Python, especially regarding functional features,
like list comprehension, map and reduce, etc.. ?

SIMD instruction sets know about "low level" data types, Python is about
objects. `map()`, `reduce()`, list comprehension work on arbitrary
iterables so how do you expect SIMD instructions handle this? Even simple
lists contain objects and those don't have to be of the same type.

Ciao,
Marc 'BlackJack' Rintsch

 

[Marc 'BlackJack' Rintsch]

Marc 'BlackJack' Rintsch escreveu:

Is there any I&D ongoing about using SIMD [1] instructions, like SSE
[2], to speed up Python, especially regarding functional features,
like list comprehension, map and reduce, etc.. ?

SIMD instruction sets know about "low level" data types, Python is about
objects. `map()`, `reduce()`, list comprehension work on arbitrary
iterables so how do you expect SIMD instructions handle this? Even simple
lists contain objects and those don't have to be of the same type.

True... But maybe in NumPy arrays that would be more feasible...?

Yes but that's in external libraries and not in the Python interpreter.
So it won't speed up Python code like list comprehensions but "just" calls
to external functions written in C, Fortran or assembler if those make use
of SIMD instructions.

Ciao,
Marc 'BlackJack' Rintsch

 

[Paul Rubin]

Yes but that's in external libraries and not in the Python interpreter.
So it won't speed up Python code like list comprehensions but "just" calls
to external functions written in C, Fortran or assembler if those make use
of SIMD instructions.

Right, Python has such poor control over side effects that it has not
much chance of parallelizing stuff like list comprehensions in
general. Maybe there's some chance of doing it for some special cases
with RPython.

See http://www.google.com/search?q="nested+data+parallelism"
for what's happening with some other languages.