when "normal" parallel computations in CPython will be implemented at last?

[dmitrey]

hi all,
are there any information about upcoming availability of parallel
computations in CPython without modules like multiprocessing? I mean
something like parallel "for" loops, or, at least, something without
forking with copying huge amounts of RAM each time and possibility to
involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
with CPython due to GIL).

AFAIK in PyPy some progress have been done (
http://morepypy.blogspot.com/2012/06/stm-with-threads.html )

Thank you in advance, D.

 

[Thomas Jollans]

hi all,
are there any information about upcoming availability of parallel
computations in CPython without modules like multiprocessing? I mean
something like parallel "for" loops, or, at least, something without
forking with copying huge amounts of RAM each time and possibility to
involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
with CPython due to GIL).

AFAIK in PyPy some progress have been done (
http://morepypy.blogspot.com/2012/06/stm-with-threads.html )

As far as I can tell, there are no concrete plans to integrate
concurrency better, or get rid of the GIL, at the moment. To quote
http://wiki.python.org/moin/GlobalInterpreterLock

"""Getting rid of the GIL is an occasional topic on the python-dev
mailing list. No one has managed it yet."""

There are currently no open or accepted PEPs on the subject of concurrency.
http://www.python.org/dev/peps/

There is, of course, Stackless Python.
http://stackless.com/

 

[Thomas Jollans]

IronPython, sadly, lacks a python standard library.

Beg pardon?

https://github.com/IronLanguages/ma...al.LCA_RESTRICTED/Languages/IronPython/27/Lib

 

[Andrew Berg]

As far as I can tell, there are no concrete plans to integrate
concurrency better, or get rid of the GIL, at the moment. To quote
http://wiki.python.org/moin/GlobalInterpreterLock

"""Getting rid of the GIL is an occasional topic on the python-dev
mailing list. No one has managed it yet."""

There's also this, recently written by one of CPython's core devs:
http://python-notes.boredomandlazin...the-gil-is-more-important-than-fixing-unicode

 

[Thomas Jollans]

On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans <[email protected]




Beg pardon?

https://github.com/IronLanguages/ma...al.LCA_RESTRICTED/Languages/IronPython/27/Lib

Perhaps things have changed.

When I last checked the situation, IronPython came with no standard
library, but you could bolt one on that hadn't been tested well - IIRC,
just a simple "import os" gave a traceback. FePy was IronPython with a
standard library and some degree of testing, but their emphasis was
windows-only.

I'd be open to using FePy instead, and I might even call it IronPython,
but I'm not in the habit of happily using software that is Windows only.

That must have been quite a while ago?

I haven't tested it recently, but I'm fairly sure that IronPython
includes a Python standard library which works reasonably well, and it's
not Windows-only. (it works with Mono)

 

[Ross Ridge]

There is, of course, Stackless Python.
http://stackless.com/

Stackless Python doesn't really address the original poster's problem
as the GIL still effectively limits Python code running in one thread
at a time.

Ross Ridge

 

[John Nagle]

hi all,
are there any information about upcoming availability of parallel
computations in CPython without modules like multiprocessing? I mean
something like parallel "for" loops, or, at least, something without
forking with copying huge amounts of RAM each time and possibility to
involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
with CPython due to GIL).

AFAIK in PyPy some progress have been done (
http://morepypy.blogspot.com/2012/06/stm-with-threads.html )

Thank you in advance, D.

It would be "un-Pythonic" to have real concurrency in Python.
You wouldn't be able to patch code running in one thread from
another thread. Some of the dynamic features of Python
would break. If you want fine-grained concurrency, you need
controlled isolation between concurrent tasks, so they interact
only at well-defined points. That's un-Pythonic.

John Nagle

 

[Stefan Behnel]

Dan Stromberg, 01.07.2012 21:28:

Perhaps things have changed.

Yes, they have. The original restriction came from Microsoft enforcing a
clean-room implementation for their code, which obviously excluded the
standard library - which is well tested and licence cleared and all that,
but nevertheless non-MS. When they dropped the IronPython project, it
became free to integrate with other software.

Clearly a cultural thing.

Stefan