when "normal" parallel computations in CPython will be implemented at last?

Discussion in 'Python' started by dmitrey, Jul 1, 2012.

  1. dmitrey

    dmitrey Guest

    hi all,
    are there any information about upcoming availability of parallel
    computations in CPython without modules like multiprocessing? I mean
    something like parallel "for" loops, or, at least, something without
    forking with copying huge amounts of RAM each time and possibility to
    involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
    with CPython due to GIL).

    AFAIK in PyPy some progress have been done (
    http://morepypy.blogspot.com/2012/06/stm-with-threads.html )

    Thank you in advance, D.
    dmitrey, Jul 1, 2012
    #1
    1. Advertising

  2. Re: when "normal" parallel computations in CPython will be implementedat last?

    On 07/01/2012 07:51 PM, dmitrey wrote:
    > hi all,
    > are there any information about upcoming availability of parallel
    > computations in CPython without modules like multiprocessing? I mean
    > something like parallel "for" loops, or, at least, something without
    > forking with copying huge amounts of RAM each time and possibility to
    > involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
    > with CPython due to GIL).
    >
    > AFAIK in PyPy some progress have been done (
    > http://morepypy.blogspot.com/2012/06/stm-with-threads.html )


    As far as I can tell, there are no concrete plans to integrate
    concurrency better, or get rid of the GIL, at the moment. To quote
    http://wiki.python.org/moin/GlobalInterpreterLock

    """Getting rid of the GIL is an occasional topic on the python-dev
    mailing list. No one has managed it yet."""

    There are currently no open or accepted PEPs on the subject of concurrency.
    http://www.python.org/dev/peps/

    There is, of course, Stackless Python.
    http://stackless.com/
    Thomas Jollans, Jul 1, 2012
    #2
    1. Advertising

  3. Thomas Jollans, Jul 1, 2012
    #3
  4. dmitrey

    Andrew Berg Guest

    Re: when "normal" parallel computations in CPython will be implementedat last?

    On 7/1/2012 1:53 PM, Thomas Jollans wrote:
    > As far as I can tell, there are no concrete plans to integrate
    > concurrency better, or get rid of the GIL, at the moment. To quote
    > http://wiki.python.org/moin/GlobalInterpreterLock
    >
    > """Getting rid of the GIL is an occasional topic on the python-dev
    > mailing list. No one has managed it yet."""

    There's also this, recently written by one of CPython's core devs:
    http://python-notes.boredomandlazin...the-gil-is-more-important-than-fixing-unicode
    --
    CPython 3.3.0a4 | Windows NT 6.1.7601.17803
    Andrew Berg, Jul 1, 2012
    #4
  5. Re: when "normal" parallel computations in CPython will be implementedat last?

    On 07/01/2012 09:28 PM, Dan Stromberg wrote:
    >
    >
    > On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans <
    > <mailto:>> wrote:
    >
    > On 07/01/2012 08:44 PM, Dan Stromberg wrote:
    > > IronPython, sadly, lacks a python standard library.

    >
    >
    > Beg pardon?
    >
    > https://github.com/IronLanguages/ma...al.LCA_RESTRICTED/Languages/IronPython/27/Lib
    >
    > Perhaps things have changed.
    >
    > When I last checked the situation, IronPython came with no standard
    > library, but you could bolt one on that hadn't been tested well - IIRC,
    > just a simple "import os" gave a traceback. FePy was IronPython with a
    > standard library and some degree of testing, but their emphasis was
    > windows-only.
    >
    > I'd be open to using FePy instead, and I might even call it IronPython,
    > but I'm not in the habit of happily using software that is Windows only.
    >


    That must have been quite a while ago?

    I haven't tested it recently, but I'm fairly sure that IronPython
    includes a Python standard library which works reasonably well, and it's
    not Windows-only. (it works with Mono)
    Thomas Jollans, Jul 1, 2012
    #5
  6. dmitrey

    Ross Ridge Guest

    Re: when "normal" parallel computations in CPython will be implementedat last?

    Thomas Jollans <> wrote:
    >There is, of course, Stackless Python.
    >http://stackless.com/


    Stackless Python doesn't really address the original poster's problem
    as the GIL still effectively limits Python code running in one thread
    at a time.

    Ross Ridge

    --
    l/ // Ross Ridge -- The Great HTMU
    [oo][oo]
    -()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
    db //
    Ross Ridge, Jul 1, 2012
    #6
  7. dmitrey

    John Nagle Guest

    Re: when "normal" parallel computations in CPython will be implementedat last?

    On 7/1/2012 10:51 AM, dmitrey wrote:
    > hi all,
    > are there any information about upcoming availability of parallel
    > computations in CPython without modules like multiprocessing? I mean
    > something like parallel "for" loops, or, at least, something without
    > forking with copying huge amounts of RAM each time and possibility to
    > involve unpiclable data (vfork would be ok, but AFAIK it doesn't work
    > with CPython due to GIL).
    >
    > AFAIK in PyPy some progress have been done (
    > http://morepypy.blogspot.com/2012/06/stm-with-threads.html )
    >
    > Thank you in advance, D.
    >


    It would be "un-Pythonic" to have real concurrency in Python.
    You wouldn't be able to patch code running in one thread from
    another thread. Some of the dynamic features of Python
    would break. If you want fine-grained concurrency, you need
    controlled isolation between concurrent tasks, so they interact
    only at well-defined points. That's un-Pythonic.

    John Nagle
    John Nagle, Jul 2, 2012
    #7
  8. Re: when "normal" parallel computations in CPython will be implementedat last?

    Dan Stromberg, 01.07.2012 21:28:
    > On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans wrote:
    >> On 07/01/2012 08:44 PM, Dan Stromberg wrote:
    >>> IronPython, sadly, lacks a python standard library.

    >>
    >> Beg pardon?
    >>
    >> https://github.com/IronLanguages/ma...al.LCA_RESTRICTED/Languages/IronPython/27/Lib

    >
    > Perhaps things have changed.


    Yes, they have. The original restriction came from Microsoft enforcing a
    clean-room implementation for their code, which obviously excluded the
    standard library - which is well tested and licence cleared and all that,
    but nevertheless non-MS. When they dropped the IronPython project, it
    became free to integrate with other software.

    Clearly a cultural thing.

    Stefan
    Stefan Behnel, Jul 2, 2012
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Yuancai \(Charlie\) Ye
    Replies:
    0
    Views:
    906
    Yuancai \(Charlie\) Ye
    Jun 7, 2004
  2. Yuancai \(Charlie\) Ye
    Replies:
    2
    Views:
    390
    Yuancai \(Charlie\) Ye
    Apr 26, 2004
  3. Yuancai \(Charlie\) Ye
    Replies:
    1
    Views:
    409
    Ken Cox [Microsoft MVP]
    Jun 7, 2004
  4. Replies:
    7
    Views:
    441
  5. Hseu-Ming Chen
    Replies:
    1
    Views:
    993
    Chris Torek
    Jun 12, 2011
Loading...

Share This Page