Re: automatic parallelization

Discussion in 'Perl' started by Michael Schlenker, Sep 13, 2007.

  1. Mikhail Teterin schrieb:
    > While C/C++ and Fortran have OpenMP (http://www.OpenMP.org/), there is
    > nothing comparable in Tcl (nor, as far as I know, in the two other
    > scripting languages).
    >
    > Or is there? I'd like to, for example, have a version of foreach loop, that
    > would split the tasks between concurrently executing threads in order to
    > scale to the number of available CPUs:


    For CPython you don't get any benefit from extra CPUs even if you tried
    (unless the code inside your loop is specially written C code), due to
    the GIL.

    For Perl i don't know, their threading model was a bit heavy last time i
    looked, but i might be off with that, so maybe its doable in Perl.

    For Tcl you would at least get the benefit of multiple CPUs, but its
    more message passing based thread model is probably not the best for
    autoparallelization like OpenMP (but you could use the tsv:: api for
    shared vars if you want to).

    >
    > For example:
    >
    > pforeach image $images {
    > set exif($image) [extract_exif $image]
    > }
    >
    > The script would be the same on a single- and a multi-CPU computer, but
    > would automatically take advantage of multiple processors, when possible.


    Its doable in principle, but as with OpenMP you need extra annotations
    to make it workable.

    If you have an event style script in Tcl you might be easier able to use
    threads, as the thread::send -async api fits very well with the event
    based style.

    Michael
     
    Michael Schlenker, Sep 13, 2007
    #1
    1. Advertising

  2. In article <>,
    Michael Schlenker <> wrote:
    >Mikhail Teterin schrieb:
    >> While C/C++ and Fortran have OpenMP (http://www.OpenMP.org/), there is
    >> nothing comparable in Tcl (nor, as far as I know, in the two other
    >> scripting languages).
    >>
    >> Or is there? I'd like to, for example, have a version of foreach loop, that
    >> would split the tasks between concurrently executing threads in order to
    >> scale to the number of available CPUs:

    >
    >For CPython you don't get any benefit from extra CPUs even if you tried
    >(unless the code inside your loop is specially written C code), due to
    >the GIL.
    >
    >For Perl i don't know, their threading model was a bit heavy last time i
    >looked, but i might be off with that, so maybe its doable in Perl.
    >
    >For Tcl you would at least get the benefit of multiple CPUs, but its
    >more message passing based thread model is probably not the best for
    >autoparallelization like OpenMP (but you could use the tsv:: api for
    >shared vars if you want to).
    >
    >>
    >> For example:
    >>
    >> pforeach image $images {
    >> set exif($image) [extract_exif $image]
    >> }
    >>
    >> The script would be the same on a single- and a multi-CPU computer, but
    >> would automatically take advantage of multiple processors, when possible.

    >
    >Its doable in principle, but as with OpenMP you need extra annotations
    >to make it workable.
    >
    >If you have an event style script in Tcl you might be easier able to use
    >threads, as the thread::send -async api fits very well with the event
    >based style.

    .
    .
    .
    There are a LOT more possibilities one might pursue, depending
    on the details of Mr. Teterin's intent. I'm fond of Linda <URL:
    http://www.unixreview.com/documents/s=10125/ur0704l/ >, Parallel
    Python <URL: http://www.parallelpython.com/ > only one of several
    initiatives which aspire to exploit multicores, and so on.
     
    Cameron Laird, Sep 13, 2007
    #2
    1. Advertising

  3. Cameron Laird wrote:

    > There are a LOT more possibilities one might pursue, depending
    > on the details of Mr. Teterin's intent.


    The reason I began wondering about this is that I like a particular
    image-gallery generating tool called `imageindex'
    (http://www.edwinh.org/imageindex/)

    Unfortunately, it processes all given pictures sequentionally ignoring my
    three other cores.

    The tool is written in Perl, but I would consider rewriting it in Tcl or
    Python, if the language made it easy to parallelize loops...

    > I'm fond of Linda <URL:
    > http://www.unixreview.com/documents/s=10125/ur0704l/ >, Parallel
    > Python <URL: http://www.parallelpython.com/ > only one of several
    > initiatives which aspire to exploit multicores, and so on.


    Linda URL does not open... I'll look into Parallel Python, but it is
    an "execution server", which is a rather different approach from OpenMP's,
    which I'm more familiar with.

    Thanks!

    -mi
     
    Mikhail Teterin, Sep 14, 2007
    #3
  4. In article <>,
    Mikhail Teterin <> wrote:
    .
    .
    .
    >> I'm fond of Linda <URL:
    >> http://www.unixreview.com/documents/s=10125/ur0704l/ >, Parallel
    >> Python <URL: http://www.parallelpython.com/ > only one of several
    >> initiatives which aspire to exploit multicores, and so on.

    >
    >Linda URL does not open... I'll look into Parallel Python, but it is

    .
    .
    .
    It comes and goes. It seems to have been OK the last several hours.
     
    Cameron Laird, Sep 14, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. P.M.
    Replies:
    60
    Views:
    1,257
    Alex Martelli
    Oct 26, 2004
  2. P.M.
    Replies:
    0
    Views:
    313
  3. Guest
    Replies:
    1
    Views:
    367
    Nicolas Lehuen
    Oct 12, 2004
  4. Mathias
    Replies:
    5
    Views:
    571
    Albert Hofkamp
    Jan 4, 2005
  5. Michael Schlenker

    Re: automatic parallelization

    Michael Schlenker, Sep 13, 2007, in forum: Python
    Replies:
    3
    Views:
    362
    Cameron Laird
    Sep 14, 2007
Loading...

Share This Page