perl threads

Discussion in 'Perl Misc' started by kath, Aug 28, 2008.

  1. kath

    kath Guest

    Hi,
    Background:
    I have task of calculating dependencies of project(s) of a scenario.
    Dependency calculation is done using an external command. Since this
    dependency calculation time depends on how big is the project is, it
    is difficult to guess how much dependency calculation of a scenario(a
    set of projects) takes. To overcome this i am using 'threads' module
    to create thread and ask each thread to do the dependency calculation.

    I have many scenarios currently and i calculate dependency foreach
    scenario. I create one thread for each project in the scenario and
    wait for all threads to finish, to take on next scenario. Because i
    know 'Perl ithreads are not lightweight!', i am using carefully, in
    the sense, I'm not using any shared variables between threads. Each
    thread will do dependency calculation, which will produces an output
    in a file separately. Later i parse those files to get dependency list
    for each project.

    Problem:
    Sometimes* the script waits forever or the script just hangs. That is
    waiting for threads. And cant continue with other scenario.

    Code:
    #this way i create threads
    #foreach scenario() ...{
    map {my $th = threads->create(\&worker, $_)} @$proj_arr; #proj_arr has
    projects of a scenario
    # ...}
    
    # This is how i wait for all threads to finish its job
    print "waiting for threads to finish";
    map {my $k = $_->join} threads->list;
    
    Is there a way i can overcome this? Or my perception about cause for
    the problem is right? Because from the log i see, only 'waiting for
    threads to finish' at last and script will continue. This is happens
    only sometimes. Most of the times the script executes fine.

    Atleast i want to be informed about the 'script is not responding or
    hanged', because the script is scheduled there will be not be any clue
    unless and otherwise, myself has to go and check.

    Hope i could explain the problem clearly. I am using 'perl, v5.8.8
    built for MSWin32-x86-multi-thread'


    Thanks in advance,
    katharnakh.
    kath, Aug 28, 2008
    #1
    1. Advertising

  2. kath

    zentara Guest

    On Thu, 28 Aug 2008 02:02:38 -0700 (PDT), kath <>
    wrote:

    >know 'Perl ithreads are not lightweight!', i am using carefully, in
    >the sense, I'm not using any shared variables between threads. Each
    >thread will do dependency calculation, which will produces an output
    >in a file separately. Later i parse those files to get dependency list
    >for each project.


    First I don't use win32, I use linux, but the advice should be the same
    in this case.
    If you don't share variables between threads, and you are writing
    results to a file, to be processed later, you don't need threads.
    Forking an independent process is better. (I realize on win32 it's all
    threads, but Perl imposes it's own weight when using threads, so you
    are better off with independent processes.

    See the part on Win32::process in
    http://perlmonks.org?node_id=500663


    >
    >Problem:
    >Sometimes* the script waits forever or the script just hangs. That is
    >waiting for threads. And cant continue with other scenario.


    Threads must reach the end of their code blocks, or return, in order
    for them to be joined. Maybe try detached threads, or use a shared
    variable to signal them to return.
    >
    >
    Code:
    >#this way i create threads
    >#foreach scenario() ...{
    >map {my $th = threads->create(\&worker, $_)} @$proj_arr; #proj_arr has
    >projects of a scenario
    ># ...}
    >
    ># This is how i wait for all threads to finish its job
    >print "waiting for threads to finish";
    >map {my $k = $_->join} threads->list;
    >
    >
    >Is there a way i can overcome this? Or my perception about cause for


    Somewhere in your worker code block, you must listen for a shared
    variable, telling the thread to immediately return. Later threads
    versions have kill signals you can send to threads, but I don't know if
    they work on win32. A thread will not end, just by telling it to join.

    zentara

    --
    I'm not really a human, but I play one on earth.
    http://zentara.net/Remember_How_Lucky_You_Are.html
    zentara, Aug 28, 2008
    #2
    1. Advertising

  3. kath

    kath Guest

    Hi,
    First, thanks for your time.

    On Aug 28, 5:57 pm, zentara <> wrote:
    > Somewhere in your worker code block, you must listen for a shared
    > variable, telling the thread to immediately return. Later threads
    > versions have kill signals you can send to threads, but I don't know if
    > they work on win32.

    Will work on this.

    >A thread will not end, just by telling it to join.

    I dint know this. May be this was my wrong understanding.

    Thanks again,
    katharnakh.
    kath, Aug 29, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. dpackwood
    Replies:
    3
    Views:
    1,767
  2. yoda
    Replies:
    2
    Views:
    419
    =?utf-8?Q?Bj=C3=B6rn_Lindstr=C3=B6m?=
    Aug 1, 2005
  3. threads without threads

    , Aug 27, 2004, in forum: C Programming
    Replies:
    4
    Views:
    383
    William Ahern
    Aug 27, 2004
  4. Pedro Pinto

    Java Threads - Get running threads

    Pedro Pinto, Apr 8, 2008, in forum: Java
    Replies:
    2
    Views:
    1,404
    Arne Vajhøj
    Apr 9, 2008
  5. Une bévue
    Replies:
    0
    Views:
    132
    Une bévue
    Jun 14, 2006
Loading...

Share This Page