Thread alternative for multi-processing?

Discussion in 'Perl Misc' started by howa, Apr 16, 2008.

  1. howa

    howa Guest

    Sometimes, I want to take the advantage of multi-core processor, and
    make the full use of my CPU time,

    e.g. I have a function, which takes a file name as input, do some text
    processing (e.g. CPU intensive), and save it back.

    I don't want to use thread, as it need to recompile perl on all my
    machines and I am not sure if it is stable enough to affect other
    apps.

    I don't mind to have a little overhead in using the fork mode (i.e.
    multi-process). I want to know are there any recommended module for
    my purpose?

    For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
    anyone want to share the experiences?


    Thanks.
    howa, Apr 16, 2008
    #1
    1. Advertising

  2. howa <> writes:

    > For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
    > anyone want to share the experiences?


    I have used Parallel::ForkManager a lot with no problems, it just
    works. I have never had the need to communicate enything more complex
    than the exit code back to the parrent though.

    //Makholm
    Peter Makholm, Apr 16, 2008
    #2
    1. Advertising

  3. On Apr 16, 4:50 am, Peter Makholm <> wrote:
    > howa <> writes:
    > > For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
    > > anyone want to share the experiences?

    >
    > I have used Parallel::ForkManager a lot with no problems, it just
    > works. I have never had the need to communicate enything more complex
    > than the exit code back to the parrent though.


    I've found it useful to pass an $ident which i can use in the
    run_on_finish callback to associate with the exit code.
    nolo contendere, Apr 16, 2008
    #3
  4. nolo contendere <> writes:

    >> I have used Parallel::ForkManager a lot with no problems, it just
    >> works. I have never had the need to communicate enything more complex
    >> than the exit code back to the parrent though.

    >
    > I've found it useful to pass an $ident which i can use in the
    > run_on_finish callback to associate with the exit code.


    Yes, but I consider this to be communication from the parrent on fork
    time to the parrent on wait time. Even if the $ident is an reference
    to something, the child can't change it for the parrent.

    //Makholm
    Peter Makholm, Apr 16, 2008
    #4
  5. On Apr 16, 11:10 am, Peter Makholm <> wrote:
    > nolo contendere <> writes:
    > >> I have used Parallel::ForkManager a lot with no problems, it just
    > >> works. I have never had the need to communicate enything more complex
    > >> than the exit code back to the parrent though.

    >
    > > I've found it useful to pass an $ident which i can use in the
    > > run_on_finish callback to associate with the exit code.

    >
    > Yes, but I consider this to be communication from the parrent on fork
    > time to the parrent on wait time. Even if the $ident is an reference
    > to something, the child can't change it for the parrent.
    >


    True, perhaps I should have replied to the OP. I was simply sharing my
    experience with Parallel::ForkManager, didn't mean to imply that the
    $ident was a child->parent communication.
    nolo contendere, Apr 16, 2008
    #5
  6. howa

    Guest

    howa <> wrote:
    > Sometimes, I want to take the advantage of multi-core processor, and
    > make the full use of my CPU time,
    >
    > e.g. I have a function, which takes a file name as input, do some text
    > processing (e.g. CPU intensive), and save it back.


    The saving it back might be a problem. Do the lines have to be saved
    back in the same order they are originally present? With parallel,
    it requires special efforts to make that happen.

    >
    > I don't want to use thread, as it need to recompile perl on all my
    > machines and I am not sure if it is stable enough to affect other
    > apps.
    >
    > I don't mind to have a little overhead in using the fork mode (i.e.
    > multi-process). I want to know are there any recommended module for
    > my purpose?
    >
    > For example, I am trying Parallel::ForkManager, POE::Wheel::Run, so
    > anyone want to share the experiences?


    In this case, I'd tend to role my own forking. I generally use
    Parallel::ForkManager for when the number of processes I want run is
    greater than the number I want to run simultaneously, which doesn't seem to
    be the case for you. You certainly *can* use it in other cases, I just
    don't see much need to. I've looked into POE::Wheel::Run a few times but
    never got over the activation barrier to actually use it.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
    , Apr 16, 2008
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hubert Hung-Hsien Chang
    Replies:
    2
    Views:
    407
    Michael Foord
    Sep 17, 2004
  2. a3321
    Replies:
    0
    Views:
    296
    a3321
    Jul 27, 2006
  3. liu yang
    Replies:
    4
    Views:
    1,991
    Antoninus Twink
    Jul 28, 2008
  4. Max
    Replies:
    2
    Views:
    1,381
    Miles Bader
    Jul 29, 2010
  5. Trans
    Replies:
    5
    Views:
    99
Loading...

Share This Page