query on fork in perl

Discussion in 'Perl Misc' started by debraj, May 25, 2004.

  1. debraj

    debraj Guest

    Hi

    I am trying out a new thing to me ie. use fork(). Now my requirement
    is like this :

    I have a file which has quite a few numbers such as :

    22,23,34 etc.
    Now I have a sub-routine say sub create_page, which when passed the
    above numbers does quite a few things and creates html pages in their
    name, such as 22.html etc.
    So instead of passing one at a time, I wanted to pass it to fork'ed
    processes so that they happen all at a time. This is where I am
    getting stuck. Can anyone help out please ? basically how to call the
    same sub with different parameters and on different processes ? and
    how to control them ?

    My code :
    open LIST , "file";
    FORKER: while (<LIST>) {
    my $newpid = fork();
    if ( not defined $newpid )
    {
    # if return value of fork() is undef, something went wrong
    die "fork didn't work: $!\n";
    }
    elsif ( $newpid == 0 )
    {
    # if return value is 0, this is the child process
    $parent = $pid; # which has a parent called $pid
    $pid = $$; # and which will have a process ID of its very
    own
    @kids = (); # the child doesn't want this baggage from the
    parent
    last FORKER; # and we don't want the child making babies
    either
    }
    else
    {
    # the parent process is returned the PID of the newborn by
    fork()
    print "$$ spawned $newpid \n";
    push @kids, $newpid;
    $i= $#kids;
    $i ++;
    }

    }

    if ( $parent ) # if I have a parent, i.e. if I'm the child process
    {
    print "I am process number $pid\n";

    &create_page("$kids[$i]");
    print "Creating $kids[$i]\n";
    exit( 0 );
    }
    else
    {
    # parent process needs to preside over the death of its kids
    while ( my $kid = shift @kids )
    {
    print "Parent waiting for $kid to die\n";
    my $reaped = waitpid( $kid, 0 );
    unless ( $reaped == $kid )
    {
    print "Something's up: $?\n";
    }
    }
    }
     
    debraj, May 25, 2004
    #1
    1. Advertising

  2. debraj wrote:
    > Hi
    >
    > I am trying out a new thing to me ie. use fork(). Now my requirement
    > is like this :
    >
    > I have a file which has quite a few numbers such as :
    >
    > 22,23,34 etc.
    > Now I have a sub-routine say sub create_page, which when passed the
    > above numbers does quite a few things and creates html pages in their
    > name, such as 22.html etc.
    > So instead of passing one at a time, I wanted to pass it to fork'ed
    > processes so that they happen all at a time. This is where I am
    > getting stuck. Can anyone help out please ? basically how to call the
    > same sub with different parameters and on different processes ? and
    > how to control them ?
    >
    > My code :
    > open LIST , "file";
    > FORKER: while (<LIST>) {
    > my $newpid = fork();
    > if ( not defined $newpid )
    > {
    > # if return value of fork() is undef, something went wrong
    > die "fork didn't work: $!\n";
    > }
    > elsif ( $newpid == 0 )
    > {
    > # if return value is 0, this is the child process
    > $parent = $pid; # which has a parent called $pid
    > $pid = $$; # and which will have a process ID of its very
    > own
    > @kids = (); # the child doesn't want this baggage from the
    > parent
    > last FORKER; # and we don't want the child making babies
    > either
    > }
    > else
    > {
    > # the parent process is returned the PID of the newborn by
    > fork()
    > print "$$ spawned $newpid \n";
    > push @kids, $newpid;
    > $i= $#kids;
    > $i ++;
    > }
    >
    > }
    >
    > if ( $parent ) # if I have a parent, i.e. if I'm the child process
    > {
    > print "I am process number $pid\n";
    >
    > &create_page("$kids[$i]");
    > print "Creating $kids[$i]\n";
    > exit( 0 );
    > }
    > else
    > {
    > # parent process needs to preside over the death of its kids
    > while ( my $kid = shift @kids )
    > {
    > print "Parent waiting for $kid to die\n";
    > my $reaped = waitpid( $kid, 0 );
    > unless ( $reaped == $kid )
    > {
    > print "Something's up: $?\n";
    > }
    > }
    > }


    ufff!

    I advise you to use Parallel::ForkManager or Proc::Queue from
    CPAN to limit the number of concurrent processes running:



    # limit number of child processes to 6
    use Proc::Queue size => 6, qw(run_back);
    use POSIX ":sys_wait_h";

    sub create_page {...}

    while(<>) {
    my $number=$_;
    chomp $number;

    # fork and call create_page sub
    run_back {
    create_page($number)
    };

    # read childs
    1 while waitpid(-1, WNOHANG)>0;
    }

    # wait for all childs to exit
    1 while wait != -1;


    Bye,

    - Salva
     
    Salvador Fandiño, May 25, 2004
    #2
    1. Advertising

  3. debraj

    Greg Bacon Guest

    In article <>,
    debraj <> wrote:

    : I have a file which has quite a few numbers such as :
    :
    : 22,23,34 etc.
    : Now I have a sub-routine say sub create_page, which when passed the
    : above numbers does quite a few things and creates html pages in their
    : name, such as 22.html etc.
    : So instead of passing one at a time, I wanted to pass it to fork'ed
    : processes so that they happen all at a time. This is where I am
    : getting stuck. Can anyone help out please ? basically how to call the
    : same sub with different parameters and on different processes ? and
    : how to control them ?

    See below:

    #! /usr/local/bin/perl

    use warnings;
    use strict;

    use POSIX ":sys_wait_h";

    sub do_some_work {
    my $param = shift;

    open my $fh, ">", "$param.html"
    or die "$0: open >$param.html: $!";

    print $fh map "$_\n",
    "<html>",
    "<head><title>Happy $param Page</title></head>",
    "<body>",
    "<h1>Happy $param Page</h1>",
    "</body>",
    "</html>";
    }

    ## main
    my %kid;

    for (qw/ 22 23 24 25 /) {
    my $pid = fork;

    if (not defined $pid) {
    warn "$0: fork (item $_): $!";
    next;
    }
    elsif ($pid == 0) {
    # child

    do_some_work $_;
    exit 0;
    }
    else {
    # parent

    $kid{$pid} = 1;
    }
    }

    my $errors = 0;
    while (scalar keys %kid > 0) {
    my $pid = wait;

    if ($pid == -1) {
    warn "$0: unreaped: " . join(", " => keys %kid);
    exit;
    }

    warn "$0: unknown kid pid $pid"
    unless exists $kid{$pid};

    ++$errors if $? != 0;

    delete $kid{$pid};
    }

    print "Done.\n";

    exit $errors;

    Hope this helps,
    Greg
    --
    The Constitution is not an instrument for the government to restrain
    the people, it is an instrument for the people to restrain the
    government -- lest it come to dominate our lives and interests.
    -- Patrick Henry
     
    Greg Bacon, May 25, 2004
    #3
  4. (debraj) writes:

    > I am trying out a new thing to me ie. use fork(). Now my requirement
    > is like this :
    >
    > I have a file which has quite a few numbers such as :
    >
    > 22,23,34 etc.
    > Now I have a sub-routine say sub create_page, which when passed the
    > above numbers does quite a few things and creates html pages in their
    > name, such as 22.html etc.
    > So instead of passing one at a time, I wanted to pass it to fork'ed
    > processes so that they happen all at a time.


    Others have addressed how to do this in Perl but I'm not sure that
    anyone has pointed out (the non-Perl issue) that unless you expect the
    child procesess to be waiting on non-local resources this is probably
    not a sensible thing to be doing.

    --
    \\ ( )
    . _\\__[oo
    .__/ \\ /\@
    . l___\\
    # ll l\\
    ###LL LL\\
     
    Brian McCauley, May 25, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Josh Denny

    fork in perl 5.8.3 on windows

    Josh Denny, Mar 2, 2004, in forum: Perl
    Replies:
    2
    Views:
    6,741
    Jim Gibson
    Mar 2, 2004
  2. Andrew Zhilenko
    Replies:
    0
    Views:
    818
    Andrew Zhilenko
    Nov 22, 2004
  3. Eric Snow

    os.fork and pty.fork

    Eric Snow, Jan 8, 2009, in forum: Python
    Replies:
    0
    Views:
    603
    Eric Snow
    Jan 8, 2009
  4. Mike

    'fork' query

    Mike, Jan 28, 2009, in forum: C Programming
    Replies:
    3
    Views:
    355
    Kaz Kylheku
    Jan 28, 2009
  5. Kelly Corcam

    Emacs debugger and perl fork

    Kelly Corcam, Aug 22, 2003, in forum: Perl Misc
    Replies:
    1
    Views:
    162
    Peter Scott
    Aug 23, 2003
Loading...

Share This Page