fork messing up parent filehandle

Discussion in 'Perl Misc' started by darksaga, Jun 20, 2006.

  1. darksaga

    darksaga Guest

    hi everybody,

    i wrote a script where i use the fork command to speed up computation
    of a program. the parent process spawns a limited number of children,
    which process something. after a child process finishes its task a new
    child is generated. this is repeated until the parent process reaches
    the end of a file it runs through.

    problem is that the fork command seems to mess up the parent filehandle
    to my file, altough the children do not use this filehandle. when
    looping through the file, the filehandle is magically reset to the
    beginning of the file, or a position near the beginning. (see code
    below)

    i've read some other posts about the same prob. e.g.:
    http://snipurl.com/s0pq
    http://snipurl.com/s0ps
    but there was no solution :/.

    only thing was a workaround to read the file before forking into an
    array, this solution is suboptimal for me coz my file could be >1GB.

    sysnfo:
    os: sun4u sparc 5.8
    perl: v5.8.6 built for sun4-solaris

    maybe there's someone out there, who knows a solution...

    greets darksaga
    ____________________
    #! /usr/bin/env perl -w
    use strict;
    use FileHandle;

    my %children;
    my $maxChildren = 10;
    my $fileHandle = undef;

    my $file = 'testFile';
    createTestFile($file);

    # start of main prog.
    print "initial child spawning...\n";
    for (1..$maxChildren)
    {
    if(my $line = getNextLine($file))
    {
    print "$line\n";
    if(scalar(keys(%children)) < $maxChildren)
    {
    my $delay = int(rand 2)+1;
    my $child = create_child($delay);
    $children{$child} = "(slept $delay seconds)";
    }
    }
    }
    print "initial child spawning done...\n";

    while(1) # loop until all done
    {
    my @pids = sort keys %children;
    if (@pids)
    {
    #print "currently active children: @pids\n";
    if(my $line = getNextLine($file))
    {
    print "$line\n";
    if(scalar(keys(%children)) < $maxChildren)
    {
    my $delay = int(rand 2)+1;
    my $child = create_child($delay);
    $children{$child} = "(slept $delay seconds)";
    }
    }
    }
    else
    {
    print "all child processes are finished...\n";
    }
    if (($_ = wait) == -1)
    {
    print "wait() says no more children...\n";
    last;
    }
    else
    {
    #print "child $_ has finished processing...". $children{$_} ."\n";
    delete $children{$_};
    }
    }
    print "parent is done...\n";


    sub create_child
    {
    my ($delay) = @_;
    my $pid = fork();
    die "Unable to fork: $!" unless defined $pid;
    return $pid if $pid; # Parent
    child_routine($delay); # Child
    die "Child $$ returned when it should have exited";
    }

    sub child_routine
    {
    my ($delay) = @_;
    #print "Child $$ sleeping for $delay seconds\n";
    sleep $delay;
    #print "Child $$ exiting\n";
    exit 0;
    }

    sub getNextLine
    {
    my ($fileName) = @_;
    if (!$fileHandle)
    {
    $fileHandle = new FileHandle;
    $fileHandle->open("< $fileName") or die $!;
    }
    my $entry = $fileHandle->getline();
    unless($entry)
    {
    close $fileHandle;
    return undef;
    }
    chomp($entry);
    return $entry;
    }

    sub createTestFile
    {
    my ($fileName) = @_;
    open(my $write, ">", "$file") or die $!;
    for(1..30)
    {
    print $write "$_\n";
    }
    close $write;
    }
     
    darksaga, Jun 20, 2006
    #1
    1. Advertisements

  2. Just for the record. I cannot replicate your problem on perl 5.8.4 on
    Linux. The program prints:

    ----8<-------8<-------8<-------8<-------8<-------8<-------8<-------8<---
    initial child spawning...
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    initial child spawning done...
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    readline() on closed filehandle GEN0 at /usr/lib/perl/5.8/IO/Handle.pm line 1
    all child processes are finished...
    wait() says no more children...
    parent is done...
    ----8<-------8<-------8<-------8<-------8<-------8<-------8<-------8<---
    which is what I expected. If you expected some different output, please
    specify what you expected.

    (I noticed a few other problems with your program - but they don't seem
    to be related to the problem you describe, so I guess they are caused by
    cutting down a real program down to a minimal test case)

    hp
     
    Peter J. Holzer, Jun 20, 2006
    #2
    1. Advertisements

  3. darksaga

    darksaga Guest

    problem won't occur on a linux system, seems to be sun solaris related.
    just read the posts i've linked above, there problems also occured on
    solaris machines only.
    jupp, just stripped testcode.
    on a sun solaris machines you never get the "readline() on closed
    filehandle" errorMsg,
    coz the prog. is somehow caught in endless loop reading the file.

    there seems to be some problem with the EOF detection of a file handle
    on solaris machines. somehow the current filehandle position is
    corrupted, me wonders why, and then is reset to a position near the
    beginning of the file.

    greetz

    darksaga
     
    darksaga, Jun 20, 2006
    #3
  4. darksaga

    xhoster Guest

    Solaris is buggy on closing forked file handles--when one copy closes
    it screws up the other copies.

    Change the child's exit to a POSIX::_exit. This bypasses the implicit
    closing of the child's file handles.

    Make sure you explicitly close all child's open file handles, other than
    the ones shared with the parent, before calling _exit.

    Xho
     
    xhoster, Jun 20, 2006
    #4
  5. darksaga

    darksaga Guest

    Change the child's exit to a POSIX::_exit. This bypasses the implicit
    the POSIX::_exit() command solves the prob.

    did some research about it, and found the following statement @ cpan:

    "The exit() function does not always exit immediately. It calls any
    defined END routines first, but these END routines may not themselves
    abort the exit. Likewise any object destructors that need to be called
    are called before the real exit. If this is a problem, you can call
    POSIX:_exit($status) to avoid END and destructor processing. See
    perlmod for details."

    seems that these "defined END routines" close or somehow mess up the
    filehandle of the parent process on solaris machines, if you call a
    simple exit in one of your spawned child processes to end it. then its
    seems to be reopened or shifted to somewhere near the beginning of your
    file in the parent process. result is a endless loop.

    tricky thing, imo. me wonders how a M$ os behaves ;-).

    thanks @ Xho

    greetz darksaga
     
    darksaga, Jun 20, 2006
    #5
  6. This seems to be a rather severe violation of POSIX behaviour. Closing a
    file descriptor in one process should never affect a file descriptor in
    another process (unless they refer to opposite ends of a pipe or socket,
    of course). Is this new in Solaris 10 or a long-standing Solaris bug?

    hp
     
    Peter J. Holzer, Jun 20, 2006
    #6
  7. darksaga

    xhoster Guest

    Currently I can only test it on SunOS 5.8, but I think I've seen it
    under older versions without really understanding what was going on.
    (I have no idea what the difference is between Solaris and SunOS,
    uname says SunOS while OSTYPE and perl -v say Solaris. If this
    distinction is important, then take what I say with a grain of salt.)

    Xho
     
    xhoster, Jun 20, 2006
    #7
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.