Threads and Directory Handles

G

Graham Drabble

Hi,

I've just finished debugging a problem with one of my modules and
whilst I've now got it fixed I don't really understand why it broke.

I've reproduced a simplified version below.

My original module was:
---BEGIN---
package Me::File;

use strict;
use warnings;

BEGIN {
use Exporter ();
our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);

$VERSION = 1.00;

@ISA = qw(Exporter);

@EXPORT_OK = qw(file_stats);
}

our @EXPORT_OK;


sub file_stats{

my $dir = shift;
opendir (DIR,$dir) or warn "Can't open $dir: $!";

my %files;

while (my $file = readdir(DIR)){

$files{$file}{'x'} = -x "$dir/$file";
$files{$file}{'e'} = -e "$dir/$file";
# Lots of other tests go here
}

return \%files;

}

1
---END---

Which worked as expected in a non-threaded program.

---BEGIN---
use strict;
use warnings;
use Me::File qw(file_stats);

my $files = file_stats('LOGS');

foreach my $file (keys %$files){
print "MAIN\t$file\n";

}
---END---
Outputs:
C:\Perl\script>perl threadtest.pl
MAIN 20100419.log
MAIN ..
MAIN .

however if you have a threaded program if fails.

---BEGIN---
use strict;
use warnings;
use Me::File qw(file_stats);

use threads;


my $files = file_stats('LOGS');

foreach my $file (keys %$files){
print "MAIN\t$file\n";

}


my $thr = threads->create('new_thread');
sleep(1);
print "Pre\n";
$thr->join;
print "Post\n";


sub new_thread{
my $files = file_stats('LOGS');

foreach my $file (keys %$files){
print threads->tid()."\t$file\n";

}
}
---END---

Outputs
C:\Perl\script>perl logtest.pl
MAIN 20100419.log
MAIN ..
MAIN .
Free to wrong pool 17b9a48 not 1963e58 at Me/File.pm line 23.
Pre

and perl crashes.


There are two fixes for this.

One is to add a close(DIR) before the return() in file_stats(). The
second is to change file_stats() to

---BEGIN---
sub file_stats{

my $dir = shift;
opendir (my $dirh,$dir) or warn "Can't open $dir: $!";

my %files;

while (my $file = readdir($dirh)){

$files{$file}{'x'} = -x "$dir/$file";
$files{$file}{'e'} = -e "$dir/$file";
# Lots of other tests go here
}

return \%files;

}
---END---

It looks as though the problem is that then open DIRHANDLE breaks
when the new thread is created but I don't really understand why. Any
ideas?

If it helps:

C:\Perl\script>perl -v

This is perl, v5.10.0 built for MSWin32-x86-multi-thread
(with 5 registered patches, see perl -V for more detail)

running under Windows Vista.
 
G

Graham Drabble

There are two fixes for this.

One is to add a close(DIR) before the return() in file_stats(). The
second is to change file_stats() to

I do, of course, mean closedir(DIR), in the above.
 
S

Steve C

Graham said:
Hi,

I've just finished debugging a problem with one of my modules and
whilst I've now got it fixed I don't really understand why it broke.
It looks as though the problem is that then open DIRHANDLE breaks
when the new thread is created but I don't really understand why. Any
ideas?

perldoc perlopentut

handles are global to the package, so you can't open two different
directories on one dirhandle. open will close an open handle before
opening a new file on it. That means whatever the first opendir
associated with the handle in order to support readdir is now wrong.

closedir DIR is the wrong fix unless you know that another
thread hasn't already started and called opendir. Using a lexical
variable to create a new anonymous handle instead of using DIR is
a better fix. It also means that the dir is automatically closed
at the end of the sub.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Merge files 1
Dynamic directory handles? 18
Help with Hash 2
Very Sluggish Code 4
Cannot read directory c:\test 2
threads and logfile rotation 5
File locking using threads 14
Efficiently searching multiple files 10

Members online

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,898
Latest member
BlairH7607

Latest Threads

Top