Open All files one by one

[Billy]

All,

I am looking for a way to open ALL files in a directory one by one in perl.

I am able to open one file... read in the data and write it out to another
file without any problems. But I have to tell the script the file I am
opening. I would like the script to open all .txt files read out some data
and append it to another file. The only part I am stuck on is the opening of
the files without providing the filenames.

I searched around and got tired of going in circles.

Say I have a directory that has...

bob.txt
ted.txt
sam.txt
sue.txt
index.html

Is there a simple way to open all the .txt files?

If this needs more info please let me know.

Billy

 

[Sherm Pendley]

I am looking for a way to open ALL files in a directory one by one in perl.

Have a look at the opendir() and readdir() functions:

perldoc -f opendir
perldoc -f readdir

sherm--

 

[Keith Keller]

I am looking for a way to open ALL files in a directory one by one in perl.

Are you sure?

I am able to open one file... read in the data and write it out to another
file without any problems. But I have to tell the script the file I am
opening. I would like the script to open all .txt files read out some data
and append it to another file. The only part I am stuck on is the opening of
the files without providing the filenames.

I'd suggest getting a list of filenames, then opening a filehandle on
each one at a time, doing what you need, and closing the filehandle.
Presuming you already know the opening and closing part, you probably
want glob to make a list of filenames; perldoc -f glob for help.

If that's not useful advice, you might consider posting a short
script which demonstrates what you're trying to do, and where you
need help.

--keith

 

[Jürgen Exner]

Billy wrote:
[...]

I am able to open one file... read in the data and write it out to
another file without any problems. [...]
Is there a simple way to open all the .txt files?

See 'perldoc -f glob' and then simply loop through the list.

jue

 

[Michele Dondi]

Say I have a directory that has...

bob.txt
ted.txt
sam.txt
sue.txt
index.html

Is there a simple way to open all the .txt files?

In addition to the answers you already got, that you certainly
*should* take into account, depending on what you *really* need to do
with those files, you can also "trick" @ARGV to do the job for you (of
course some care must be taken if you're already using it for
something else):

@ARGV=<*.txt>;
while (<>) {
# ...
}


Michele

 

[Tintin]

All,

I am looking for a way to open ALL files in a directory one by one in
perl.

I am able to open one file... read in the data and write it out to another
file without any problems. But I have to tell the script the file I am
opening. I would like the script to open all .txt files read out some data
and append it to another file. The only part I am stuck on is the opening
of
the files without providing the filenames.

I searched around and got tired of going in circles.

Say I have a directory that has...

bob.txt
ted.txt
sam.txt
sue.txt
index.html

Is there a simple way to open all the .txt files?

Do you mean one at a time? If so, then

#!/usr/bin/perl
use strict;

foreach my $file (</path/to/dir/*>) {
open FILE, $file or die "Can not open $file $!\n";

while (<FILE>) {
print; # or whatever
}

close FILE;
}

 

[foobar]

#!/../..perl -w

use strict;

$my_txt_dir = ' / .. / .. /someplace';

my $files_ref = ReadDir ($my_txt_dir);

foreach my $file (@$files_ref){
if ($file =~ /\.txt/){
my $file_recs_ref = ReadFile ("$my_txt_dir/$file");
foreach $line (@$file_recs_ref){
......
}

}

sub ReadFile{
my $file = shift @_;
open (FH,"$file") or die "Could not open file $file";
my @recs = <FH>;
close(FH);
return \@recs;
}

sub ReadDir{
my $dir = shift @_;
opendir DIR, $dir or die "could not open dir $dir";
my @files = readdir DIR;
closedir DIR;
return \@files;
}

 

[A. Sinan Unur]

Please provide some context about what you are responding to. It looks to
me like you are asking for a code review.

#!/../..perl -w

As one of my friends would say, what is this crap?!

use strict;

use warnings;

is preferable to -w.

$my_txt_dir = ' / .. / .. /someplace';

Post code that actually compiles.

my $files_ref = ReadDir ($my_txt_dir);

Not wrong per se but what is the point of writing a sub whose name closely
resembles the name of one Perl's functions, but behaves differently?

Not to mention that you are not gaining anything by doing this. Instead,
you run the risk of creating a bottleneck by potentially reading in a list
of thousands of files when all you need is one file name at a time.

foreach my $file (@$files_ref){
if ($file =~ /\.txt/){

Do you really want to match the file name if .txt appears anywhere in the
name? That is, is 'my.doc.txt.pdf' really an acceptable filename?

my $file_recs_ref = ReadFile ("$my_txt_dir/$file");

If you really want to slurp, maybe you should use File::Slurp.

A better way might be this:

use strict;
use warnings;

use File::Slurp;
use File::Spec::Functions 'catfile';

use constant DIR => 'd:/home';

opendir my $DIR, DIR
or die 'Cannot open directory ', DIR, ": $!";

while(my $fn = readdir $DIR) {
next unless -f $fn && $fn =~ /\.txt$/i;
my $lines = read_file($fn, array_ref => 1);
process_file($lines);
}

closedir $DIR
or die 'Cannot close directory ', DIR, ": $!";

sub process_file {
# For lack of anything better to make up
# right now
print @{ $_[0] };
}
__END__

 

[yapp]

My sincere apologies for the response. This is the first time I'm
actually posting on a discussion group. I should have read the rules.
I just realized all the flaws in my code and appreciate the critical
comments.

 

[Billy]

All,
I am looking for a way to open ALL files in a directory one by one in perl.
I am able to open one file... read in the data and write it out to another
file without any problems. But I have to tell the script the file I am
opening. I would like the script to open all .txt files read out some data
and append it to another file. The only part I am stuck on is the opening of
the files without providing the filenames.
I searched around and got tired of going in circles.
Say I have a directory that has...
bob.txt
ted.txt
sam.txt
sue.txt
index.html
Is there a simple way to open all the .txt files?
If this needs more info please let me know.
Billy

All how about:

#!/usr/bin/perl
use CGI::Carp qw(fatalsToBrowser);
use strict;
my @files = <*.txt>;
while (<*.txt>) {
open (HANDLE, $_) || die ("Can't open $_: $!");
while (<HANDLE>) {
print; # or something else
}
}
close HANDLE;

Anything wrong with this approach?

Thank you,

Billy

 

[Billy]

All how about:

#!/usr/bin/perl
use CGI::Carp qw(fatalsToBrowser);
use strict;
my @files = <*.txt>;
while (<*.txt>) {
open (HANDLE, $_) || die ("Can't open $_: $!");
while (<HANDLE>) {
print; # or something else
}
}
close HANDLE;

Anything wrong with this approach?

Thank you,

Billy

I found a problem right off... my @files = <*.txt>; only works if I have it
in the same file directory.

Billy

 

[A. Sinan Unur]

use File::Spec::Functions 'catfile';
....

next unless -f $fn && $fn =~ /\.txt$/i;

Spot the bug! In my defense, I did run the script before posting it, but I
only ran it in D:/Home. The proper check above should have been:

next unless -f catfile(DIR, $fn) && $fn =~ /\.txt$/i;

Sorry.

 

[Tad McClellan]

#!/usr/bin/perl
use CGI::Carp qw(fatalsToBrowser);
use strict;
my @files = <*.txt>;
while (<*.txt>) {
open (HANDLE, $_) || die ("Can't open $_: $!");
while (<HANDLE>) {
print; # or something else
}
}
close HANDLE;

Anything wrong with this approach?

Yes, your indenting got all messed up.

 

[Tintin]

All how about:

#!/usr/bin/perl
use CGI::Carp qw(fatalsToBrowser);
use strict;
my @files = <*.txt>;
while (<*.txt>) {
open (HANDLE, $_) || die ("Can't open $_: $!");
while (<HANDLE>) {
print; # or something else
}
}
close HANDLE;

Anything wrong with this approach?

You read all filenames into the @files array, but never use it.

Instead of a while loop, I think a foreach loop looks cleaner, eg:

foreach my $file (<*.txt>) {
open FILE, $file or die "Can not open $file $!\n";
..
}

 

[Michele Dondi]

#!/../..perl -w

Ouch! Quite a strange shebang line!! In particular is your perl
interpreter really called '..perl'?!? Of course this would work under
Windows, and also under *NIX (I guess), *if* the script is called with

perl programname

use strict;

Worth mentioning that with recent enough perls it's better to

use warnings; # instead of -w

$my_txt_dir = ' / .. / .. /someplace';

Huh?!? Quite a strange path, isn't it? Also, since we're under strict,
this won't compile at all.

my $files_ref = ReadDir ($my_txt_dir);

Do you really think it is *convenient* (i) to create a dedicated sub
to be used only *once* and that (ii) really adds only a very thin
layer around Perl's readdir(), (iii) is called very similarly to it
causing potential confusion, (iv) returns (a reference to an array
containing) a full list of entries whereas these may be examined
sequentially one at a time?

foreach my $file (@$files_ref){

This may be to a large extent a matter of personal tastes, but what is
the point of unnecessary referencing/dereferencing? Aren't we abusing
this a little too much?!?

if ($file =~ /\.txt/){

This may well be what you want, but it may be safer to /\.txt$/
instead.

my $file_recs_ref = ReadFile ("$my_txt_dir/$file");
foreach $line (@$file_recs_ref){

Ditto as above (wrt referencing/dereferencing).

Oh, and this won't compile either, at least under strict. Good reason
to at least perl -c your code.

Oh, and while I'm here: quite a weird indenting style too! Please note
that it doesn't make your code much readable...

sub ReadFile{
my $file = shift @_;

shift; # shift @_; doesn't add to clarity!

open (FH,"$file") or die "Could not open file $file";

Better include $! in the error message. It only requires a pair more
keystrokes...

BTW: it is convenient to use lexical FHs nowadays...

sub ReadDir{
my $dir = shift @_;

Ditto as above!

opendir DIR, $dir or die "could not open dir $dir";

Ditto as above!

BTW: one can use lexical dirhandles too!

All in all *if* I wanted to maintain the logic of your script (which I
do *not* really want!), then at least I'd rewrite it like


#!/usr/bin/perl

use strict;
use warnings;

sub get_dir_entries;
sub read_file;

my $txt_dir='/path/to/dir';

for (get_dir_entries $txt_dir) {
next unless /\.txt$/;
for (read_file "$txt_dir/$_") {
# ...
}
}

sub get_dir_entries {
my $dir=shift;
opendir my $dh, $dir or
die "Could not open dir `$dir': $!\n";
readdir $dh;
}

sub read_file{
my $file=shift;
open my $fh, '<', $file or
die "Could not open file `$file': $!\n";
<$fh>;
}

__END__


HTH,
Michele

 

[Michele Dondi]

All how about:

#!/usr/bin/perl
use CGI::Carp qw(fatalsToBrowser);

Oh, it was a CGI thing then... ;-)

use strict;

While you're there do a favour to yourself and

use warnings; # too

my @files = <*.txt>;

Why?!? Do you realize that you do *not* use @files at all,
subsequently?

while (<*.txt>) {

A faq entry warns about potential issues with glob() in scalar
context. Also, since C<< while (<$fh>) >> is Perl's typical idiom for
"iterate over filehandle", this could be confusing at best. I'd do

open (HANDLE, $_) || die ("Can't open $_: $!");

Maybe it's just me but I'd use a lexical FH anyway:

open my $fh, '<', $_ or die ...

Anything wrong with this approach?

Apart the cmts above, I would say: no. Still, if the script is really
that simple I'd use @ARGV's magic as of my previous post instead:


#!/usr/bin/perl

use strict;
use warnings;
# use CGI::Carp qw(fatalsToBrowser);

@ARGV=<*.txt>;
while (<>) {
print; # or something else
}

__END__


HTH,
Michele

 

[Michele Dondi]

My sincere apologies for the response. This is the first time I'm
actually posting on a discussion group. I should have read the rules.
I just realized all the flaws in my code and appreciate the critical
comments.

Well, then another "rule" you should get familiar with is that you'd
better quote some relevant context...


Michele

 

[Michele Dondi]

I found a problem right off... my @files = <*.txt>; only works if I have it
in the same file directory.

Please note that nothing prevents you to from doing

my @files = <path/to/dir/*.txt>;


HTH,
Michele

 

[Billy]

Apart the cmts above, I would say: no. Still, if the script is really

that simple I'd use @ARGV's magic as of my previous post instead:
#!/usr/bin/perl

use strict;
use warnings;
# use CGI::Carp qw(fatalsToBrowser);

@ARGV=<*.txt>;
while (<>) {
print; # or something else
}

__END__

HTH,
Michele

Thank you for all the replies... Since I don't know enough of perl yet I
don't know the construct above. Plus I don't know what this will turn into,
I prefer to keep it in a form I can learn from then refine later.

Thank you again....

Billy