Counting files

S

Shawn Corey

Hi,

I have a list:

file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
....

I want to count them like:

$File{.}{file1} = count;
$File{.}{file2} = count;
$File{dir1}{file3} = count;
$File{dir1}{file4} = count;
$File{dir1}{subdir1}{file5} = count;
$file{dir2}{file6} = count;
....

Is the a nifty way of doing this in Perl-speak?
 
J

James Willmore

I have a list:

file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
...

I want to count them like:

$File{.}{file1} = count;
$File{.}{file2} = count;
$File{dir1}{file3} = count;
$File{dir1}{file4} = count;
$File{dir1}{subdir1}{file5} = count;
$file{dir2}{file6} = count;
...

Is the a nifty way of doing this in Perl-speak?
Click to expand...

If you're just looking to count the entries in the list, you may want
to do ...

==untested==
my @list =
qw(file1 file2 dir1/file3 dir1/file4 dir1/subdir1/file5 dir2/file6 );
my $count = @list;
print "The number of file entries is: $count\n";

Is this what you were after?

HTH

--
Jim

Copyright notice: all code written by the author in this post is
released under the GPL. http://www.gnu.org/licenses/gpl.txt
for more information.

a fortune quote ...
Trying to be happy is like trying to build a machine for which
the only specification is that it should run noiselessly.
 
G

Greg Bacon

: I have a list:
:
: file1
: file2
: dir1/file3
: dir1/file4
: dir1/subdir1/file5
: dir2/file6
: ...
:
: I want to count them like:
:
: $File{.}{file1} = count;
: $File{.}{file2} = count;
: $File{dir1}{file3} = count;
: $File{dir1}{file4} = count;
: $File{dir1}{subdir1}{file5} = count;
: $file{dir2}{file6} = count;
: ...
:
: Is the a nifty way of doing this in Perl-speak?

I don't know about nifty, but see below.

#! /usr/local/bin/perl

use strict;
use warnings;

use Data::Dumper;
use File::Find;

sub store {
my $file = shift;
my $count = shift;

return unless $file && @_;

my $hd = shift;
if (@_) {
$file->{$hd} ||= {};
store($file->{$hd}, $count, @_);
}
else {
$file->{$hd} = $$count++;
}
}

sub make_counter {
my $count = 0;
my %file;

my $sub = sub {
return unless -f;

store \%file, \$count, grep /./s,
split /\//, $File::Find::name;
};

(sub { %file }, $sub);
}

## main
die "Usage: $0 dir..\n" unless @ARGV;

my($file,$wanted) = make_counter;

find $wanted, @ARGV;

print Dumper $file->();

Hope this helps,
Greg
 
S

Shawn Corey

Purl said:
Your article is gibberish.

Precisely what do you want to count?

Work towards developing an ability to write articles
which are clear, concise, coherent and free of gibberish.


Purl Gurl
Click to expand...

OK, is this better?

1. I have a list of files like:
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
....
They are stored in an array, call it @listOfFiles:
@listOfFiles = qw(
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
....
);

2. The files are already stored in an array. I do not need to call
File::Find to get them. They are in @listOfFiles.

3. The files are already stored in an array. I do not need to call
opendir...; readdir...; closedir...; They are in @listOfFiles.

4. All the files are relative paths. None start with a slash.

5. All the files follow the UNIX convention. None have their directories
separated by a backslash. None of them follow the MS-DOS method of
separating directories in a path. If a backslash appears in a file, it
is part of the name.

6. For each file in the list of files, I want to build a hash of hashes
of hashes, one level for each directory in the path, with the last level
being the basename of the file, that count the number of times the file
appears in the list. The exception is that if there is no directories,
assume it has one (and only one) called '.'; if the file is "file1"
treat it as "./file1"

7. In other words, if the file is "file1" then execute a command that
has the same effect as: $File{ '.' }{ 'file1' } ++;

8. If the file has exactly one directory, called 'dir1', execute a
command that has the same effect as: $File{ 'dir1' }{ 'file3' } ++;

9. If the file has exactly two directories, called 'dir1' and 'subdir1',
execute a command that has the same effect as:
$File{ 'dir1' }{ 'subdir1' }{ 'file5' } ++;

10. Etc.

11. The number of directories may be any number and is not known until
the program is run. The number of directories is undeterminate but finite.

12. Question: is there some Perl command, like map, or some Perl module
I can use in the following algorithm?

foreach my $file ( @listOfFiles )
{
my @directory_or_basename = split /\//, $file;

# add the dot directory if there are none.
unshift @directory_or_basename, '.'
if @directory_or_basename == 1;

my $refHash = \%File;

while( my $directory_or_basename = shift @directory_or_basename )
{
if( @directory_or_basename ) # if this list is not empty,
# $directory_or_basename must
# be a directory.
{
$refHash->{ $directory_or_basename } = {}
unless defined $refHash->{ $directory_or_basename };

$refHash = $refHash->{ $directory_or_basename };
}
else # the list is empty,
# therefore $directory_or_basename must be a basename.
{
$refHash->{ $directory_or_basename } ++;
}
}
}
 
T

Tom

Shawn Corey said:
Hi,

I have a list:

file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
...

I want to count them like:

$File{.}{file1} = count;
$File{.}{file2} = count;
$File{dir1}{file3} = count;
$File{dir1}{file4} = count;
$File{dir1}{subdir1}{file5} = count;
$file{dir2}{file6} = count;
...

Is the a nifty way of doing this in Perl-speak?
Click to expand...

If your list is from a directory, you can do this to count the number
of files in each directory:

#!/usr/bin/perl -w

use strict;
use File::stat;

my %counts;
countFiles($ARGV[0]);
foreach my $file(sort keys %counts)
{
print "$file = $counts{$file}\n";
}

sub countFiles
{
my ($directory) = $_[0];
my $fh;
if(! opendir $fh,$directory)
{
print "Error in accessing $directory.";
return;
}
while( defined(my $file=readdir($fh)))
{
next if $file =~ /^\./;
my $stat = stat("$directory/$file");
if($stat->mode()&040000) { countFiles("$directory/$file") }
else { $counts{"$directory"}++ }
}
closedir($fh);
}

The output generated will be like this:

/list = 2
/list/dir1 = 2
/list/dir1/subdir1 = 1
/list/dir2 = 1

If the output is in a format as you specified, there is no counting.

Tom
ztml.com
 
C

Chief Squawtendrawpet

Shawn said:
Is there some Perl command, like map, or some Perl module
I can use in the following algorithm?
Click to expand...

I don't have a ready-made solution for you or any brilliant
suggestions, but if you stick with your current algorithm you could
make two changes.

Your while() conditional won't do what you want if the dir evaluates
to false -- say, a directory named '0'.

Also, if speed is an issue, you could eliminate one of the
conditionals inside the while() loop like this:

$r = \%file;
while (@dir > 1){
$d = shift @dir;
$r->{$d} = {} unless defined $r->{$d};
$r = $r->{$d};
}
$r->{shift @dir} ++;

Chief S.
 
J

John W. Krahn

Shawn said:
1. I have a list of files like:
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
...
They are stored in an array, call it @listOfFiles:
@listOfFiles = qw(
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
...
);

[snip]

4. All the files are relative paths. None start with a slash.

5. All the files follow the UNIX convention. None have their directories
separated by a backslash. None of them follow the MS-DOS method of
separating directories in a path. If a backslash appears in a file, it
is part of the name.

6. For each file in the list of files, I want to build a hash of hashes
of hashes, one level for each directory in the path, with the last level
being the basename of the file, that count the number of times the file
appears in the list. The exception is that if there is no directories,
assume it has one (and only one) called '.'; if the file is "file1"
treat it as "./file1"

7. In other words, if the file is "file1" then execute a command that
has the same effect as: $File{ '.' }{ 'file1' } ++;

8. If the file has exactly one directory, called 'dir1', execute a
command that has the same effect as: $File{ 'dir1' }{ 'file3' } ++;

9. If the file has exactly two directories, called 'dir1' and 'subdir1',
execute a command that has the same effect as:
$File{ 'dir1' }{ 'subdir1' }{ 'file5' } ++;
Click to expand...


Based on your description, each path/filename will have a count of one.
If you want to base the count on the file name alone then you should the
file name as the key for the count.



John
 
S

Shawn Corey

John said:
Based on your description, each path/filename will have a count of one.
If you want to base the count on the file name alone then you should the
file name as the key for the count.



John
Click to expand...

The path/filename should have a count of one, unless I screwed up
somewhere else in my program. One of the reason I was counting them was
a sanity check to ensure the other parts are working correctly.
 
S

Shawn Corey

Chief said:
I don't have a ready-made solution for you or any brilliant
suggestions, but if you stick with your current algorithm you could
make two changes.

Your while() conditional won't do what you want if the dir evaluates
to false -- say, a directory named '0'.
Click to expand...
The only stupider than naming a directory '0' is to name one '*'.
*WARNING: DO NOT ATTEMPT THIS AT HOME. DO NOT ATTEMPT THIS AT WORK. DO
NOT ATTEMPT THIS ON ANY MACHINE YOU EVER TOUCH.*
Also, if speed is an issue, you could eliminate one of the
conditionals inside the while() loop like this:

$r = \%file;
while (@dir > 1){
$d = shift @dir;
$r->{$d} = {} unless defined $r->{$d};
$r = $r->{$d};
}
$r->{shift @dir} ++;

Chief S.
Click to expand...

Thanks, that's the type of thing I am looking for.
 
M

Mike Flannigan

Tom said:
If your list is from a directory, you can do this to count the number
of files in each directory:

#!/usr/bin/perl -w

use strict;
use File::stat;

my %counts;
countFiles($ARGV[0]);
foreach my $file(sort keys %counts)
{
print "$file = $counts{$file}\n";
}

sub countFiles
{
my ($directory) = $_[0];
my $fh;
if(! opendir $fh,$directory)
{
print "Error in accessing $directory.";
return;
}
while( defined(my $file=readdir($fh)))
{
next if $file =~ /^\./;
my $stat = stat("$directory/$file");
if($stat->mode()&040000) { countFiles("$directory/$file") }
else { $counts{"$directory"}++ }
}
closedir($fh);
}
Click to expand...

That is pretty cool. I'm looking for something like this that will
simply output a list of all files to a text file, maybe along with
some size, date, etc. data. I could probably do this myself
with this script as a start. Thanks alot.


Mike
 
T

Tom

..
..
The only stupider than naming a directory '0' is to name one '*'.
*WARNING: DO NOT ATTEMPT THIS AT HOME. DO NOT ATTEMPT THIS AT WORK. DO
NOT ATTEMPT THIS ON ANY MACHINE YOU EVER TOUCH.*
Click to expand...
..
..

Instead of warning someone not to do that, you should have just simply
taken care of this condition in your program yourself. If you tell
people not to do something, I'm sure someone will do it just to see
what would happen. BTW, I don't think anyone will be smart enough to
name the directory ‘*'.


#!/usr/bin/perl -w

use strict;

my @listOfFiles = qw(
file1
file2
dir1/file3
dir1/file4
dir1/subdir1/file5
dir2/file6
0/sub0/file7
);

my %File;
foreach my $file ( @listOfFiles )
{
my @dir = split /\//, $file;
unshift @dir, '.' if @dir == 1;
my $r = \%File;
while (@dir > 1)
{
my $d = shift @dir;
if(!$d)
{
print "WARNING: directory named '0' is renamed to NULL\n";
$d = "NULL";
}
$r->{$d} = {} unless defined $r->{$d};
$r = $r->{$d};
}
$r->{shift @dir} ++;
}
print "\n\nCOUNTS:\n\n";
opcounts("",\%File);
exit;

sub opcounts
{
my ($name,$files) = @_;

foreach my $file(sort keys %{$files})
{

if($name) { print "{$name}" } # print directory name

# is there a better way than the one below (checking for hash) ???
if($files->{$file} =~ /HASH/) { opcounts($file,$files->{$file}) }
else { print "{$file} = $files->{$file}\n" } # print file name and
total
}
}


Tom
ztml.com
 
T

Tad McClellan

Tom said:
I don't think anyone will be smart enough to
name the directory ‘*'.
Click to expand...
^^^
^^^ supposed to be ?*


You don't have to be very smart at all:

perl -e "mkdir('?*')"
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

os.path.walk() to get full path of all files 5
A script to separate out file names from the path? 12
Convert zip file contents list string to file system 0
Open Multiples Files at same time with multiprocessing - How"declare" dynamically the var? 1
Cleanup file path 6
Module import via sys.path and py2exe 2
What "might" I be doing wrong with this snippet? 6
Do anyone already have code to copy nested files to a flat directory? 4

Members online

No members online now.
Total: 34 (members: 0, guests: 34)
Robots: 129

Forum statistics

Threads
473,787
Messages
2,569,630
Members
45,334
Latest member
66Zeinab9

Latest Threads

Top