Getting the size of files from a list?

M

Math55

hi, lets say i have this list (its in a file):


32k /var/log/XFree86.0.log
76k /var/log/auth.log
116k /var/log/auth.log.0
8.0k /var/log/auth.log.1.gz
228k /var/log/kdm.log
20k /var/log/kern.log
1.2M /var/log/kern.log.0
12k /var/log/kern.log.1.gz
2.8M /var/log/ksymoops
228k /var/log/ksymoops/20030628062520.ksyms
4.0k /var/log/ksymoops/20030628062520.modules
228k /var/log/ksymoops/20030629062502.ksyms
4.0k /var/log/ksymoops/20030629062502.modules
12k /var/log/ksymoops/20030630.log
228k /var/log/ksymoops/20030630062525.ksyms
4.0k /var/log/ksymoops/20030630062525.modules
12k /var/log/ksymoops/20030701.log
228k /var/log/ksymoops/20030701062504.ksyms

how can i get the size from the 9 files in /var/log/ksymoops and how
the size of the 8 files in /var/log? the list is noct always the same,
it can have more or less directories. i tried it like that so far:

1. get all directories and uniqed them. looks like that:

/var/log
/var/log/ksymoops

2. put them into a hash as keys

3. filtered the list so i get something that can be compared to the
keys
12k /var/log/ksymoops/20030701.log-->/var/log

4. when this already exists inside the hash, i get the size and add it
to the key /var/log as value. i do this with all keys and i think the
correct size for both dirs should be calculated. but the size is
wrong.

this is what i want to have:

/var/log:1720.8
/var/log/ksymoops:3815.2

is therea better and less complicated way to do that?

THANK YOU :)
 
G

Gunnar Hjalmarsson

Math55 said:
hi, lets say i have this list (its in a file):

32k /var/log/XFree86.0.log
76k /var/log/auth.log
116k /var/log/auth.log.0
8.0k /var/log/auth.log.1.gz
228k /var/log/kdm.log
20k /var/log/kern.log
1.2M /var/log/kern.log.0
12k /var/log/kern.log.1.gz
2.8M /var/log/ksymoops
228k /var/log/ksymoops/20030628062520.ksyms
4.0k /var/log/ksymoops/20030628062520.modules
228k /var/log/ksymoops/20030629062502.ksyms
4.0k /var/log/ksymoops/20030629062502.modules
12k /var/log/ksymoops/20030630.log
228k /var/log/ksymoops/20030630062525.ksyms
4.0k /var/log/ksymoops/20030630062525.modules
12k /var/log/ksymoops/20030701.log
228k /var/log/ksymoops/20030701062504.ksyms

how can i get the size from the 9 files in /var/log/ksymoops
and how the size of the 8 files in /var/log?
Click to expand...

this is what i want to have:

/var/log:1720.8
/var/log/ksymoops:3815.2
Click to expand...

One way:

#!/usr/bin/perl
use strict;
use warnings;

my ($dir1, $dir2);
my $path = '/some/path/to/the/file';
open FH, "< $path" or die $!;
while (<FH>) {
my ($value, $sort, $file) = /^([\d.]+)(\w)\t(.+)/;
$value *= 1024 if $sort eq 'M';
if ($file =~ m!^/var/log/ksymoops!) {
$dir2 += $value;
} else {
$dir1 += $value;
}
}
close FH;
print "/var/log:$dir1\n/var/log/ksymoops:$dir2\n";
 
G

Greg Bacon

: how can i get the size from the 9 files in /var/log/ksymoops and how
: the size of the 8 files in /var/log? the list is noct always the same,
: it can have more or less directories. i tried it like that so far:

Here's my implementation:

#! /usr/local/bin/perl

use strict;
use warnings;

use File::Basename;

sub to_bytes {
my $size = shift;

if ($size =~ s/k$//) {
$size *= 1024;
}
elsif ($size =~ s/M$//) {
$size *= 1024 * 1024;
}

$size;
}

sub to_unit {
my $size = shift;

if ($size > 1024 * 1024) {
$size = sprintf "%.1fM", $size / (1024*1024);
}
elsif ($size > 1024) {
$size = sprintf "%.1fk", $size / 1024;
}

$size;
}

## main
my %size;
my @files = <DATA>;

for (@files) {
chomp;
my($size,$path) = split;

my $bytes = to_bytes $size;
my $dir = dirname $path;
$size{$dir} += $bytes;
}

# pass two: zap subdirs from parents' totals
for (@files) {
chomp;
my($size,$path) = split;

next unless exists $size{$path};

my $dir = dirname $path;
my $bytes = to_bytes $size;
$size{$dir} -= $bytes;
}

for (sort { $a cmp $b } keys %size) {
print "$_ - ", to_unit($size{$_}), "\n";
}

__DATA__
32k /var/log/XFree86.0.log
76k /var/log/auth.log
116k /var/log/auth.log.0
8.0k /var/log/auth.log.1.gz
228k /var/log/kdm.log
20k /var/log/kern.log
1.2M /var/log/kern.log.0
12k /var/log/kern.log.1.gz
2.8M /var/log/ksymoops
228k /var/log/ksymoops/20030628062520.ksyms
4.0k /var/log/ksymoops/20030628062520.modules
228k /var/log/ksymoops/20030629062502.ksyms
4.0k /var/log/ksymoops/20030629062502.modules
12k /var/log/ksymoops/20030630.log
228k /var/log/ksymoops/20030630062525.ksyms
4.0k /var/log/ksymoops/20030630062525.modules
12k /var/log/ksymoops/20030701.log
228k /var/log/ksymoops/20030701062504.ksyms

I make two passes. On the first, I sum the sizes, using dirname
from the File::Basename module to grab the parent directory's name.
On the second pass, I substract reported directories' sizes from
their parents' totals (e.g., /var//var/log/ksymoops in this example).

: this is what i want to have:
:
: /var/log:1720.8
: /var/log/ksymoops:3815.2
:
: is therea better and less complicated way to do that?

My results are below:

% ./try
/var/log - 1.7M
/var/log/ksymoops - 948.0k

Hope this helps,
Greg
 
M

Math55

Abigail said:
Math55 ([email protected]) wrote on MMMDXCIII September MCMXCIII in
<URL:][ hi, lets say i have this list (its in a file):
][
][
][ 32k /var/log/XFree86.0.log
][ 76k /var/log/auth.log
][ 116k /var/log/auth.log.0
][ 8.0k /var/log/auth.log.1.gz
][ 228k /var/log/kdm.log
][ 20k /var/log/kern.log
][ 1.2M /var/log/kern.log.0
][ 12k /var/log/kern.log.1.gz
][ 2.8M /var/log/ksymoops
][ 228k /var/log/ksymoops/20030628062520.ksyms
][ 4.0k /var/log/ksymoops/20030628062520.modules
][ 228k /var/log/ksymoops/20030629062502.ksyms
][ 4.0k /var/log/ksymoops/20030629062502.modules
][ 12k /var/log/ksymoops/20030630.log
][ 228k /var/log/ksymoops/20030630062525.ksyms
][ 4.0k /var/log/ksymoops/20030630062525.modules
][ 12k /var/log/ksymoops/20030701.log
][ 228k /var/log/ksymoops/20030701062504.ksyms


Something like (untested code):

my %total;
while (<>) {
my ($size, $dir) = m!(\S+)\s+(\S+)/! or next;
if ($size =~ s/k$//) {$size *= 1024}
elsif ($size =~ s/M$//) {$size *= 1024 * 1024}
$total {$dir} += $size;
}
while (my ($dir, $size) = each %total) {
printf "%s:%.1f\n" => $dir, $size / 1024;
}



Abigail
Click to expand...



hi, thanks you. i will try that and post back:) as soon as possible.
 
M

Math55

: how can i get the size from the 9 files in /var/log/ksymoops and how
: the size of the 8 files in /var/log? the list is noct always the same,
: it can have more or less directories. i tried it like that so far:

Here's my implementation:

#! /usr/local/bin/perl

use strict;
use warnings;

use File::Basename;

sub to_bytes {
my $size = shift;

if ($size =~ s/k$//) {
$size *= 1024;
}
elsif ($size =~ s/M$//) {
$size *= 1024 * 1024;
}

$size;
}

sub to_unit {
my $size = shift;

if ($size > 1024 * 1024) {
$size = sprintf "%.1fM", $size / (1024*1024);
}
elsif ($size > 1024) {
$size = sprintf "%.1fk", $size / 1024;
}

$size;
}

## main
my %size;
my @files = <DATA>;

for (@files) {
chomp;
my($size,$path) = split;

my $bytes = to_bytes $size;
my $dir = dirname $path;
$size{$dir} += $bytes;
}

# pass two: zap subdirs from parents' totals
for (@files) {
chomp;
my($size,$path) = split;

next unless exists $size{$path};

my $dir = dirname $path;
my $bytes = to_bytes $size;
$size{$dir} -= $bytes;
}

for (sort { $a cmp $b } keys %size) {
print "$_ - ", to_unit($size{$_}), "\n";
}

__DATA__
32k /var/log/XFree86.0.log
76k /var/log/auth.log
116k /var/log/auth.log.0
8.0k /var/log/auth.log.1.gz
228k /var/log/kdm.log
20k /var/log/kern.log
1.2M /var/log/kern.log.0
12k /var/log/kern.log.1.gz
2.8M /var/log/ksymoops
228k /var/log/ksymoops/20030628062520.ksyms
4.0k /var/log/ksymoops/20030628062520.modules
228k /var/log/ksymoops/20030629062502.ksyms
4.0k /var/log/ksymoops/20030629062502.modules
12k /var/log/ksymoops/20030630.log
228k /var/log/ksymoops/20030630062525.ksyms
4.0k /var/log/ksymoops/20030630062525.modules
12k /var/log/ksymoops/20030701.log
228k /var/log/ksymoops/20030701062504.ksyms

I make two passes. On the first, I sum the sizes, using dirname
from the File::Basename module to grab the parent directory's name.
On the second pass, I substract reported directories' sizes from
their parents' totals (e.g., /var//var/log/ksymoops in this example).

: this is what i want to have:
:
: /var/log:1720.8
: /var/log/ksymoops:3815.2
:
: is therea better and less complicated way to do that?

My results are below:

% ./try
/var/log - 1.7M
/var/log/ksymoops - 948.0k

Hope this helps,
Greg
Click to expand...


hi, i got this to work. now i can submit an argument and i get the
size of this argument. but there is one little problem. when a
directory has a size near 1mb i get a size like 1020.4 for example.
how can i make this a bit more accurate?

THANKS FOR YOUR EFFORTS:)
 
J

Joe Smith

directory has a size near 1mb i get a size like 1020.4 for example.
how can i make this a bit more accurate?
Click to expand...

It's already too accurate. If you want to avoid having four digits
before the decimal point, just change the threshold. For example:

:: if ($size > 999.949 * 1024) {

That way the next number after "999.9k" will be "1.0M".

-Joe
 
M

Math55

You can use one of my old programs to do that.

unix% lsf -du -x -o files.lsf /
unix% head files.du
K-bytes dirs files directory
2756727 25 13 .
1092378 33 14 usr
1063652 5 0 utility
716172 23 8 utility/local
478649 33 0 var
394761 30 126 utility/local/lib
247475 6 3 usr/openwin
195077 3 21 var/log
194757 8 1 utility/gnu

Look for 'lsf' in http://www.inwap.com/mybin/ .
(I tried to e-mail that file to you but <[email protected]> bounced.)
-Joe
Click to expand...

hi, thanks. i tried it, but somehow i do not understand the output.
how can i add this to the program you posted before?

THANKS
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

array question 6
Select files based on text list of filenames(part of the name:date) with condition 0
Getting the actual size of a sparse file 9
How to speed up this slow part of my program 14
Getting list of the files from a virtual directory 4
How to only get a list of the names of the non-directory files incurrent directory ('.')? 8
Calculate directorysize from a list? 7
Size of list pointed by key, in a hash 5

Members online

No members online now.
Total: 39 (members: 0, guests: 39)
Robots: 443

Forum statistics

Threads
473,770
Messages
2,569,584
Members
45,075
Latest member
MakersCBDBloodSupport

Latest Threads

Top