How to concatenate 'like' files in a dir?

[wilson_work]

Hi All,
I have a directory of .txt files and need to concatenate all files
belonging to each user (oldest first, no set number per user). The
username (M08x) is embedded in the filename, along with other info. I
would like to delete the smaller individual logs/files once they have
been concatenated. Any advice is greatly appreciated!

Here is a sample of the filenames...
-rw-r--r-- 1 christine christine 28046 Oct 11 21:40
KCD-M087326-NA-server.name-2005Oct11-20:42:14.txt
KCD-M087326-NA-server.name-2005Oct16-23:55:26.txt
KCD-M087326-NA-server.name-2005Oct17-22:44:00.txt
KCD-M087326-NA-server.name-2005Oct19-21:35:06.txt
KCD-M087350-NA-server.name-2005Oct03-19:20:56.txt
KCD-M087350-NA-server.name-2005Oct05-21:13:19.txt
KCD-M087350-NA-server.name-2005Sep27-19:26:09.txt
.......


Thank you,
Christine

 

[A. Sinan Unur]

(e-mail address removed) wrote in @z14g2000cwz.googlegroups.com:

Hi All,
I have a directory of .txt files and need to concatenate all files
belonging to each user (oldest first, no set number per user). The
username (M08x) is embedded in the filename, along with other info. I
would like to delete the smaller individual logs/files once they have
been concatenated. Any advice is greatly appreciated!

Well, please first read the posting guidelines for this group. You have
a much better chance of getting useful help if you post some code.

Here is a sample of the filenames...
-rw-r--r-- 1 christine christine 28046 Oct 11 21:40
KCD-M087326-NA-server.name-2005Oct11-20:42:14.txt
KCD-M087326-NA-server.name-2005Oct16-23:55:26.txt
KCD-M087326-NA-server.name-2005Oct17-22:44:00.txt
KCD-M087326-NA-server.name-2005Oct19-21:35:06.txt
KCD-M087350-NA-server.name-2005Oct03-19:20:56.txt
KCD-M087350-NA-server.name-2005Oct05-21:13:19.txt
KCD-M087350-NA-server.name-2005Sep27-19:26:09.txt
......

Simple ... use a hash ;-)

1. opendir and readdir to read the filesnames
2. Use a capturing regex match to grab the user name
3. Add the filename to the list of filenames belonging to the user
4. Custom sort routine to sort filenames by date component.
5. Open a file for user to write to.
6. Read and write each file in required order.

Here is something quick and dirty to get you started:

#!/usr/bin/perl

use strict;
use warnings;

my %months = ( Jan => '01', Feb => '02', Mar => '03',
Apr => '04', May => '05', Jun => '06',
Jul => '07', Aug => '08', Sep => '09',
Oct => '10', Nov => '11', Dec => '12',
);

my %users;

while (my $filename = <DATA>) {
chomp $filename;
if ( $filename =~ m{
\A
KCD-
(M\d{6})-
NA-server.name-
(\d{4})(\w{3})(\d{2})-
(\d{2}:\d{2}:\d{2})
\.txt
\z
}x ) {
my ($user, $date) = ($1, "$2$months{$3}$4$5");
push @{ $users{$user} }, { filename => $filename, date => $date
};
}
}

for my $user (keys %users) {
print "Files for $user:\n";
my @files = sort {
$b->{date} cmp $a->{date}
} @{ $users{$user} };

print $_->{filename}, "\n" for @files;
print "\n";
}


__DATA__
KCD-M087326-NA-server.name-2005Oct11-20:42:14.txt
KCD-M087326-NA-server.name-2005Oct16-23:55:26.txt
KCD-M087326-NA-server.name-2005Oct17-22:44:00.txt
KCD-M087326-NA-server.name-2005Oct19-21:35:06.txt
KCD-M087350-NA-server.name-2005Oct03-19:20:56.txt
KCD-M087350-NA-server.name-2005Oct05-21:13:19.txt
KCD-M087350-NA-server.name-2005Sep27-19:26:09.txt

D:\Home\asu1\UseNet\clpmisc\dir> files
Files for M087350:
KCD-M087350-NA-server.name-2005Oct05-21:13:19.txt
KCD-M087350-NA-server.name-2005Oct03-19:20:56.txt
KCD-M087350-NA-server.name-2005Sep27-19:26:09.txt

Files for M087326:
KCD-M087326-NA-server.name-2005Oct19-21:35:06.txt
KCD-M087326-NA-server.name-2005Oct17-22:44:00.txt
KCD-M087326-NA-server.name-2005Oct16-23:55:26.txt
KCD-M087326-NA-server.name-2005Oct11-20:42:14.txt

 

[it_says_BALLS_on_your_forehead]

(e-mail address removed) wrote in @z14g2000cwz.googlegroups.com:


Well, please first read the posting guidelines for this group. You have
a much better chance of getting useful help if you post some code.


Simple ... use a hash ;-)

1. opendir and readdir to read the filesnames
2. Use a capturing regex match to grab the user name
3. Add the filename to the list of filenames belonging to the user
4. Custom sort routine to sort filenames by date component.
5. Open a file for user to write to.
6. Read and write each file in required order.

Here is something quick and dirty to get you started:

#!/usr/bin/perl

use strict;
use warnings;

my %months = ( Jan => '01', Feb => '02', Mar => '03',
Apr => '04', May => '05', Jun => '06',
Jul => '07', Aug => '08', Sep => '09',
Oct => '10', Nov => '11', Dec => '12',
);

my %users;

while (my $filename = <DATA>) {
chomp $filename;
if ( $filename =~ m{
\A
KCD-
(M\d{6})-
NA-server.name-
(\d{4})(\w{3})(\d{2})-
(\d{2}:\d{2}:\d{2})
\.txt
\z
}x ) {
my ($user, $date) = ($1, "$2$months{$3}$4$5");
push @{ $users{$user} }, { filename => $filename, date => $date
};
}
}

for my $user (keys %users) {
print "Files for $user:\n";
my @files = sort {
$b->{date} cmp $a->{date}
} @{ $users{$user} };

print $_->{filename}, "\n" for @files;
print "\n";
}


__DATA__
KCD-M087326-NA-server.name-2005Oct11-20:42:14.txt
KCD-M087326-NA-server.name-2005Oct16-23:55:26.txt
KCD-M087326-NA-server.name-2005Oct17-22:44:00.txt
KCD-M087326-NA-server.name-2005Oct19-21:35:06.txt
KCD-M087350-NA-server.name-2005Oct03-19:20:56.txt
KCD-M087350-NA-server.name-2005Oct05-21:13:19.txt
KCD-M087350-NA-server.name-2005Sep27-19:26:09.txt

D:\Home\asu1\UseNet\clpmisc\dir> files
Files for M087350:
KCD-M087350-NA-server.name-2005Oct05-21:13:19.txt
KCD-M087350-NA-server.name-2005Oct03-19:20:56.txt
KCD-M087350-NA-server.name-2005Sep27-19:26:09.txt

Files for M087326:
KCD-M087326-NA-server.name-2005Oct19-21:35:06.txt
KCD-M087326-NA-server.name-2005Oct17-22:44:00.txt
KCD-M087326-NA-server.name-2005Oct16-23:55:26.txt
KCD-M087326-NA-server.name-2005Oct11-20:42:14.txt

i realize this is just an example, but why did you choose to sort such
that the oldest file is last?

 

[A. Sinan Unur]

....

i realize this is just an example, but why did you choose to sort such
that the oldest file is last?

I misread the OP's statement and thought that she wanted oldest last,
not first.

In any case, please quote only the relevant parts of the message to
which you are replying.

Sinan

 

[it_says_BALLS_on_your_forehead]

I misread the OP's statement and thought that she wanted oldest last,
not first.

gotcha. i didn't know if there was some esoteric file concat method
that took a reverse sorted list as an argument. sorry about the
over-quoting.

 

[A. Sinan Unur]

gotcha. i didn't know if there was some esoteric file concat method
that took a reverse sorted list as an argument.

No there isn't (not that I know of ;-). But my code was missing a map
that would have made life much easier:

for my $user (keys %users) {
print "Files for $user:\n";
my @files = map { $_->{filename} }
sort { $a->{date} cmp $b->{date} }
@{ $users{$user} };
system "cat @files > $user.txt";
}

sorry about the over-quoting.

No problem.

Sinan

 

[wilson_work]

Many many thanks! I'm fairly new to Perl and this was a
life/time-saver for me. I really appreciate all the help.

It's just what I needed.
Thank you,
Christine

 

[A. Sinan Unur]

(e-mail address removed) wrote in @g44g2000cwa.googlegroups.com:

Many many thanks! I'm fairly new to Perl and this was a
life/time-saver for me. I really appreciate all the help.

You are welcome.

The best thing you can do to improve your Perl skills would be to read the
posting guidelines, and try to come up with a short script that aims to do
what you want, and exhibits the problems you are having.

Then, post it here, and listen to the critique. You'll find that it works
like an accelerated training course (the likes of which you would have to
pay thousands of dollars to receive).

Sinan