need some help excluding with file::find::rule

Discussion in 'Perl Misc' started by solaristar, Jan 22, 2010.

  1. solaristar

    solaristar Guest

    (I think I may have posted this msg in the wrong group before, so my
    apologies for a double post, i believe this is the right place)

    ==

    First please forgive me if this is the wrong way to go about asking
    this question.

    I have a script im working on that is looking to only get the
    filenames with ".mw.|.cw.|.uw." and exclude any filenames (which
    happen to be FQDNs of servers) that do not have that criteria

    the structure to search is /data*/backups/$server/daily.0/$server
    (where $server would have the .mw.|.cw.|.uw. characteristic)

    this is what I have thus far, I dont feel this is the fastest way to
    go about doing this (im not sure), I also want to make sure to exclude
    and not even "parse" any dirs that dont have the afore mentioned
    criteria,

    any feedback is appreciated

    ====

    #!/usr/bin/perl

    use strict;
    use warnings;
    use File::Find::Rule;
    use File::Basename qw/basename dirname/;

    my @data_dir =
    qw { /data/backups }; # list here the data dir if you want to
    loop on it.
    foreach my $dir (@data_dir) {
    print "looking at $dir..\n";
    my ( $bkpcount, $dbcount ) = 0; # db and backup file counter

    # Gather server name with .mw, .cw, .uw on fqdn
    my %server_w_log;
    # This part will search for every directory with .mw, .uw. cw and
    take the base name as key to hash
    opendir( DIR, $dir ) or warn "can't open $dir\n";
    my @servers = readdir(DIR);
    foreach my $server (@servers) {
    next if $server =~ m/^\./;
    %server_w_log =
    map { my $tempfile = basename $_; $tempfile => $_ }
    File::Find::Rule->directory->name(qr/.*\.(mw|uw|cw).*/)
    ->in("$dir/$server/daily.0");

    print "server is $server..\n";

    }
    close(DIR);
    ...etc...

    (there's more to the script but this is the first part that's giving
    me problems.

    any help is greatly appreciated.
     
    solaristar, Jan 22, 2010
    #1
    1. Advertising

  2. solaristar wrote:
    >
    > First please forgive me if this is the wrong way to go about asking
    > this question.
    >
    > I have a script im working on that is looking to only get the
    > filenames with ".mw.|.cw.|.uw." and exclude any filenames (which
    > happen to be FQDNs of servers) that do not have that criteria
    >
    > the structure to search is /data*/backups/$server/daily.0/$server
    > (where $server would have the .mw.|.cw.|.uw. characteristic)


    It looks like you could use a file glob to get the list of file names
    you need, for example (UNTESTED):

    my $server = '.[cmu]w';

    my @files = glob "/data*/backups/$server/daily.0/*$server";


    > this is what I have thus far, I dont feel this is the fastest way to
    > go about doing this (im not sure), I also want to make sure to exclude
    > and not even "parse" any dirs that dont have the afore mentioned
    > criteria,
    >
    > any feedback is appreciated
    >
    > ====
    >
    > #!/usr/bin/perl
    >
    > use strict;
    > use warnings;
    > use File::Find::Rule;
    > use File::Basename qw/basename dirname/;
    >
    > my @data_dir =
    > qw { /data/backups }; # list here the data dir if you want to
    > loop on it.
    > foreach my $dir (@data_dir) {
    > print "looking at $dir..\n";
    > my ( $bkpcount, $dbcount ) = 0; # db and backup file counter


    That is the same as saying:

    my $bkpcount = 0;
    my $dbcount;

    If you want both variables initialized to 0 then you have to assign to both:

    my ( $bkpcount, $dbcount ) = ( 0, 0 );

    Or:

    my ( $bkpcount, $dbcount ) = ( 0 ) x 2;

    Or:

    my $bkpcount = my $dbcount = 0;


    > # Gather server name with .mw, .cw, .uw on fqdn
    > my %server_w_log;
    > # This part will search for every directory with .mw, .uw. cw and
    > take the base name as key to hash
    > opendir( DIR, $dir ) or warn "can't open $dir\n";


    You should include the $! variable in the error message so you know why
    opendir() failed.


    > my @servers = readdir(DIR);


    Even though opendir() failed you are still trying to read from the
    invalid directory handle?


    > foreach my $server (@servers) {
    > next if $server =~ m/^\./;
    > %server_w_log =
    > map { my $tempfile = basename $_; $tempfile => $_ }
    > File::Find::Rule->directory->name(qr/.*\.(mw|uw|cw).*/)


    The '.*' at the beginning and end are superfluous, it should be just
    qr/\.(mw|uw|cw)/, or even better qr/\.[muc]w/.


    > ->in("$dir/$server/daily.0");
    >
    > print "server is $server..\n";
    >
    > }
    > close(DIR);
    > ..etc...
    >
    > (there's more to the script but this is the first part that's giving
    > me problems.
    >
    > any help is greatly appreciated.




    John
    --
    The programmer is fighting against the two most
    destructive forces in the universe: entropy and
    human stupidity. -- Damian Conway
     
    John W. Krahn, Jan 22, 2010
    #2
    1. Advertising

  3. solaristar

    solaristar Guest

    John,

    thanks for the reply, ok so i read your notes and tried to incorporate
    them, I must still be doing something wrong as i still get non ",[muc]
    w." servers listed via the last print

    here's the code

    #!/usr/bin/perl

    use strict;
    use warnings;
    use File::Find::Rule;
    use File::Basename qw/basename dirname/;

    $|=1;

    my @data_dir =
    qw { /data/backups }; # list here the data dir if you want to
    loop on it.
    foreach my $dir (@data_dir) {
    print "looking at $dir..\n";

    my $server = '.[cmu]w';
    my @files = glob "/data*/backups/$server/daily.0/$server";

    my ( $bkpcount, $dbcount ) = ( 0, 0 );

    my %server_w_log;
    opendir( DIR, $dir ) or warn "can't open $dir\n";
    my @servers = readdir(DIR);
    foreach my $server (@servers) {
    next if $server =~ m/^\./;
    %server_w_log =
    map { my $tempfile = basename $_; $tempfile => $_ }
    File::Find::Rule
    ->directory
    ->name(qr/\.[muc]w/)
    ->in("$dir/$server/daily.0");

    print "server is $server..\n";

    }
    close(DIR);

    }
     
    solaristar, Jan 22, 2010
    #3
  4. solaristar

    solaristar Guest

    On Jan 22, 2:46 pm, Tad McClellan <> wrote:
    > solaristar <> wrote:
    > > John,

    >
    > John who?
    >
    > Please learn the proper way of composing a followup:
    >
    >    http://web.presby.edu/~nnqadmin/nnq/nquote.html
    >
    > Have you seen the Posting Guidelines that are posted here frequently?


    Ok pardon me for not being an avid "Usenet" subscriber, lesson learned

    >
    > > thanks for the reply, ok so i read your notes

    >
    > What notes?
    >
    > (we wouldn't have to ask if you had quoted the part of John's post
    > that you are referring to...)
    >


    Understood.

    > > I must still be doing something wrong

    >
    > You are doing many things wrong.
    >
    > > $|=1;

    >
    > Why do you think that you need to enable autoflush?
    >


    Trying to debug the program

    > > my @data_dir =
    > >   qw { /data/backups };    # list here the data dir if you want to
    > > loop on it.

    >
    > You should either
    >
    >    1) ensure all lines are shorter than your newsreader's line length
    > or
    >    2) figure out how to disable your newsreader's word wrapping
    >
    > so that the code you post is code that can be run without syntax errors.
    >


    No clue how to do that, I'll look into it. I'm simply using firefox

    > > my $server = '.[cmu]w';

    >
    > That matches only 3 things:
    >     .cw
    >     .mw
    >     .uw
    >
    > I thought you wanted to match any string that *ended* with those:
    >
    >     my $server = '*.[cmu]w';
    >
    > What you have matches only if the string is *equal* to one of those.
    >


    My intent is to return any "filename" that contains .[umc]w. within
    the name; for example /data/backups/db01.foobar.mw.blahbalh.com or /
    data3/backups/db02.foobar2.uw.blahblah.com, but NOT
    db01.blahbalah.ml.blahblah.com

    > > my @files = glob "/data*/backups/$server/daily.0/$server";

    >
    > What is the point of this operation?
    >
    > You never use the @files variable again...
    >


    that was for later in the script, i just posted a snippet


    > --
    > Tad McClellan
    > email: perl -le "print scalar reverse qq/moc.liamg\100cm.j.dat/"
     
    solaristar, Jan 23, 2010
    #4
  5. solaristar

    solaristar Guest

    so i did some more work on my script and i made some headway, wanted
    to get another critique, right now its having issues with taking
    $server and then getting the $server_w_dir printed properly,

    (im not replying to anyone so i removed the previous comments,
    hopefully that was the right thing to do :p)

    here's the script for critique

    #!/usr/bin/perl

    use 5.010;
    use strict;
    use warnings;
    use File::Find::Rule;
    use File::Basename qw/basename dirname/;

    $| = 1;

    my @data_dir =
    qw { /data/backups };
    foreach my $dir (@data_dir) {
    print "looking at $dir..\n";
    my ( $bkpcount, $dbcount ) = 0;
    my %server_w_log;
    my %server_w_dir;

    opendir( DIR, $dir ) or warn "can't open $dir\n";
    my @servers = readdir(DIR);
    foreach my $server (@servers) {
    my $str = qw#\.(m|c|u)[w]\.#;
    chomp $server;
    next if $server =~ m/^\./;
    next unless $server =~ m/$str/;
    print "$server matched\n";
    %server_w_log =
    map { my $tempfile = basename $_; $tempfile => $_ }
    File::Find::Rule->directory->name($server)
    ->in("$dir/$server/daily.0");

    %server_w_dir =
    map { my $tempfile = basename $_; $tempfile => $_ }
    File::Find::Rule->directory->name(qr/Microsoft\sSQL\sServer/
    i)
    ->in("$dir/$server/daily.0/$server");

    }
    close(DIR);

    # Run this lets find out what it got

    foreach my $server ( keys %server_w_log ) {
    print "We have this server:\n";
    print "$server : \t path is : $server_w_log{$server} \n";
    }

    # Gather server with sql dirctory

    my @backups;
    my $total_db_count = 0;
    ; # this should contain all dbs less than 1 day

    # Gather servers with maintenance log
    for ( keys %server_w_log ) {
    @backups =
    File::Find::Rule->file->name(qr/.*\.(bak|BAK)$/)->modified("
    =< 1")
    ->in("$server_w_log{$_}");
    my $log = "$server_w_log{$_}/C/maintenanceplan.log";
    print "log path is: $log \n";
    print "$server_w_log{$_}\n";

    # If maintenanceplan.log exist, compare the db backup against it,
    output the list on stdout

    if ( -f $log ) {
    $server_w_log{$_} = $log;
    compare_db_w_log( $log, $_, \@backups, \$bkpcount, \
    $dbcount );
    }
    else { print "$log none existing\n"; }
    $total_db_count += scalar @backups;
    }
     
    solaristar, Jan 23, 2010
    #5
  6. solaristar wrote:
    >
    > My intent is to return any "filename" that contains .[umc]w. within
    > the name; for example /data/backups/db01.foobar.mw.blahbalh.com or /
    > data3/backups/db02.foobar2.uw.blahblah.com, but NOT
    > db01.blahbalah.ml.blahblah.com


    In the *code* you posted earlier your path has the string '/daily.0/' in
    it but it is not present in your example above so what does the *actual*
    path name look like?


    John
    --
    The programmer is fighting against the two most
    destructive forces in the universe: entropy and
    human stupidity. -- Damian Conway
     
    John W. Krahn, Jan 23, 2010
    #6
  7. solaristar

    Uri Guttman Guest

    >>>>> "s" == solaristar <> writes:

    s> $| = 1;

    again, not needed. it doesn't help with debugging as you think it does.

    s> my @data_dir =
    s> qw { /data/backups };
    s> foreach my $dir (@data_dir) {
    s> print "looking at $dir..\n";
    s> my ( $bkpcount, $dbcount ) = 0;
    s> my %server_w_log;
    s> my %server_w_dir;

    s> opendir( DIR, $dir ) or warn "can't open $dir\n";

    why warn and not die or next? if the dir can't be opened, why continue?

    s> my @servers = readdir(DIR);
    s> foreach my $server (@servers) {

    no need for @servers:

    foreach my $server (readdir(DIR)) {

    or use File::Slurp which means no need for the opendir and closedir
    calls. it also strips . and .. for you.

    foreach my $server (read_dir( $dir )) {


    s> my $str = qw#\.(m|c|u)[w]\.#;

    that is constant and should be factored out of this sub.
    the parens are grabbing when a char class is simpler and faster. don't
    use alternate delimiters unless / is inside a regex. choose a better
    name than $str which conveys no information about this. also why use qw
    when you only have one 'word' there. you use this as a regex so it
    should be qr.

    my $suff_re = qr/\.[mcu]w\./ ;

    s> chomp $server;

    useless use of chomp. dirs will not have extraneous newlines. if there
    is a newline it will be real and in the filename.

    s> next if $server =~ m/^\./;
    s> next unless $server =~ m/$str/;
    s> print "$server matched\n";
    s> %server_w_log =
    s> map { my $tempfile = basename $_; $tempfile => $_ }
    s> File::Find::Rule->directory->name($server)
    -> in("$dir/$server/daily.0");

    s> %server_w_dir =
    s> map { my $tempfile = basename $_; $tempfile => $_ }
    s> File::Find::Rule->directory->name(qr/Microsoft\sSQL\sServer/
    s> i)
    -> in("$dir/$server/daily.0/$server");

    s> }
    s> close(DIR);

    that should be closedir. if you used lexical handles, it would close
    upon scope exit and the closedir call wouldn't even be needed. or use
    file::slurp as mentioned above

    no comments on the guts and logic since i haven't followed your
    requirements.

    uri

    --
    Uri Guttman ------ -------- http://www.sysarch.com --
    ----- Perl Code Review , Architecture, Development, Training, Support ------
    --------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
     
    Uri Guttman, Jan 23, 2010
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jon Maz

    Excluding a file from VS.Net Project

    Jon Maz, Nov 18, 2004, in forum: ASP .Net
    Replies:
    10
    Views:
    550
    Jon Maz
    Nov 19, 2004
  2. Andy Fish

    excluding some rows when databinding

    Andy Fish, Mar 31, 2005, in forum: ASP .Net
    Replies:
    1
    Views:
    704
    Brock Allen
    Mar 31, 2005
  3. Replies:
    0
    Views:
    1,377
  4. C++ Newbie
    Replies:
    1
    Views:
    312
    Richard Herring
    Oct 14, 2008
  5. Arvin Portlock

    File::Find::Rule for files younger than 24 hours

    Arvin Portlock, Jan 24, 2006, in forum: Perl Misc
    Replies:
    8
    Views:
    315
    Arvin Portlock
    Jan 24, 2006
Loading...

Share This Page