A Grep and a couple Awks <and a lot of Tassilo help>

Discussion in 'Perl Misc' started by Agrapha, Feb 2, 2004.

  1. Agrapha

    Agrapha Guest

    First let me state that Tassilo v. Parseval helped me a bunch last
    April. I would not have been able to complete this project without his
    help. I promised last year to post a working copy of this script. It's
    not perfect but it works. I'll clean it up a bit more this year.

    Tassilo, would you e_mail me directly in order that I may thank you
    properly? mail to: nethaniel at box201.com

    Up top is the results from the script. The way this script was forged
    is documented in the history of this forum. Search on "Grep Once Awk
    Twice" to find it. The names and numbers here have been slightly
    altered to protect the innocent.

    [agrapha@f3dd43-01 scripts]$ ./triag.pl 5555554999 0127 lon1

    total_calls 523 total_failures 14 access number 5555554999
    success rate for dates selected: 97.3231357552581%

    ErrorCodes / Totals
    -------------------
    31,10 / 3
    31,185 / 8
    33,185 / 1
    75,81 / 1
    75,185 / 1

    PhoneNumber / TotalFail
    -----------------------
    5550001111 / 2
    5550001118 / 2
    5550001110 / 1
    5550001131 / 1
    5550001811 / 4
    5550001114 / 1
    5550001211 / 1
    5550001191 / 1
    5550001119 / 1

    Number / Error / Totals
    -----------------------
    5550001110 - 75,185,75,81 - 2
    5550001111 - 31,185 - 1
    5550001114 - 31,185 - 2
    5550001118 - 31,185 - 4
    5550001119 - 31,10 - 1
    5550001131 - 31,10 - 1
    5550001191 - 31,10 - 1
    5550001211 - 33,185 - 1
    5550001811 - 31,185 - 1


    triag.pl script complete

    [agrapha@f3dd43-01 scripts]$ cat triag.pl

    #!/usr/bin/perl -w
    use strict;

    #####
    # This script was made possible by the excellent help
    # from the people on the perl usenet forum.
    # The project would not have been possible without them.
    # triage.pl was designed to look up an number and gather
    # info about it. We begin by making sure they enter one
    # on the command line or give them an example if they
    # fail to do so.
    ###


    if (!$ARGV[0] or !$ARGV[1] or !$ARGV[2]) {
    print "Syntax is \"./triag.pl accessnumber yyyymmdd mkt\"\n";
    print "example: ./triag.pl 2015551212 20040101 lon1
    die "Include an access number, date and market when starting
    Triag\n";
    }

    ### Variable Initialization

    my %error_codes;
    my %error_users;
    my %err_per_usr;
    my $code = 0;
    my $users = 0;
    my $badusr = 0;

    ### End Variable Init

    #####
    # Next we go and find all the radius log files
    # the push here will push the line into @selected
    # if the line matches @ARGV (the command line value)
    ###
    my @selected=`zgrep $ARGV[0]
    ../radiuslog/$ARGV[1]/server.$ARGV[2].Detail.$ARGV[1].gz`;
    my @badcalls= grep /,0 /, @selected;

    #####
    # Ok so now we have a zgrep pushed into our
    # @selected array. This is because the files are gunzipped.
    # That give us all the calls made to the access number.
    # Next we need to sort out the calls with zero session time
    # by looking for /,0_/ that is a space after the ,0
    # because typically failed calls have zero browsing time
    ###

    #####
    # now with the failed calls in @badcalls we filter and sort
    ###

    foreach (@badcalls[0..$#badcalls]) {
    my @errors = split /\s+/, $_;
    $code = $errors[7];
    $users = $errors[3];
    $error_codes{$code}++;
    $error_users{$users}++;
    }

    #####
    # ok here is a difficult part to my script.
    # Thanks Tassilo,
    # this script would not have been possible without him.
    ###

    my %errorsz;
    foreach (@badcalls[0..$#badcalls]) {
    my ($phonez, $err_codez) = (split)[3,7];
    $errorsz{ $phonez }->{ $err_codez }++;
    }



    #####
    # So the phone-numbers are the primary keys of the hash and the
    # error-codes the keys of the nested hash.
    # Next we perform a little math to get and average call sucess ratio
    ###

    my $total = @selected;
    my $fail = @badcalls;
    print "\ntotal_calls $total\ttotal_failures $fail\taccess number
    $ARGV[0]\n";
    print "success rate for dates selected: ", (100 - (($fail / $total) *
    100)), "%\n";

    #####
    # Finally I print out the information to the screen
    ###
    print "\nErrorCodes / Totals\n";
    print "-------------------\n";
    foreach my $key (keys %error_codes) {
    print "$key\t\/\t$error_codes{$key}\n";
    }
    print "\nPhoneNumber / TotalFail\n";
    print "-----------------------\n";
    foreach my $key (keys %error_users) {
    print "$key\t\/\t$error_users{$key}\n";
    }
    #####
    # this one is now fixed. Adding the perfect flourish
    # to my first perl program.
    ###

    print "\nNumber / Error / Totals\n";
    print "-----------------------\n";

    for my $n (sort { $a <=> $b } keys %errorsz) {
    # $errorsz{ $n } is now itself a reference to a hash
    my %codesz = %{ $errorsz{ $n } };

    # compute the total numbers of errors for this phone-number
    my $qty;
    for ( keys %codesz ) {
    $qty += $codesz{ $_ };
    }

    # display the stats for a given phone-number $n
    print "$n - ", join (",", keys %codesz), " - $qty\n";
    }

    print "\n\ntriag.pl script complete\n";
     
    Agrapha, Feb 2, 2004
    #1
    1. Advertising

  2. Agrapha

    Uri Guttman Guest

    >>>>> "A" == Agrapha <> writes:

    A> #!/usr/bin/perl -w
    A> use strict;

    good!


    A> if (!$ARGV[0] or !$ARGV[1] or !$ARGV[2]) {

    eww!
    if ( @ARGV != 3 ) {



    A> print "Syntax is \"./triag.pl accessnumber yyyymmdd mkt\"\n";
    A> print "example: ./triag.pl 2015551212 20040101 lon1
    A> die "Include an access number, date and market when starting
    A> Triag\n";

    why not a single die? and lose the \" stuff. here docs to the rescue!!

    die <<DIE ;
    Syntax is "./triag.pl accessnumber yyyymmdd mkt"
    example: ./triag.pl 2015551212 20040101 lon1
    Include an access number, date and market when starting Triag
    DIE

    what i like to do is make a usage sub like this:

    sub usage {

    my ( $err_text ) = @_ ;

    $err_text ||= '' ; # stop undef warnings

    die <<DIE ;
    $err_text

    Syntax is "./triag.pl accessnumber yyyymmdd mkt"
    example: ./triag.pl 2015551212 20040101 lon1
    Include an access number, date and market when starting Triag
    DIE

    }

    then you can call that in different places in the arg parsing stuff like
    this:

    usage( "missing arguments ) if @ARGV < 3 ;
    usage( "too many arguments ) if @ARGV > 3 ;


    A> }

    A> ### Variable Initialization

    A> my %error_codes;
    A> my %error_users;
    A> my %err_per_usr;
    A> my $code = 0;
    A> my $users = 0;
    A> my $badusr = 0;


    declare your variables in the tightest scope possible. these are file
    scoped globals and are possibly too visible.

    A> my @selected=`zgrep $ARGV[0]
    A> ./radiuslog/$ARGV[1]/server.$ARGV[2].Detail.$ARGV[1].gz`;

    CPAN has modules to read .gz files so you can save on a fork.

    A> my @badcalls= grep /,0 /, @selected;

    A> #####
    A> # ok here is a difficult part to my script.
    A> # Thanks Tassilo,
    A> # this script would not have been possible without him.
    A> ###

    A> my %errorsz;

    that is a better place to declare something, right before its first use.

    also use _ to make your var names more readable:

    my %errors_z;

    A> foreach (@badcalls[0..$#badcalls]) {
    A> my ($phonez, $err_codez) = (split)[3,7];
    A> $errorsz{ $phonez }->{ $err_codez }++;
    ^^

    no need for that -> between pairs of nest derefs (hash or array).

    A> }

    A> my $total = @selected;
    A> my $fail = @badcalls;
    A> print "\ntotal_calls $total\ttotal_failures $fail\taccess number
    A> $ARGV[0]\n";
    A> print "success rate for dates selected: ", (100 - (($fail / $total) *
    A> 100)), "%\n";

    here docs again! precalculate the rate value:

    my $rate = 100 - (($fail / $total) * 100)

    print <<TOTALS ;
    total_calls $total total_failures $fail access number $ARGV[0]
    success rate for dates selected: $rate
    TOTALS

    A> #####
    A> # Finally I print out the information to the screen
    A> ###
    A> print "\nErrorCodes / Totals\n";
    A> print "-------------------\n";
    A> foreach my $key (keys %error_codes) {
    A> print "$key\t\/\t$error_codes{$key}\n";
    ^^

    there is no need to escape / in a plain double quoted string. that is
    only needed (and another delimiter should be chosen) in regex ops.
    A> }
    A> print "\nPhoneNumber / TotalFail\n";
    A> print "-----------------------\n";
    A> foreach my $key (keys %error_users) {
    A> print "$key\t\/\t$error_users{$key}\n";
    A> }

    again, i would build it all up in a single string and then print it. it
    is cleaner, faster and lets you control where you print it all. and
    statement modifiers will make it look even better:

    my $text = '' ;

    $text .= <<TXT ;

    ErrorCodes / Totals
    -------------------
    TXT

    $text .= <<TXT foreach keys %error_codes ;
    $_ / $error_codes{$_}
    TXT

    $text .= <<TXT ;

    PhoneNumber / TotalFail
    -----------------------
    TXT

    $text .= <<TXT foreach keys %error_users ;
    $_ / $error_users{$_}
    TXT

    etc.

    notice how much easier it is to read what will be printed and also to
    line it up? no extra " or \n or \t chars to provide visual noise. of
    course some don't like here docs as much as i do but they are infidels! :)


    A> for my $n (sort { $a <=> $b } keys %errorsz) {
    A> # $errorsz{ $n } is now itself a reference to a hash
    A> my %codesz = %{ $errorsz{ $n } };

    no need to copy that hash. see below

    A> # compute the total numbers of errors for this phone-number
    A> my $qty;
    A> for ( keys %codesz ) {

    for ( keys %{ $errorsz{ $n } } ) {

    A> $qty += $codesz{ $_ };
    A> }

    and that can be shrunk with a statement modifier (i like those too!).

    $qty += %{ $errorsz{ $_ } } for keys %{ $errorsz{ $n } } ;

    but you are only accessing and summing the values of the subhash and you
    don't need the keys for that. use the values function:

    $qty += $_ for values %{ $errorsz{ $n } } ;

    that is all you need there AFIACT from your code.


    A> # display the stats for a given phone-number $n
    A> print "$n - ", join (",", keys %codesz), " - $qty\n";
    A> }

    A> print "\n\ntriag.pl script complete\n";

    here docs there as well.

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
     
    Uri Guttman, Feb 2, 2004
    #2
    1. Advertising

  3. Also sprach Agrapha:

    > First let me state that Tassilo v. Parseval helped me a bunch last
    > April. I would not have been able to complete this project without his
    > help. I promised last year to post a working copy of this script. It's
    > not perfect but it works. I'll clean it up a bit more this year.
    >
    > Tassilo, would you e_mail me directly in order that I may thank you
    > properly? mail to: nethaniel at box201.com


    Worry not. I did get your email after christmas. Btw, most people in
    this group wont expect an explicit thank you (but it can't hurt to
    send one nonetheless of course).

    However, it's a good idea to make a post summing up your experiences and
    showing your results as you do right now.

    > [agrapha@f3dd43-01 scripts]$ cat triag.pl
    >
    > #!/usr/bin/perl -w
    > use strict;
    >
    > #####
    > # This script was made possible by the excellent help
    > # from the people on the perl usenet forum.
    > # The project would not have been possible without them.
    > # triage.pl was designed to look up an number and gather
    > # info about it. We begin by making sure they enter one
    > # on the command line or give them an example if they
    > # fail to do so.
    > ###
    >
    >
    > if (!$ARGV[0] or !$ARGV[1] or !$ARGV[2]) {
    > print "Syntax is \"./triag.pl accessnumber yyyymmdd mkt\"\n";
    > print "example: ./triag.pl 2015551212 20040101 lon1
    > die "Include an access number, date and market when starting
    > Triag\n";
    > }


    This is mostly a question of good style: usage-messages should go to
    stderr really:

    if (@ARGV != 3) {
    warn <<EOWARN;
    Syntax is $0 accessnumber yyyymmdd mkt
    example: $0 2015551212 20040101 lon1
    Include an access number, date and market when starting Triag
    EOWARN
    exit(1);
    }

    I use $0 because it is shorter and because it contains the exact path
    which you used to call your script.

    > ### Variable Initialization
    >
    > my %error_codes;
    > my %error_users;
    > my %err_per_usr;
    > my $code = 0;
    > my $users = 0;
    > my $badusr = 0;


    Some of these variables shouldn't have file-scope. As far as I can see,
    $code and $users is only used in a for-loop later on. Better restrict
    their scope to this loop.

    > ### End Variable Init
    >
    > #####
    > # Next we go and find all the radius log files
    > # the push here will push the line into @selected
    > # if the line matches @ARGV (the command line value)
    > ###
    > my @selected=`zgrep $ARGV[0]
    > ./radiuslog/$ARGV[1]/server.$ARGV[2].Detail.$ARGV[1].gz`;
    > my @badcalls= grep /,0 /, @selected;


    Maybe a pipe-open is more appropriate here:

    open ZGREP, "|", "zgrep $ARGV[0] ./radiuslog/..."
    or die "Could not spawn zgrep: $!";
    my (@badcalls, $total);
    while (<ZGREP>) {
    $total++;
    push @badcalls, $_ if /,0 /;
    }
    close ZGREP;

    This needs less memory and gets rid of @selected altogether.

    > #####
    > # Ok so now we have a zgrep pushed into our
    > # @selected array. This is because the files are gunzipped.
    > # That give us all the calls made to the access number.
    > # Next we need to sort out the calls with zero session time
    > # by looking for /,0_/ that is a space after the ,0
    > # because typically failed calls have zero browsing time
    > ###
    >
    > #####
    > # now with the failed calls in @badcalls we filter and sort
    > ###
    >
    > foreach (@badcalls[0..$#badcalls]) {
    > my @errors = split /\s+/, $_;
    > $code = $errors[7];
    > $users = $errors[3];
    > $error_codes{$code}++;
    > $error_users{$users}++;
    > }


    foreach (@badcalls) {
    my ($code, $users) = (split /\s+/, $_)[3,7];
    $error_codes{ $code }++;
    $error_users{ $users }++;
    }

    > #####
    > # ok here is a difficult part to my script.
    > # Thanks Tassilo,
    > # this script would not have been possible without him.
    > ###
    >
    > my %errorsz;
    > foreach (@badcalls[0..$#badcalls]) {
    > my ($phonez, $err_codez) = (split)[3,7];
    > $errorsz{ $phonez }->{ $err_codez }++;
    > }


    There is quite some redundancy in the above two loops. You can squeeze
    that into one:

    my %errorsz;
    foreach (@badcalls) {
    my ($code, $users) = (split)[3,7];
    $error_codes{ $code }++;
    $error_users{ $users }++;
    $errorsz{ $code }->{ $users }++;
    }

    I am a little bit confused over the naming of the variables. In the
    first loop, the 4th and 7th field of each @badcall-item are named $code
    and $users respectively. In the second loop however, you call them
    $phonez and $err_codez.

    As for what Uri said on not using the explicit dereference operator '->':
    This is mostly a question of personal style. They aren't needed when
    working with nested references. However, it adds a little bit of
    consistency. Leaving them off somewhat implies that this:

    my $hash_ref = { key => value };
    print $hash_ref{ key };

    would also work, but it doesn't.

    > #####
    > # So the phone-numbers are the primary keys of the hash and the
    > # error-codes the keys of the nested hash.
    > # Next we perform a little math to get and average call sucess ratio
    > ###
    >
    > my $total = @selected;


    This is now obsolete as $total has already been computed in the
    while(<ZGREP> loop.

    [...]

    For the rest, Uri has already given good advise and there#s nothing to
    add.

    Tassilo
    --
    $_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
    pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
    $_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
     
    Tassilo v. Parseval, Feb 2, 2004
    #3
  4. Agrapha

    gnari Guest

    "Uri Guttman" <> wrote in message
    news:...
    > >>>>> "A" == Agrapha <> writes:

    >
    > what i like to do is make a usage sub like this:
    >
    > sub usage {
    > my ( $err_text ) = @_ ;
    > $err_text ||= '' ; # stop undef warnings
    > ...


    In this kind of cases I like to do:

    sub usage {
    my $err_text = shift || 'defaultvalue' ;
    ...

    of course that might not work as expected when '0' is a legal
    parameter, different from default, but is applicable in many
    situations.

    gnari
     
    gnari, Feb 2, 2004
    #4
  5. Agrapha

    Uri Guttman Guest

    >>>>> "g" == gnari <> writes:

    g> "Uri Guttman" <> wrote in message
    g> news:...
    >> >>>>> "A" == Agrapha <> writes:

    >>
    >> what i like to do is make a usage sub like this:
    >>
    >> sub usage {
    >> my ( $err_text ) = @_ ;
    >> $err_text ||= '' ; # stop undef warnings
    >> ...


    g> In this kind of cases I like to do:

    g> sub usage {
    g> my $err_text = shift || 'defaultvalue' ;
    g> ...

    i just stay away from shift @_ in most cases. i do use it where it does
    something important (like when i want to use a shorter @_ in more calls)
    but i stick with my () = @_ in almost all cases.

    and in this case you could do (but i wouldn't!):

    my( $err_text ) = ( @_, '' ) ;

    which works for empty @_ which is the only case i was concerned about
    (stopping the undef warning). now this still fails with:

    usage( undef ) ;

    but that coder deserves to get the warning!

    :)

    g> of course that might not work as expected when '0' is a legal
    g> parameter, different from default, but is applicable in many
    g> situations.

    but who uses '' or '0' as an error string? so that is fine for usage().

    anyhow, my main point is writing a usage sub is a good idea. and there
    is a module to make the pod and usage use common text. i saw some hacks
    for this once and i used them on a coupld of scripts on a project IIRC.

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
     
    Uri Guttman, Feb 2, 2004
    #5
  6. Agrapha

    Agrapha Guest

    "Tassilo v. Parseval" <> wrote in message news:<bvkubd$7ko$-Aachen.DE>...

    > > # info about it. We begin by making sure they enter one
    > > # on the command line or give them an example if they
    > > # fail to do so.
    > > ###
    > >
    > >
    > > if (!$ARGV[0] or !$ARGV[1] or !$ARGV[2]) {
    > > print "Syntax is \"./triag.pl accessnumber yyyymmdd mkt\"\n";
    > > print "example: ./triag.pl 2015551212 20040101 lon1
    > > die "Include an access number, date and market when starting
    > > Triag\n";
    > > }

    >
    > This is mostly a question of good style: usage-messages should go to
    > stderr really:
    >
    > if (@ARGV != 3) {
    > warn <<EOWARN;
    > Syntax is $0 accessnumber yyyymmdd mkt
    > example: $0 2015551212 20040101 lon1
    > Include an access number, date and market when starting Triag
    > EOWARN
    > exit(1);
    > }
    >
    > I use $0 because it is shorter and because it contains the exact path
    > which you used to call your script.


    $0 !? Very cool. I did not know what that variable was for even after
    I looked it up. I had to run it once. Ok so the @ARGV is the global
    array for the command line. Feed it 10 bits of info separated by ' '
    and I can test for the number of elements by (@ARGV <operand> 10).
    EOWARN is your Handle and << means to append into something called
    "warn". The exit(1) must print out the "warn" we just prepopulated.
    The terminator EOWARN seems to work only when I take all the
    whitespace from in front and place it on the hard left. Othewise Perl
    doesn't seem to find it. Would you describe "warn" ? What is that
    variable to stderr? How does perl know what is what here? no "", no
    echos (except an exit(1)) no ";" I'm missing a concept with "<<"
    somewhere.

    >
    > Some of these variables shouldn't have file-scope. As far as I can see,
    > $code and $users is only used in a for-loop later on. Better restrict
    > their scope to this loop.


    Agreed. This is bad form. I've corrected the issue (legacy bad code
    style from the start of this script)

    >
    > Maybe a pipe-open is more appropriate here:
    >
    > open ZGREP, "|", "zgrep $ARGV[0] ./radiuslog/..."
    > or die "Could not spawn zgrep: $!";
    > my (@badcalls, $total);
    > while (<ZGREP>) {
    > $total++;
    > push @badcalls, $_ if /,0 /;
    > }
    > close ZGREP;
    >
    > This needs less memory and gets rid of @selected altogether.


    Confession here. The files this script opens are quite large. and
    there are 30 files. The nifty shell script this came from was banned
    due to memory issues. Thats why we created a perl script. Just that
    helped dramatically. Still heavy on the machine but it takes 5 minutes
    and not 15-20 minutes to run. If I can save memory cpu cycles then I'm
    all for it.

    > There is quite some redundancy in the above two loops. You can squeeze
    > that into one:
    >
    > my %errorsz;
    > foreach (@badcalls) {
    > my ($code, $users) = (split)[3,7];
    > $error_codes{ $code }++;
    > $error_users{ $users }++;
    > $errorsz{ $code }->{ $users }++;
    > }


    The redundancy is simply due to my lack of understanding. The complex
    structure that is built here was quite beyond my skill level for the
    moment. The issue is a hash of hashes. Up to this point I simply count
    how many error codes or how many users. This is different. I have a
    structure like this:

    my %errors = (
    2005551212 => {
    101,4 => 10,
    33,185 => 33,
    },
    2005551213 => {
    63,43 => 15,
    65,42 => 26,
    },
    ...
    );

    the 10 digit numbers are phone numbers. The 101,4 is an example of an
    error code. The 10 is the count. I need to sort this by phone number.
    I.E.:

    print "\nNumber / Error / Totals\n";

    for my $n (sort { $a <=> $b } keys %errorsz) {
    # $errorsz{ $n } is now itself a reference to a hash
    my %codesz = %{ $errorsz{ $n } };

    # compute the total numbers of errors for this phone-number
    my $qty;
    for ( keys %codesz ) {
    $qty += $codesz{ $_ };
    }

    # display the stats for a given phone-number $n
    print "$n - ", join ("/", keys %codesz), " - $qty\n";
    }

    I separated these out for 3 different prints just so I could keep it
    straight in my mind.

    > For the rest, Uri has already given good advise and there#s nothing to
    > add.



    Uri made a good point that all prints should come from the same
    statement. just load up a variable and then print once. I'll look over
    that section in a minute.

    P.S. Could someone help me find the perldocs? Tassilo has mentioned
    them multiple times and I have failed to ask him how do I access those
    docs. Is it a web based application or something I can download?
     
    Agrapha, Feb 9, 2004
    #6
  7. Agrapha

    Uri Guttman Guest

    perldoc perl
    perldoc perlXXXX

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
     
    Uri Guttman, Feb 9, 2004
    #7
  8. Agrapha

    Ben Morrow Guest

    (Agrapha) wrote:
    > "Tassilo v. Parseval" <> wrote in message news:<bvkubd$7ko$-Aachen.DE>...
    > > This is mostly a question of good style: usage-messages should go to
    > > stderr really:
    > >
    > > if (@ARGV != 3) {
    > > warn <<EOWARN;
    > > Syntax is $0 accessnumber yyyymmdd mkt
    > > example: $0 2015551212 20040101 lon1
    > > Include an access number, date and market when starting Triag
    > > EOWARN
    > > exit(1);
    > > }
    > >
    > > I use $0 because it is shorter and because it contains the exact path
    > > which you used to call your script.

    >
    > $0 !? Very cool. I did not know what that variable was for even after
    > I looked it up. I had to run it once. Ok so the @ARGV is the global
    > array for the command line. Feed it 10 bits of info separated by ' '
    > and I can test for the number of elements by (@ARGV <operand> 10).
    > EOWARN is your Handle and << means to append into something called
    > "warn". The exit(1) must print out the "warn" we just prepopulated.
    > The terminator EOWARN seems to work only when I take all the
    > whitespace from in front and place it on the hard left. Othewise Perl
    > doesn't seem to find it. Would you describe "warn" ? What is that
    > variable to stderr? How does perl know what is what here? no "", no
    > echos (except an exit(1)) no ";" I'm missing a concept with "<<"
    > somewhere.


    Very much so :).

    <<HERE
    ....
    HERE

    is a here-doc, exactly as in shell. The statement

    warn <<EOWARN;
    blah
    EOWARN

    is equivalent to

    warn "blah\n";

    .. See perldoc perlop. warn is simply a function being called with one
    argument: see perldoc -f warn.

    Ben

    --
    Musica Dei donum optimi, trahit homines, trahit deos. |
    Musica truces molit animos, tristesque mentes erigit. |
    Musica vel ipsas arbores et horridas movet feras. |
     
    Ben Morrow, Feb 9, 2004
    #8
  9. Agrapha

    Agrapha Guest

    Uri Guttman <> wrote in message news:<>...
    > >>>>> "A" == Agrapha <> writes:

    >
    > A> #!/usr/bin/perl -w
    > A> use strict;
    >
    > good!
    >


    Good! :)

    >
    > A> if (!$ARGV[0] or !$ARGV[1] or !$ARGV[2]) {
    >
    > eww!
    > if ( @ARGV != 3 ) {
    >


    dern it :( I admit, the OR statements look a lot less appealing then
    your neat 7 syllable command. I really need to tighten up my
    statements.

    >
    >
    > A> print "Syntax is \"./triag.pl accessnumber yyyymmdd mkt\"\n";
    > A> print "example: ./triag.pl 2015551212 20040101 lon1
    > A> die "Include an access number, date and market when starting
    > A> Triag\n";
    >
    > why not a single die? and lose the \" stuff. here docs to the rescue!!
    >
    > die <<DIE ;
    > Syntax is "./triag.pl accessnumber yyyymmdd mkt"
    > example: ./triag.pl 2015551212 20040101 lon1
    > Include an access number, date and market when starting Triag
    > DIE
    >
    > what i like to do is make a usage sub like this:
    >
    > sub usage {
    >
    > my ( $err_text ) = @_ ;
    >
    > $err_text ||= '' ; # stop undef warnings
    >
    > die <<DIE ;
    > $err_text
    >
    > Syntax is "./triag.pl accessnumber yyyymmdd mkt"
    > example: ./triag.pl 2015551212 20040101 lon1
    > Include an access number, date and market when starting Triag
    > DIE
    >
    > }
    >
    > then you can call that in different places in the arg parsing stuff like
    > this:
    >
    > usage( "missing arguments ) if @ARGV < 3 ;
    > usage( "too many arguments ) if @ARGV > 3 ;
    >


    very slick..... Thats a very tight style. I don't fully understand the
    die <<DIE
    ....
    DIE
    idea yet. Where can I find some online docs about the
    initiator/terminator statements? The code above is so much easier to
    read.

    >
    > declare your variables in the tightest scope possible. these are file
    > scoped globals and are possibly too visible.


    agreed. The scope was set too wide at first simply because I honestly
    didn't know any better. I think it's fixed now keep the variables used
    in the for- loops isolated to the for-loops.

    > A> my @selected=`zgrep $ARGV[0]
    > A> ./radiuslog/$ARGV[1]/server.$ARGV[2].Detail.$ARGV[1].gz`;
    >
    > CPAN has modules to read .gz files so you can save on a fork.


    I'll search CPAN tonight. This is a serious bottleneck in the program.
    The files I'm gleening through here are 1-5meg big and there are 30
    files. Anything at all that will either let me slurp the files up or
    zgrep them or what. Thanks for the tip on where to find the info on
    zgreps.


    > here docs again! precalculate the rate value:
    > my $rate = 100 - (($fail / $total) * 100)
    >
    > print <<TOTALS ;
    > total_calls $total total_failures $fail access number $ARGV[0]
    > success rate for dates selected: $rate
    > TOTALS


    This must have been one of those gaps in my perl education. I think
    after the third of fourth time I'm getting the idea. The TOTALS acts
    like a " thats why there needs to be a terminator TOTALS as well. the
    semicolon after the first TOTALS has me confused though whay doesn't
    perl just end that statement after the ";" and error out the next 3
    lines?

    >
    > again, i would build it all up in a single string and then print it. it
    > is cleaner, faster and lets you control where you print it all. and
    > statement modifiers will make it look even better:
    >
    > my $text = '' ;
    >
    > $text .= <<TXT ;
    >
    > ErrorCodes / Totals
    > -------------------
    > TXT
    >
    > $text .= <<TXT foreach keys %error_codes ;
    > $_ / $error_codes{$_}
    > TXT
    >
    > $text .= <<TXT ;
    >
    > PhoneNumber / TotalFail
    > -----------------------
    > TXT
    >
    > $text .= <<TXT foreach keys %error_users ;
    > $_ / $error_users{$_}
    > TXT
    >
    > etc.
    >
    > notice how much easier it is to read what will be printed and also to
    > line it up? no extra " or \n or \t chars to provide visual noise. of
    > course some don't like here docs as much as i do but they are infidels! :)


    I really like the clarity there. I need to study the strings a bit
    better. The semicolon in the statement
    $text .= <<TXT foreach keys %error_users ;
    has me confused. the .= means to add to the variable I think. but the
    ; seems to me to end the statement.

    > A> for my $n (sort { $a <=> $b } keys %errorsz) {
    > A> # $errorsz{ $n } is now itself a reference to a hash
    > A> my %codesz = %{ $errorsz{ $n } };
    >
    > no need to copy that hash. see below
    >
    > A> # compute the total numbers of errors for this phone-number
    > A> my $qty;
    > A> for ( keys %codesz ) {
    >
    > for ( keys %{ $errorsz{ $n } } ) {
    >
    > A> $qty += $codesz{ $_ };
    > A> }
    >
    > and that can be shrunk with a statement modifier (i like those too!).
    >
    > $qty += %{ $errorsz{ $_ } } for keys %{ $errorsz{ $n } } ;
    >
    > but you are only accessing and summing the values of the subhash and you
    > don't need the keys for that. use the values function:
    >
    > $qty += $_ for values %{ $errorsz{ $n } } ;
    >
    > that is all you need there AFIACT from your code.
    >


    whew, ok Uri you just went over my head. This whole section is very
    difficult for me to understand. Where can I find some good information
    on how to understand a hash of hashes. ?
     
    Agrapha, Feb 9, 2004
    #9
  10. Agrapha

    Uri Guttman Guest

    >>>>> "A" == Agrapha <> writes:

    A> Uri Guttman <> wrote in message news:<>...

    A> if (!$ARGV[0] or !$ARGV[1] or !$ARGV[2]) {
    >>
    >> eww!
    >> if ( @ARGV != 3 ) {
    >>


    A> dern it :( I admit, the OR statements look a lot less appealing then
    A> your neat 7 syllable command. I really need to tighten up my
    A> statements.

    just don't tighten too much. good style is a balance between tight code
    and clarity.

    >> what i like to do is make a usage sub like this:
    >>
    >> sub usage {
    >>
    >> my ( $err_text ) = @_ ;
    >>
    >> $err_text ||= '' ; # stop undef warnings
    >>
    >> die <<DIE ;
    >> $err_text
    >>
    >> Syntax is "./triag.pl accessnumber yyyymmdd mkt"
    >> example: ./triag.pl 2015551212 20040101 lon1
    >> Include an access number, date and market when starting Triag
    >> DIE
    >>
    >> }


    A> very slick..... Thats a very tight style. I don't fully understand the
    A> die <<DIE
    A> ...
    A> DIE
    A> idea yet. Where can I find some online docs about the
    A> initiator/terminator statements? The code above is so much easier to
    A> read.

    that is called a here document. covered in perldata in the literals
    section. it is a very clean way to do multiline strings.

    >> here docs again! precalculate the rate value:
    >> my $rate = 100 - (($fail / $total) * 100)
    >>
    >> print <<TOTALS ;
    >> total_calls $total total_failures $fail access number $ARGV[0]
    >> success rate for dates selected: $rate
    >> TOTALS


    A> This must have been one of those gaps in my perl education. I think
    A> after the third of fourth time I'm getting the idea. The TOTALS acts
    A> like a " thats why there needs to be a terminator TOTALS as well. the
    A> semicolon after the first TOTALS has me confused though whay doesn't
    A> perl just end that statement after the ";" and error out the next 3
    A> lines?

    the print statement is what needs the ;. the here doc literal has 2
    parts. the <<TOKEN is where the string ends up in the expression. its
    content is all the following lines up to the matching close TOKEN.

    A> I really like the clarity there. I need to study the strings a bit
    A> better. The semicolon in the statement
    A> $text .= <<TXT foreach keys %error_users ;
    A> has me confused. the .= means to add to the variable I think. but the
    A> ; seems to me to end the statement.

    same as above. you always need a ; to end (actually separate) a
    statement. there are minor exceptions (last one in a block) but don't
    fall into that trap as it will bite you when you need to add more code
    later
    >>
    >> and that can be shrunk with a statement modifier (i like those too!).
    >>
    >> $qty += %{ $errorsz{ $_ } } for keys %{ $errorsz{ $n } } ;
    >>
    >> but you are only accessing and summing the values of the subhash and you
    >> don't need the keys for that. use the values function:
    >>
    >> $qty += $_ for values %{ $errorsz{ $n } } ;
    >>
    >> that is all you need there AFIACT from your code.
    >>


    A> whew, ok Uri you just went over my head. This whole section is very
    A> difficult for me to understand. Where can I find some good information
    A> on how to understand a hash of hashes. ?

    perldoc perlreftut
    perldoc perllol
    perldoc perldsc

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
     
    Uri Guttman, Feb 9, 2004
    #10
  11. Agrapha

    Joe Smith Guest

    Agrapha wrote:

    > P.S. Could someone help me find the perldocs? Tassilo has mentioned
    > them multiple times and I have failed to ask him how do I access those
    > docs. Is it a web based application or something I can download?


    Yes to both.
    -Joe

    unix% perldoc perl # Unix (Linux) command line

    C:\>perldoc perl # MS-DOS command window

    Start -> Programs -> ActiveState ActivePerl 5.8 -> Documentation

    http://www.perldoc.com/perl5.8.0/bin/perldoc.html

    --
    I love my TiVo - http://www.inwap.com/u/joe/tivo/
     
    Joe Smith, Feb 9, 2004
    #11
  12. Agrapha

    gnari Guest

    "Joe Smith" <> wrote in message
    news:rNGVb.209351$nt4.987931@attbi_s51...
    >
    > unix% perldoc perl # Unix (Linux) command line


    and if perldoc is not installed, usually
    man perl
    man perlfunc
    etc ...

    >
    > C:\>perldoc perl # MS-DOS command window
    >
    > Start -> Programs -> ActiveState ActivePerl 5.8 -> Documentation
    >
    > http://www.perldoc.com/perl5.8.0/bin/perldoc.html



    gnari
     
    gnari, Feb 9, 2004
    #12
  13. Agrapha

    Agrapha Guest

    "Tassilo v. Parseval" <> wrote in message news:<bvkubd$7ko$-Aachen.DE>...
    > > my @selected=`zgrep $ARGV[0]
    > > ./radiuslog/$ARGV[1]/server.$ARGV[2].Detail.$ARGV[1].gz`;
    > > my @badcalls= grep /,0 /, @selected;

    >
    > Maybe a pipe-open is more appropriate here:
    >
    > open ZGREP, "|", "zgrep $ARGV[0] ./radiuslog/..."
    > or die "Could not spawn zgrep: $!";
    > my (@badcalls, $total);
    > while (<ZGREP>) {
    > $total++;
    > push @badcalls, $_ if /,0 /;
    > }
    > close ZGREP;
    >
    > This needs less memory and gets rid of @selected altogether.
    >


    I think this section is giving me a hard time.
    open ZGREP, "|", "zgrep $ARGV[0] ./radiuslog/..." gives me an error.
    specifically "Unknown open() mode '|' at ./triag.pl line 54."
    if I change the line to
    open ZGREP, "| zgrep $ARGV[0] ./radiuslog/..." then it works but sends
    all the data to the screen and doesn't push into a variable or count
    $total
     
    Agrapha, Feb 9, 2004
    #13
  14. Agrapha

    Agrapha Guest

    Uri Guttman <> wrote in message news:<>...
    > >>>>> "A" == Agrapha <> writes:

    >
    > A> whew, ok Uri you just went over my head. This whole section is very
    > A> difficult for me to understand. Where can I find some good information
    > A> on how to understand a hash of hashes. ?
    >
    > perldoc perlreftut
    > perldoc perllol
    > perldoc perldsc
    >

    NICE! I read all the posts helping me use and get around perldoc.
    Thats a vital piece of info I didn't know. Thank you all.
     
    Agrapha, Feb 9, 2004
    #14
  15. Agrapha

    Agrapha Guest

    "Tassilo v. Parseval" <> wrote in message news:<bvkubd$7ko$-Aachen.DE>...
    > Also sprach Agrapha:
    > > my @selected=`zgrep $ARGV[0]
    > > ./radiuslog/$ARGV[1]/server.$ARGV[2].Detail.$ARGV[1].gz`;
    > > my @badcalls= grep /,0 /, @selected;

    >
    > Maybe a pipe-open is more appropriate here:
    >
    > open ZGREP, "|", "zgrep $ARGV[0] ./radiuslog/..."
    > or die "Could not spawn zgrep: $!";
    > my (@badcalls, $total);
    > while (<ZGREP>) {
    > $total++;
    > push @badcalls, $_ if /,0 /;
    > }
    > close ZGREP;
    >
    > This needs less memory and gets rid of @selected altogether.


    my epitaph will be "he persevered" the "perldoc -f open" helped me
    here. The pipe was giving me an error. I changed just 1 thing...no
    doubt left for me to dig a little as well. The open I think should
    look like -| opening inbound to us.

    open ZGREP, '-|', "zgrep $ARGV[0] ./radiuslog/..."

    and quick!!! I thought it was broke it went so quick. 8, 5meg files in
    about 3 to 5 seconds. thats brilliant. Let me clean this script up a
    bit and I'll post a better copy. I need to incorporate a few ideas yet
    from Uri and a couple others.
     
    Agrapha, Feb 9, 2004
    #15
  16. Agrapha

    Agrapha Guest

    "gnari" <> wrote in message news:<c07hhg$un4$>...
    > "Joe Smith" <> wrote in message
    > news:rNGVb.209351$nt4.987931@attbi_s51...
    > >
    > > unix% perldoc perl # Unix (Linux) command line
    > >
    > > C:\>perldoc perl # MS-DOS command window
    > >
    > > Start -> Programs -> ActiveState ActivePerl 5.8 -> Documentation
    > >
    > > http://www.perldoc.com/perl5.8.0/bin/perldoc.html

    >
    > and if perldoc is not installed, usually
    > man perl
    > man perlfunc
    > etc ...


    I assumed perldoc was installed as part of the perl language when it
    was originally untarred. If perl was installed on my local windows
    machine, I would have been truly lost. This is a great tool.
     
    Agrapha, Feb 9, 2004
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?TWlrZUppbmdKaW5n?=

    HELP Access DB, ASP.NET 2.0 and whole lot of pain

    =?Utf-8?B?TWlrZUppbmdKaW5n?=, Dec 6, 2005, in forum: ASP .Net
    Replies:
    5
    Views:
    483
    Patrice
    Dec 6, 2005
  2. E.U.
    Replies:
    3
    Views:
    774
    CBFalconer
    Nov 1, 2004
  3. Bil Kleb

    Help DRY a couple ifs

    Bil Kleb, Jan 28, 2007, in forum: Ruby
    Replies:
    3
    Views:
    113
    William James
    Jan 29, 2007
  4. Rex Gustavus Adolphus

    Ben and Tassilo: about calling subs with &

    Rex Gustavus Adolphus, Apr 23, 2004, in forum: Perl Misc
    Replies:
    6
    Views:
    92
    Tad McClellan
    Apr 24, 2004
  5. kj

    Question for Tassilo

    kj, Dec 3, 2004, in forum: Perl Misc
    Replies:
    2
    Views:
    93
    KKramsch
    Dec 3, 2004
Loading...

Share This Page