Reset <> without having it fail once?

Discussion in 'Perl Misc' started by Hunter Johnson, Aug 11, 2004.

  1. I've got a filter written that needs to read into the target files
    (logs) for some additional information (finding a person's numeric ID
    by the name passed in) and then pull the records from the logs
    matching that additional information. This can include records
    earlier in the file than the record that let me match up the name to
    the ID.

    What I've done is close(ARGV) so that my first while(<>) fails, assign
    the (saved-off) parameters back to ARGV and then start a second
    while(<>) loop. If I just "last" out of the first loop and reassign
    @ARGV, it finished processing the file it was in the middle of.

    Is there a way to reset the <> without forcing the failure by closing
    it first?

    Hunter
    --
    http://www.hunterandlori.com
    Hunter Johnson, Aug 11, 2004
    #1
    1. Advertising

  2. On 11 Aug 2004, Hunter Johnson wrote:

    >What I've done is close(ARGV) so that my first while(<>) fails, assign
    >the (saved-off) parameters back to ARGV and then start a second
    >while(<>) loop. If I just "last" out of the first loop and reassign
    >@ARGV, it finished processing the file it was in the middle of.


    I believe that's because you haven't changed the filehandle ARGV, which is
    actually what is being read from.

    >Is there a way to reset the <> without forcing the failure by closing
    >it first?


    Can you show your code so I can get a better understanding of what it is
    you're doing now?

    --
    Jeff "japhy" Pinyan % How can we ever be the sold short or
    RPI Acacia Brother #734 % the cheated, we who for every service
    RPI Corporation Secretary % have long ago been overpaid?
    http://japhy.perlmonk.org/ %
    http://www.perlmonks.org/ % -- Meister Eckhart
    Jeff 'japhy' Pinyan, Aug 11, 2004
    #2
    1. Advertising

  3. Hunter Johnson

    Greg Bacon Guest

    In article <>,
    Hunter Johnson <> wrote:

    : I've got a filter written that needs to read into the target files
    : (logs) for some additional information (finding a person's numeric ID
    : by the name passed in) and then pull the records from the logs
    : matching that additional information. This can include records
    : earlier in the file than the record that let me match up the name to
    : the ID.

    Are these large logs? Can you store the records and then search for the
    interesting bits in a second pass?

    : What I've done is close(ARGV) so that my first while(<>) fails, assign
    : the (saved-off) parameters back to ARGV and then start a second
    : while(<>) loop. If I just "last" out of the first loop and reassign
    : @ARGV, it finished processing the file it was in the middle of.
    :
    : Is there a way to reset the <> without forcing the failure by closing
    : it first?

    It's a bigger-hammer approach, but here's another way:

    $ cat try
    #! /usr/local/bin/perl

    use warnings;
    use strict;

    my @copy = @ARGV;
    my $remember;

    eval {
    while (<>) {
    if (/f5/) {
    chomp;
    $remember = $_;
    close ARGV; # to get $. right
    die "OKAY";
    }
    }
    };

    die "$0: $@\n" unless $@ =~ /\bOKAY\b/;

    @ARGV = @copy;
    while (<>) {
    chomp;

    print "$0: $ARGV:$.: ", ($_ eq $remember ? "eq" : "ne"), "\n";
    }
    continue {
    close ARGV if eof;
    }

    $ for i in 1 2 3 4 5; do echo f$i >f$i; done

    $ ./try f?
    ./try: f1:1: ne
    ./try: f2:1: ne
    ./try: f3:1: ne
    ./try: f4:1: ne
    ./try: f5:1: eq

    Hope this helps,
    Greg
    --
    War is no longer merely a crime; it is an absurdity. It is no longer
    merely immoral and cruel; it is stupid. It is no longer merely murder
    on a large scale; it is suicide and voluntary ruin.
    -- Frederic Passy
    Greg Bacon, Aug 11, 2004
    #3
  4. Jeff 'japhy' Pinyan <> wrote in message news:<>...
    > On 11 Aug 2004, Hunter Johnson wrote:
    >
    > >What I've done is close(ARGV) so that my first while(<>) fails, assign
    > >the (saved-off) parameters back to ARGV and then start a second
    > >while(<>) loop. If I just "last" out of the first loop and reassign
    > >@ARGV, it finished processing the file it was in the middle of.

    >
    > I believe that's because you haven't changed the filehandle ARGV, which is
    > actually what is being read from.
    >
    > >Is there a way to reset the <> without forcing the failure by closing
    > >it first?

    >
    > Can you show your code so I can get a better understanding of what it is
    > you're doing now?


    Here's a test script that demonstrates the effect:

    #!/usr/bin/perl

    $term = shift;

    @saved = @ARGV;

    while (<>) {
    next unless /$term;ID=(.*)/;
    $ID = $1;
    last;
    }

    @ARGV = @saved;

    while (<>) {
    print if /\b$ID\b/;
    }

    __END__

    If this is the contents of files log1:
    This is some test data for id 12
    For id 14 too
    Username JILL;ID=14
    More 12 data
    More 14
    And just a bit more for 12

    and log2:
    This is some test data for id 12
    For id 14 too
    Username JOHN;ID=12
    More 12 data
    More 14
    And just a bit more for 12

    then the output from 'test.pl JOHN log1 log2' is:
    More 12 data
    And just a bit more for 12
    This is some test data for id 12
    More 12 data
    And just a bit more for 12
    This is some test data for id 12
    Username JOHN;ID=12
    More 12 data
    And just a bit more for 12


    The first two lines of the output shouldn't be there, and they do
    disappear if I use 'close (ARGV)' instead of 'last'. Which is no real
    hardship, but in the actual program, I end up with a test like this:

    $opt{'b'} ? last : close(ARGV);

    because I may not be done with the initial "recon" reading, and I was
    curious if there was Another Way To Do It. It seemed like something
    that could be done, but the I only found pointers to having <> fail
    once in the docs.

    Thanks,
    Hunter
    --
    Hunter Johnson, Aug 12, 2004
    #4
  5. (Greg Bacon) wrote in message news:<>...
    > In article <>,
    > Hunter Johnson <> wrote:
    >
    > : I've got a filter written that needs to read into the target files
    > : (logs) for some additional information (finding a person's numeric ID
    > : by the name passed in) and then pull the records from the logs
    > : matching that additional information. This can include records
    > : earlier in the file than the record that let me match up the name to
    > : the ID.
    >
    > Are these large logs? Can you store the records and then search for the
    > interesting bits in a second pass?


    Each log is about 20 megs, but a days' worth of logs is 320 megs, and
    there's an outside chance that the script could be run against all of
    the logs (currently capped at 2 gigs, but configurable). So I'd rather
    not.

    > : What I've done is close(ARGV) so that my first while(<>) fails, assign
    > : the (saved-off) parameters back to ARGV and then start a second
    > : while(<>) loop. If I just "last" out of the first loop and reassign
    > : @ARGV, it finished processing the file it was in the middle of.
    > :
    > : Is there a way to reset the <> without forcing the failure by closing
    > : it first?
    >
    > It's a bigger-hammer approach, but here's another way:


    [snip code]

    Yeah, that seems even less pleasing that mine. :)

    Thanks,
    Hunter
    Hunter Johnson, Aug 12, 2004
    #5
  6. [posted & mailed]

    On 12 Aug 2004, Hunter Johnson wrote:

    >#!/usr/bin/perl
    >
    >$term = shift;
    >
    >@saved = @ARGV;
    >
    >while (<>) {
    > next unless /$term;ID=(.*)/;
    > $ID = $1;
    > last;
    >}
    >
    >@ARGV = @saved;
    >
    >while (<>) {
    > print if /\b$ID\b/;
    >}


    I see your situation now.

    my @saved = @ARGV;
    while (<>) {
    if ($ID) { print if /\b$ID\b/ }
    elsif (/$term;ID=(.*)/) { $ID = $1; @ARGV = @saved; close ARGV; }
    }

    This code has not been tested, but it looks correct to me.

    That uses one while loop, but does still need to close ARGV. You have to
    close ARGV.

    --
    Jeff "japhy" Pinyan % How can we ever be the sold short or
    RPI Acacia Brother #734 % the cheated, we who for every service
    RPI Corporation Secretary % have long ago been overpaid?
    http://japhy.perlmonk.org/ %
    http://www.perlmonks.org/ % -- Meister Eckhart
    Jeff 'japhy' Pinyan, Aug 12, 2004
    #6
  7. Hunter Johnson

    Greg Bacon Guest

    In article <>,
    Hunter Johnson <> wrote:

    : [...]
    :
    : The first two lines of the output shouldn't be there, and they do
    : disappear if I use 'close (ARGV)' instead of 'last'. Which is no real
    : hardship, but in the actual program, I end up with a test like this:
    :
    : $opt{'b'} ? last : close(ARGV);
    :
    : because I may not be done with the initial "recon" reading, and I was
    : curious if there was Another Way To Do It. It seemed like something
    : that could be done, but the I only found pointers to having <> fail
    : once in the docs.

    Use exec!

    $ cat try
    #! /usr/local/bin/perl

    use warnings;
    use strict;

    sub usage { "Usage: $0 [<term> | --id=<id>] file..\n" }

    sub find_id {
    my $who = shift;

    my @saved = @ARGV;

    my $pat = qr/\Q$who\E;ID=(.*)/o;
    while (<>) {
    next unless /$pat/;

    no warnings 'exec';
    exec $0, "--id=$1", @saved;
    die "$0: exec: $!";
    }

    die "$0: no ID found!\n";
    }

    sub print_matching_records {
    my $id = shift;

    while (<>) {
    print if /\b$id\b/;
    }
    }

    ## main
    die usage unless @ARGV >= 2;

    my $arg = shift;

    if ($arg =~ /^--id=(.+)/) {
    print_matching_records $1;
    }
    else {
    find_id $arg;
    }

    $ ./try JOHN log1 log2
    This is some test data for id 12
    More 12 data
    And just a bit more for 12
    This is some test data for id 12
    Username JOHN;ID=12
    More 12 data
    And just a bit more for 12

    $ ./try JILL log1 log2
    For id 14 too
    Username JILL;ID=14
    More 14
    For id 14 too
    More 14

    $ ./try should-not-work log1 log2
    ./try: no ID found!

    Hope this helps,
    Greg
    --
    People say we make the market a god. It's more correct to say that we see
    in commerce the hand of God using the free actions and choices of billions
    of people to create orderliness where the pseudo-god of government only
    creates chaos and destruction. -- Lew Rockwell
    Greg Bacon, Aug 12, 2004
    #7
  8. (Greg Bacon) wrote in message news:<>...
    > In article <>,
    > Hunter Johnson <> wrote:
    >
    > : I was
    > : curious if there was Another Way To Do It. It seemed like something
    > : that could be done, but the I only found pointers to having <> fail
    > : once in the docs.
    >
    > Use exec!


    [code snipped]

    Say, that's slick, and no (explicit) close. Thanks!

    Hunter
    --
    http://www.hunterandlori.com
    Hunter Johnson, Aug 13, 2004
    #8
  9. Hunter Johnson

    Greg Bacon Guest

    In article <>,
    Greg Bacon <> wrote:

    : print if /\b$id\b/;

    That should have a /o:

    print if /\b$id\b/o;

    Greg
    --
    That's kinda scary powerful. I'm not sure I want to document that...
    ["Too late!" whispers Evil Damian.]
    -- Larry Wall, Apocalypse 6
    Greg Bacon, Aug 13, 2004
    #9
  10. Hunter Johnson

    Peter Scott Guest

    In article <>,
    (Greg Bacon) writes:
    >Use exec!


    Nice solution.

    > $ cat try
    > #! /usr/local/bin/perl
    >
    > use warnings;
    > use strict;
    >
    > sub usage { "Usage: $0 [<term> | --id=<id>] file..\n" }
    >
    > sub find_id {
    > my $who = shift;
    >
    > my @saved = @ARGV;
    >
    > my $pat = qr/\Q$who\E;ID=(.*)/o;
    > while (<>) {
    > next unless /$pat/;
    >
    > no warnings 'exec';


    The line above should not be necessary; the only statement following
    exec in the control flow is a die.

    > exec $0, "--id=$1", @saved;
    > die "$0: exec: $!";
    > }
    >
    > die "$0: no ID found!\n";
    > }

    [snip]

    --
    Peter Scott
    http://www.perldebugged.com/
    *** NEW *** http://www.perlmedic.com/
    Peter Scott, Aug 13, 2004
    #10
  11. Hunter Johnson

    Greg Bacon Guest

    In article <973Tc.93661$M95.70116@pd7tw1no>,
    Peter Scott <> wrote:

    : In article <>,
    : (Greg Bacon) writes:
    :
    : >Use exec!
    :
    : Nice solution.

    Thanks!

    : >[...]
    : > while (<>) {
    : > next unless /$pat/;
    : >
    : > no warnings 'exec';
    :
    : The line above should not be necessary; the only statement following
    : exec in the control flow is a die.

    The behavior surprised me too, but see below:

    $ diff -u try.orig try.nonesuch
    --- try.orig 2004-08-13 09:58:52.163202400 -0500
    +++ try.nonesuch 2004-08-13 10:03:47.860649900 -0500
    @@ -15,7 +15,7 @@
    next unless /$pat/;

    no warnings 'exec';
    - exec $0, "--id=$1", @saved;
    + exec "nonesuch", "--id=$1", @saved;
    die "$0: exec: $!";
    }


    $ ./try.nonesuch JOHN log?
    ../try.nonesuch: exec: No such file or directory at ./try.nonesuch
    line 19, <> line 9.

    $ diff -u try.orig try.nonesuch-warningson
    --- try.orig 2004-08-13 09:58:52.163202400 -0500
    +++ try.nonesuch-warningson 2004-08-13 10:01:37.613150700 -0500
    @@ -14,8 +14,8 @@
    while (<>) {
    next unless /$pat/;

    - no warnings 'exec';
    - exec $0, "--id=$1", @saved;
    + #no warnings 'exec';
    + exec "nonesuch", "--id=$1", @saved;
    die "$0: exec: $!";
    }

    $ ./try.nonesuch-warningson JOHN log?
    Can't exec "nonesuch": No such file or directory at
    ../try.nonesuch-warningson line 18, <> line 9.
    ../try.nonesuch-warningson: exec: No such file or directory at
    ../try.nonesuch-warningson line 19, <> line 9.

    The warnings pragma enables an autowarning on a failed exec, but I'm not
    sure where this is documented.

    Hope this helps,
    Greg
    --
    If a potential disaster can't be used to justify the expansion of the
    government, the media ignore it or else bury it on page 17.
    -- Gary North
    Greg Bacon, Aug 13, 2004
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Craig G
    Replies:
    0
    Views:
    294
    Craig G
    Feb 24, 2005
  2. Wenjie

    if (f() != FAIL) or if (FAIL != f())?

    Wenjie, Jul 28, 2003, in forum: C Programming
    Replies:
    3
    Views:
    440
    E. Robert Tisdale
    Jul 31, 2003
  3. Homer
    Replies:
    2
    Views:
    386
    Homer
    Aug 8, 2007
  4. Niv
    Replies:
    8
    Views:
    619
  5. Gancy
    Replies:
    4
    Views:
    176
    Rasto Levrinc
    Feb 3, 2005
Loading...

Share This Page