Q: Find, print, and remove lines from logfile

M

Magnus

Hi!

Lets say I have a logfile which looks like:

Text bla bla...
Text....
foo1
More text bla bla...
Even more...
bar1
Text again...
foo2
and more..
bar2
bla bla..

I know (I think) how to print foo1 and bar1 and whats between:

while(<>){
if (/foo1/ .. /bar1/) {
print $_;
}
}

But what if I want to search the file all over again and print foo2
and bar2 and whats between and how do I remove the printed lines from
the logfile?

Any help is appreciated, even if it is just a hint.

Thanks in advance!
Magnus
 
T

Tore Aursand

Lets say I have a logfile which looks like:

Text bla bla...
Text....
foo1
More text bla bla...
Even more...
bar1
Text again...
foo2
and more..
bar2
bla bla..

I know (I think) how to print foo1 and bar1 and whats between:

while(<>){
if (/foo1/ .. /bar1/) {
print $_;
}
}

But what if I want to search the file all over again and print foo2
and bar2 and whats between [...]

Your requirements are quite vague; What's the _actual_ format of the
logfile(s) you're dealing with? Why do you need to read the file over
again (once should be enough, you know). Can't you just match on 'foo\d+'
and the corresponding 'bar'?

while ( <LOG> ) {
print if ( /foo(\d+)/ .. /bar$1/ );
}
[...] and how do I remove the printed lines from the logfile?

Write the data you _want_ to a temporary file, delete the old file, and
then rename the temporary file to the name of the old file.
 
B

Bruce Horrocks

Magnus said:
Hi!

Lets say I have a logfile which looks like:

Text bla bla...
Text....
foo1
More text bla bla...
Even more...
bar1
Text again...
foo2
and more..
bar2
bla bla..

I know (I think) how to print foo1 and bar1 and whats between:

while(<>){
if (/foo1/ .. /bar1/) {
print $_;
}
}

This won't work because having found foo1 you do not keep track of the
fact so it will only print each time it finds a line with foo1 in it,
not the lines following.

At the risk of bringing down the wrath of the group on me, this is
neater to do with the awk one-liner:

#! awk
/foo1/,/bar1/
But what if I want to search the file all over again and print foo2
and bar2 and whats between and how do I remove the printed lines from
the logfile?

I'm not quite sure what you mean. Why do you want to remove the lines
when you can test for foo2 and bar2 at the same time?

#! awk
/foo1/,/bar1/
/foo2/,/bar2/

will print out lines between foo1/bar1 and foo2/bar2. If you want
foo2/bar2 without foo1/bar1 then simply use the second line and delete
the first.

Regards,
 
B

Ben Morrow

Bruce Horrocks said:
This won't work because having found foo1 you do not keep track of the
fact so it will only print each time it finds a line with foo1 in it,
not the lines following.

At the risk of bringing down the wrath of the group on me, this is
neater to do with the awk one-liner:

#! awk
/foo1/,/bar1/

You need to read up on what the .. operator does in scalar context :).

Ben
 
T

Tore Aursand

This won't work because having found foo1 you do not keep track of the
fact so it will only print each time it finds a line with foo1 in it,
not the lines following.

Actually, it does. 'perldoc perlop' tells ut the following:

In scalar context, ".." returns a boolean value. The operator is
bistable, like a flip-flop, and emulates the line-range (comma) opera-
tor of sed, awk, and various editors. Each ".." operator maintains its
own boolean state. It is false as long as its left operand is false.
Once the left operand is true, the range operator stays true until the
right operand is true, AFTER which the range operator becomes false
again. It doesn't become false till the next time the range operator is
evaluated. It can test the right operand and become false on the same
evaluation it became true (as in awk), but it still returns true once.
 
M

Magnus

Your requirements are quite vague; What's the _actual_ format of the
logfile(s) you're dealing with? Why do you need to read the file over
again (once should be enough, you know). Can't you just match on 'foo\d+'
and the corresponding 'bar'?

The goal is to write a perlscript for analyzing exceptions and
stacktraces in a huge java logfile. I want to print out known
exceptions and there stacktraces and remove them from the logfile.
while ( <LOG> ) {
print if ( /foo(\d+)/ .. /bar$1/ );
}
[...] and how do I remove the printed lines from the logfile?

Write the data you _want_ to a temporary file, delete the old file, and
then rename the temporary file to the name of the old file.

Okey! I understand. Here is what I have done so far...I seems to work
even if it slow because the logfile is so big.

for-loop which set startstring and stopstring) {
open (LOGFILE, "./logfile.txt");
open (KNOWNEXCEPTIONS, ">./known.txt");
open (UNKNOWNEXCEPTIONS, ">./unknown.txt");
while (<LOGFILE>) {
if (/startstring/ .. /stopstring/) {
print KNOWNEXCEPTIONS "$_";
}
if (! (/startstring/ .. /stopstring/) {
print UNKNOWNEXCEPTIONS "$_";
}
close the files...
rename (unknown.txt, logfile.txt);
}
}

Thank you for your help!

Magnus
 
T

Tore Aursand

for-loop which set startstring and stopstring) {
open (LOGFILE, "./logfile.txt");
open (KNOWNEXCEPTIONS, ">./known.txt");
open (UNKNOWNEXCEPTIONS, ">./unknown.txt");

You _really_ want to know what goes wrong _when_ something goes wrong.
You also want to drop those unnecessary double quotes;

open(LOGFILE, './logfile.txt') or die "$!\n";
while (<LOGFILE>) {
if (/startstring/ .. /stopstring/) {
print KNOWNEXCEPTIONS "$_";
}
if (! (/startstring/ .. /stopstring/) {
print UNKNOWNEXCEPTIONS "$_";
}

Still no need to use the double quotes, and please consider not using the
'!' operator (for readability). That's my personal opionion. As far as I
can see, the above can be written as:

if ( /start/ .. /stop/ ) {
print KNOWNEXCEPTIONS $_;
}
else {
print UNKNOWNEXCEPTIONS $_;
}
close the files...

Do you _really_ want to close the files while processing them? I think
you should move the close()'ing to the same scope as where you open()'ed
the files.
rename (unknown.txt, logfile.txt);

Same goes with this one.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top