Repeatedly parsing a file to "clean" it.

[Graeme Stewart]

All,

I've done some lurking and found the answers to most of my questions,
but this one is more of a "best practice" issue, I've not been able to
find an answer for - my apologies if this is blantantly obvious and
I'm a looser newbie for even asking!

I've got a perl script that repeatedly opens and closes a couple of
files then excutes a while loop against those open files (example
below). Rather than repeatedly open and close the files, can I do this
in a better / more efficient way?

The objective is to take a jumbled text file and create individual
lines (records) that I can later manipulate.

*******************************************************

#!C:\perl\bin\perl.exe # Yes Windows! Don't hate me!

use strict;
open FILE, "./QM1.txt" or die "Can not open QM1.txt $!\n";
open COMMENTS, ">./temp/comments.txt" or die "Can not create
comments.txt $!\n";

while (<FILE>){

etc
etc

}

close FILE;
close COMMENTS;
open COMMENTS, "./temp/comments.txt" or die "Can not create
comments.txt $!\n";
open TEMPFILE, ">./temp/temp.txt" or die "Can not create temp.txt
$!\n";


while (<COMMENTS>){

etc
etc

}

close TEMPFILE;
close COMMENTS;
open COMMENTS, ">./temp/comments.txt" or die "Can not create
comments.txt $!\n";
open TEMPFILE, "./temp/temp.txt" or die "Can not create temp.txt
$!\n";


while (<TEMPFILE>){

etc
etc

}

close TEMPFILE;
close COMMENTS;
open COMMENTS, "./temp/comments.txt" or die "Can not create
comments.txt $!\n";
open TEMPFILE, ">./temp/temp.txt" or die "Can not create temp.txt
$!\n";

while (<COMMENTS>){

etc
etc

}

close TEMPFILE;
close COMMENTS;

*******************************************************

 

[Ala Qumsieh]

I've got a perl script that repeatedly opens and closes a couple of
files then excutes a while loop against those open files (example
below). Rather than repeatedly open and close the files, can I do this
in a better / more efficient way?

You can seek() to the beginning of your file instead of closing it and
opening it each time:

seek COMMENTS, 0, 0;

checkout 'perldoc -f seek'.

--Ala

 

[Wolfgang Hommel]

Hi Graeme,

I've got a perl script that repeatedly opens and closes a couple of
files then excutes a while loop against those open files (example
below). Rather than repeatedly open and close the files, can I do this
in a better / more efficient way?

Not sure what exactly you consider as "efficient", but it's rather
unlikely that the performance of your program suffers from opening and
closing files. Instead, regarding the code you posted, I'd recommend

a) using subfunctions instead of repeating code

b) "slurp mode" for reading each of those textfiles and splitting the
whole thing by line breaks instead of reading them in line by line.

If you really want to just avoid closing and re-opening a file, indeed
seek() is your friend.


Regards,
Wolfgang

 

[Tad McClellan]

Not sure what exactly you consider as "efficient", but it's rather
unlikely that the performance of your program suffers from opening and
closing files. Instead, regarding the code you posted, I'd recommend

a) using subfunctions instead of repeating code

That is "anti in-lining", I guess.

Seems that would move things toward a more INefficient way,
with regards to performance, what with all that pushing/popping
stack accessing and whatnot.

It _would_ move it toward a more efficient way with regards
to maintenance however.


As Wolfgang said, "efficient" doesn't help us help the OP.

Tell us what you want to "optimize".

Optimize performance? Optimize for labor? Optimize memory usage? ...



Speed at the expense of maintainability, or maintainability
at the expense of speed?



Which costs you more, CPU cycles or labor?

<wait for the answer>

Optimize that one.

 

[Graeme Stewart]

Thank you, thank you, thank you.

Both the utilization of subfunctions and the implementation of seek()
will make this code significantly more pleasing to the eye!

 

[Graeme Stewart]

I do appreciate that my original post was somewhat vauge, but I was
originally looking for a way to make the code a little easier to read,
and understand the way Perl manipulates data. I had a gut feeling that
it was incorrect best practice (if their is such a thing) to
continually open and close text files.

Personally I would always rather the system do this in memory.
Although as I'm new to Perl I might soon realize that text files are
the way to go. I was hoping the experience of the group could offer
some insight! Which everyone very kindly did!

P.S I was very careful in my original post to NOT use the word
"optimize". Efficient to me = the readability of the code!

Thanks,

 

[Tad McClellan]

[ Please do not top-post. It is seen as being rude.
Text rearranged into a more conventional order.
]

P.S I was very careful in my original post to NOT use the word
"optimize". Efficient to me = the readability of the code!

^^^^^
^^^^^

Then when you are talking to yourself, use "efficient".

When you are writing to other people, you should use universally
accepted meanings for words, we can't read your mind so we are
unlikey to assume your definition (unless you tell it to us).