Repeatedly parsing a file to "clean" it.

Discussion in 'Perl Misc' started by Graeme Stewart, Sep 10, 2004.

  1. All,

    I've done some lurking and found the answers to most of my questions,
    but this one is more of a "best practice" issue, I've not been able to
    find an answer for - my apologies if this is blantantly obvious and
    I'm a looser newbie for even asking!

    I've got a perl script that repeatedly opens and closes a couple of
    files then excutes a while loop against those open files (example
    below). Rather than repeatedly open and close the files, can I do this
    in a better / more efficient way?

    The objective is to take a jumbled text file and create individual
    lines (records) that I can later manipulate.

    *******************************************************

    #!C:\perl\bin\perl.exe # Yes Windows! Don't hate me!

    use strict;
    open FILE, "./QM1.txt" or die "Can not open QM1.txt $!\n";
    open COMMENTS, ">./temp/comments.txt" or die "Can not create
    comments.txt $!\n";

    while (<FILE>){

    etc
    etc

    }

    close FILE;
    close COMMENTS;
    open COMMENTS, "./temp/comments.txt" or die "Can not create
    comments.txt $!\n";
    open TEMPFILE, ">./temp/temp.txt" or die "Can not create temp.txt
    $!\n";


    while (<COMMENTS>){

    etc
    etc

    }

    close TEMPFILE;
    close COMMENTS;
    open COMMENTS, ">./temp/comments.txt" or die "Can not create
    comments.txt $!\n";
    open TEMPFILE, "./temp/temp.txt" or die "Can not create temp.txt
    $!\n";


    while (<TEMPFILE>){

    etc
    etc

    }

    close TEMPFILE;
    close COMMENTS;
    open COMMENTS, "./temp/comments.txt" or die "Can not create
    comments.txt $!\n";
    open TEMPFILE, ">./temp/temp.txt" or die "Can not create temp.txt
    $!\n";

    while (<COMMENTS>){

    etc
    etc

    }

    close TEMPFILE;
    close COMMENTS;

    *******************************************************
     
    Graeme Stewart, Sep 10, 2004
    #1
    1. Advertising

  2. Graeme Stewart

    Ala Qumsieh Guest

    Graeme Stewart wrote:

    > I've got a perl script that repeatedly opens and closes a couple of
    > files then excutes a while loop against those open files (example
    > below). Rather than repeatedly open and close the files, can I do this
    > in a better / more efficient way?


    You can seek() to the beginning of your file instead of closing it and
    opening it each time:

    seek COMMENTS, 0, 0;

    checkout 'perldoc -f seek'.

    --Ala
     
    Ala Qumsieh, Sep 10, 2004
    #2
    1. Advertising

  3. Hi Graeme,

    > I've got a perl script that repeatedly opens and closes a couple of
    > files then excutes a while loop against those open files (example
    > below). Rather than repeatedly open and close the files, can I do this
    > in a better / more efficient way?


    Not sure what exactly you consider as "efficient", but it's rather
    unlikely that the performance of your program suffers from opening and
    closing files. Instead, regarding the code you posted, I'd recommend

    a) using subfunctions instead of repeating code :)

    b) "slurp mode" for reading each of those textfiles and splitting the
    whole thing by line breaks instead of reading them in line by line.

    If you really want to just avoid closing and re-opening a file, indeed
    seek() is your friend.


    Regards,
    Wolfgang
     
    Wolfgang Hommel, Sep 10, 2004
    #3
  4. Wolfgang Hommel <> wrote:

    >> can I do this
    >> in a better / more efficient way?

    >
    > Not sure what exactly you consider as "efficient", but it's rather
    > unlikely that the performance of your program suffers from opening and
    > closing files. Instead, regarding the code you posted, I'd recommend
    >
    > a) using subfunctions instead of repeating code :)



    That is "anti in-lining", I guess.

    Seems that would move things toward a more INefficient way,
    with regards to performance, what with all that pushing/popping
    stack accessing and whatnot.

    It _would_ move it toward a more efficient way with regards
    to maintenance however.


    As Wolfgang said, "efficient" doesn't help us help the OP.

    Tell us what you want to "optimize".

    Optimize performance? Optimize for labor? Optimize memory usage? ...



    Speed at the expense of maintainability, or maintainability
    at the expense of speed?



    Which costs you more, CPU cycles or labor?

    <wait for the answer>

    Optimize that one.



    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Sep 10, 2004
    #4
  5. Thank you, thank you, thank you.

    Both the utilization of subfunctions and the implementation of seek()
    will make this code significantly more pleasing to the eye!


    Wolfgang Hommel <> wrote in message news:<chstc2$krg$06$-online.com>...
    > Hi Graeme,
    >
    > > I've got a perl script that repeatedly opens and closes a couple of
    > > files then excutes a while loop against those open files (example
    > > below). Rather than repeatedly open and close the files, can I do this
    > > in a better / more efficient way?

    >
    > Not sure what exactly you consider as "efficient", but it's rather
    > unlikely that the performance of your program suffers from opening and
    > closing files. Instead, regarding the code you posted, I'd recommend
    >
    > a) using subfunctions instead of repeating code :)
    >
    > b) "slurp mode" for reading each of those textfiles and splitting the
    > whole thing by line breaks instead of reading them in line by line.
    >
    > If you really want to just avoid closing and re-opening a file, indeed
    > seek() is your friend.
    >
    >
    > Regards,
    > Wolfgang
     
    Graeme Stewart, Sep 11, 2004
    #5
  6. I do appreciate that my original post was somewhat vauge, but I was
    originally looking for a way to make the code a little easier to read,
    and understand the way Perl manipulates data. I had a gut feeling that
    it was incorrect best practice (if their is such a thing) to
    continually open and close text files.

    Personally I would always rather the system do this in memory.
    Although as I'm new to Perl I might soon realize that text files are
    the way to go. I was hoping the experience of the group could offer
    some insight! Which everyone very kindly did!

    P.S I was very careful in my original post to NOT use the word
    "optimize". Efficient to me = the readability of the code!

    Thanks,


    Tad McClellan <> wrote in message news:<>...
    > Wolfgang Hommel <> wrote:
    >
    > >> can I do this
    > >> in a better / more efficient way?

    > >
    > > Not sure what exactly you consider as "efficient", but it's rather
    > > unlikely that the performance of your program suffers from opening and
    > > closing files. Instead, regarding the code you posted, I'd recommend
    > >
    > > a) using subfunctions instead of repeating code :)

    >
    >
    > That is "anti in-lining", I guess.
    >
    > Seems that would move things toward a more INefficient way,
    > with regards to performance, what with all that pushing/popping
    > stack accessing and whatnot.
    >
    > It _would_ move it toward a more efficient way with regards
    > to maintenance however.
    >
    >
    > As Wolfgang said, "efficient" doesn't help us help the OP.
    >
    > Tell us what you want to "optimize".
    >
    > Optimize performance? Optimize for labor? Optimize memory usage? ...
    >
    >
    >
    > Speed at the expense of maintainability, or maintainability
    > at the expense of speed?
    >
    >
    >
    > Which costs you more, CPU cycles or labor?
    >
    > <wait for the answer>
    >
    > Optimize that one.
     
    Graeme Stewart, Sep 13, 2004
    #6
  7. [ Please do not top-post. It is seen as being rude.
    Text rearranged into a more conventional order.
    ]


    Graeme Stewart <> wrote:
    > Tad McClellan <> wrote in message news:<>...
    >> Wolfgang Hommel <> wrote:
    >>
    >> >> can I do this
    >> >> in a better / more efficient way?
    >> >
    >> > Not sure what exactly you consider as "efficient",



    >> As Wolfgang said, "efficient" doesn't help us help the OP.
    >>
    >> Tell us what you want to "optimize".
    >>
    >> Optimize performance? Optimize for labor? Optimize memory usage? ...



    > P.S I was very careful in my original post to NOT use the word
    > "optimize". Efficient to me = the readability of the code!

    ^^^^^
    ^^^^^

    Then when you are talking to yourself, use "efficient".

    When you are writing to other people, you should use universally
    accepted meanings for words, we can't read your mind so we are
    unlikey to assume your definition (unless you tell it to us).


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Sep 18, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. SteveLB
    Replies:
    0
    Views:
    330
    SteveLB
    Aug 8, 2003
  2. Jarrod Hermer
    Replies:
    2
    Views:
    2,207
    kahlawat
    Jul 13, 2007
  3. Nick Keighley

    repeatedly trying to open a file

    Nick Keighley, Nov 22, 2006, in forum: C++
    Replies:
    1
    Views:
    316
    Pete Becker
    Nov 22, 2006
  4. Replies:
    8
    Views:
    515
  5. Jason Lillywhite
    Replies:
    8
    Views:
    149
    Fabian Streitel
    Sep 18, 2009
Loading...

Share This Page