Open All files one by one

Discussion in 'Perl Misc' started by Billy, Dec 10, 2004.

  1. Billy

    Billy Guest

    All,

    I am looking for a way to open ALL files in a directory one by one in perl.

    I am able to open one file... read in the data and write it out to another
    file without any problems. But I have to tell the script the file I am
    opening. I would like the script to open all .txt files read out some data
    and append it to another file. The only part I am stuck on is the opening of
    the files without providing the filenames.

    I searched around and got tired of going in circles.

    Say I have a directory that has...

    bob.txt
    ted.txt
    sam.txt
    sue.txt
    index.html

    Is there a simple way to open all the .txt files?

    If this needs more info please let me know.

    Billy
     
    Billy, Dec 10, 2004
    #1
    1. Advertising

  2. Billy wrote:

    > I am looking for a way to open ALL files in a directory one by one in perl.


    Have a look at the opendir() and readdir() functions:

    perldoc -f opendir
    perldoc -f readdir

    sherm--

    --
    Cocoa programming in Perl: http://camelbones.sourceforge.net
    Hire me! My resume: http://www.dot-app.org
     
    Sherm Pendley, Dec 10, 2004
    #2
    1. Advertising

  3. Billy

    Keith Keller Guest

    On 2004-12-10, Billy <> wrote:

    > I am looking for a way to open ALL files in a directory one by one in perl.


    Are you sure?

    > I am able to open one file... read in the data and write it out to another
    > file without any problems. But I have to tell the script the file I am
    > opening. I would like the script to open all .txt files read out some data
    > and append it to another file. The only part I am stuck on is the opening of
    > the files without providing the filenames.


    I'd suggest getting a list of filenames, then opening a filehandle on
    each one at a time, doing what you need, and closing the filehandle.
    Presuming you already know the opening and closing part, you probably
    want glob to make a list of filenames; perldoc -f glob for help.

    If that's not useful advice, you might consider posting a short
    script which demonstrates what you're trying to do, and where you
    need help.

    --keith

    --
    -francisco.ca.us
    (try just my userid to email me)
    AOLSFAQ=http://wombat.san-francisco.ca.us/cgi-bin/fom
     
    Keith Keller, Dec 10, 2004
    #3
  4. Billy wrote:
    [...]
    > I am able to open one file... read in the data and write it out to
    > another file without any problems. [...]
    > Is there a simple way to open all the .txt files?


    See 'perldoc -f glob' and then simply loop through the list.

    jue
     
    Jürgen Exner, Dec 10, 2004
    #4
  5. On Fri, 10 Dec 2004 05:39:04 GMT, "Billy" <>
    wrote:

    >Say I have a directory that has...
    >
    >bob.txt
    >ted.txt
    >sam.txt
    >sue.txt
    >index.html
    >
    >Is there a simple way to open all the .txt files?


    In addition to the answers you already got, that you certainly
    *should* take into account, depending on what you *really* need to do
    with those files, you can also "trick" @ARGV to do the job for you (of
    course some care must be taken if you're already using it for
    something else):

    @ARGV=<*.txt>;
    while (<>) {
    # ...
    }


    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
     
    Michele Dondi, Dec 10, 2004
    #5
  6. Billy

    Tintin Guest

    "Billy" <> wrote in message
    news:YDaud.738441$8_6.539088@attbi_s04...
    >
    > All,
    >
    > I am looking for a way to open ALL files in a directory one by one in
    > perl.
    >
    > I am able to open one file... read in the data and write it out to another
    > file without any problems. But I have to tell the script the file I am
    > opening. I would like the script to open all .txt files read out some data
    > and append it to another file. The only part I am stuck on is the opening
    > of
    > the files without providing the filenames.
    >
    > I searched around and got tired of going in circles.
    >
    > Say I have a directory that has...
    >
    > bob.txt
    > ted.txt
    > sam.txt
    > sue.txt
    > index.html
    >
    > Is there a simple way to open all the .txt files?


    Do you mean one at a time? If so, then

    #!/usr/bin/perl
    use strict;

    foreach my $file (</path/to/dir/*>) {
    open FILE, $file or die "Can not open $file $!\n";

    while (<FILE>) {
    print; # or whatever
    }

    close FILE;
    }
     
    Tintin, Dec 10, 2004
    #6
  7. Billy

    foobar Guest

    #!/../..perl -w

    use strict;

    $my_txt_dir = ' / .. / .. /someplace';

    my $files_ref = ReadDir ($my_txt_dir);

    foreach my $file (@$files_ref){
    if ($file =~ /\.txt/){
    my $file_recs_ref = ReadFile ("$my_txt_dir/$file");
    foreach $line (@$file_recs_ref){
    ......
    }

    }

    sub ReadFile{
    my $file = shift @_;
    open (FH,"$file") or die "Could not open file $file";
    my @recs = <FH>;
    close(FH);
    return \@recs;
    }

    sub ReadDir{
    my $dir = shift @_;
    opendir DIR, $dir or die "could not open dir $dir";
    my @files = readdir DIR;
    closedir DIR;
    return \@files;
    }
     
    foobar, Dec 10, 2004
    #7
  8. "foobar" <> wrote in
    news::

    Please provide some context about what you are responding to. It looks to
    me like you are asking for a code review.

    > #!/../..perl -w


    As one of my friends would say, what is this crap?!

    > use strict;


    use warnings;

    is preferable to -w.

    > $my_txt_dir = ' / .. / .. /someplace';


    Post code that actually compiles.

    > my $files_ref = ReadDir ($my_txt_dir);


    Not wrong per se but what is the point of writing a sub whose name closely
    resembles the name of one Perl's functions, but behaves differently?

    Not to mention that you are not gaining anything by doing this. Instead,
    you run the risk of creating a bottleneck by potentially reading in a list
    of thousands of files when all you need is one file name at a time.

    > foreach my $file (@$files_ref){
    > if ($file =~ /\.txt/){


    Do you really want to match the file name if .txt appears anywhere in the
    name? That is, is 'my.doc.txt.pdf' really an acceptable filename?

    > my $file_recs_ref = ReadFile ("$my_txt_dir/$file");


    If you really want to slurp, maybe you should use File::Slurp.

    A better way might be this:

    use strict;
    use warnings;

    use File::Slurp;
    use File::Spec::Functions 'catfile';

    use constant DIR => 'd:/home';

    opendir my $DIR, DIR
    or die 'Cannot open directory ', DIR, ": $!";

    while(my $fn = readdir $DIR) {
    next unless -f $fn && $fn =~ /\.txt$/i;
    my $lines = read_file($fn, array_ref => 1);
    process_file($lines);
    }

    closedir $DIR
    or die 'Cannot close directory ', DIR, ": $!";

    sub process_file {
    # For lack of anything better to make up
    # right now
    print @{ $_[0] };
    }
    __END__
     
    A. Sinan Unur, Dec 11, 2004
    #8
  9. Billy

    yapp Guest

    My sincere apologies for the response. This is the first time I'm
    actually posting on a discussion group. I should have read the rules.
    I just realized all the flaws in my code and appreciate the critical
    comments.
     
    yapp, Dec 11, 2004
    #9
  10. Billy

    Billy Guest

    "Billy" <> wrote in message
    news:YDaud.738441$8_6.539088@attbi_s04...
    > All,
    > I am looking for a way to open ALL files in a directory one by one in

    perl.
    > I am able to open one file... read in the data and write it out to another
    > file without any problems. But I have to tell the script the file I am
    > opening. I would like the script to open all .txt files read out some data
    > and append it to another file. The only part I am stuck on is the opening

    of
    > the files without providing the filenames.
    > I searched around and got tired of going in circles.
    > Say I have a directory that has...
    > bob.txt
    > ted.txt
    > sam.txt
    > sue.txt
    > index.html
    > Is there a simple way to open all the .txt files?
    > If this needs more info please let me know.
    > Billy



    All how about:

    #!/usr/bin/perl
    use CGI::Carp qw(fatalsToBrowser);
    use strict;
    my @files = <*.txt>;
    while (<*.txt>) {
    open (HANDLE, $_) || die ("Can't open $_: $!");
    while (<HANDLE>) {
    print; # or something else
    }
    }
    close HANDLE;

    Anything wrong with this approach?

    Thank you,

    Billy
     
    Billy, Dec 11, 2004
    #10
  11. Billy

    Billy Guest


    > All how about:
    >
    > #!/usr/bin/perl
    > use CGI::Carp qw(fatalsToBrowser);
    > use strict;
    > my @files = <*.txt>;
    > while (<*.txt>) {
    > open (HANDLE, $_) || die ("Can't open $_: $!");
    > while (<HANDLE>) {
    > print; # or something else
    > }
    > }
    > close HANDLE;
    >
    > Anything wrong with this approach?
    >
    > Thank you,
    >
    > Billy
    >


    I found a problem right off... my @files = <*.txt>; only works if I have it
    in the same file directory.

    Billy
     
    Billy, Dec 11, 2004
    #11
  12. "A. Sinan Unur" <> wrote in
    news:Xns95BBC734C48B1asu1cornelledu@132.236.56.8:

    > use File::Spec::Functions 'catfile';


    ....

    > next unless -f $fn && $fn =~ /\.txt$/i;


    Spot the bug! In my defense, I did run the script before posting it, but I
    only ran it in D:/Home. The proper check above should have been:

    next unless -f catfile(DIR, $fn) && $fn =~ /\.txt$/i;

    Sorry.
    --
    A. Sinan Unur
    d
    (remove '.invalid' and reverse each component for email address)
     
    A. Sinan Unur, Dec 11, 2004
    #12
  13. Billy <> wrote:

    > #!/usr/bin/perl
    > use CGI::Carp qw(fatalsToBrowser);
    > use strict;
    > my @files = <*.txt>;
    > while (<*.txt>) {
    > open (HANDLE, $_) || die ("Can't open $_: $!");
    > while (<HANDLE>) {
    > print; # or something else
    > }
    > }
    > close HANDLE;
    >
    > Anything wrong with this approach?



    Yes, your indenting got all messed up.


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Dec 11, 2004
    #13
  14. Billy

    Tintin Guest

    "Billy" <> wrote in message
    news:tDsud.173736$V41.53872@attbi_s52...
    >
    > All how about:
    >
    > #!/usr/bin/perl
    > use CGI::Carp qw(fatalsToBrowser);
    > use strict;
    > my @files = <*.txt>;
    > while (<*.txt>) {
    > open (HANDLE, $_) || die ("Can't open $_: $!");
    > while (<HANDLE>) {
    > print; # or something else
    > }
    > }
    > close HANDLE;
    >
    > Anything wrong with this approach?


    You read all filenames into the @files array, but never use it.

    Instead of a while loop, I think a foreach loop looks cleaner, eg:

    foreach my $file (<*.txt>) {
    open FILE, $file or die "Can not open $file $!\n";
    ..
    }
     
    Tintin, Dec 11, 2004
    #14
  15. On 10 Dec 2004 12:42:59 -0800, "foobar" <> wrote:

    >#!/../..perl -w


    Ouch! Quite a strange shebang line!! In particular is your perl
    interpreter really called '..perl'?!? Of course this would work under
    Windows, and also under *NIX (I guess), *if* the script is called with

    perl programname

    >use strict;


    Worth mentioning that with recent enough perls it's better to

    use warnings; # instead of -w

    >$my_txt_dir = ' / .. / .. /someplace';


    Huh?!? Quite a strange path, isn't it? Also, since we're under strict,
    this won't compile at all.

    >my $files_ref = ReadDir ($my_txt_dir);


    Do you really think it is *convenient* (i) to create a dedicated sub
    to be used only *once* and that (ii) really adds only a very thin
    layer around Perl's readdir(), (iii) is called very similarly to it
    causing potential confusion, (iv) returns (a reference to an array
    containing) a full list of entries whereas these may be examined
    sequentially one at a time?

    >foreach my $file (@$files_ref){


    This may be to a large extent a matter of personal tastes, but what is
    the point of unnecessary referencing/dereferencing? Aren't we abusing
    this a little too much?!?

    >if ($file =~ /\.txt/){


    This may well be what you want, but it may be safer to /\.txt$/
    instead.

    >my $file_recs_ref = ReadFile ("$my_txt_dir/$file");
    >foreach $line (@$file_recs_ref){


    Ditto as above (wrt referencing/dereferencing).

    Oh, and this won't compile either, at least under strict. Good reason
    to at least perl -c your code.

    Oh, and while I'm here: quite a weird indenting style too! Please note
    that it doesn't make your code much readable...

    >sub ReadFile{
    >my $file = shift @_;


    shift; # shift @_; doesn't add to clarity!

    >open (FH,"$file") or die "Could not open file $file";


    Better include $! in the error message. It only requires a pair more
    keystrokes...

    BTW: it is convenient to use lexical FHs nowadays...

    >sub ReadDir{
    >my $dir = shift @_;


    Ditto as above!

    >opendir DIR, $dir or die "could not open dir $dir";


    Ditto as above!

    BTW: one can use lexical dirhandles too!

    All in all *if* I wanted to maintain the logic of your script (which I
    do *not* really want!), then at least I'd rewrite it like


    #!/usr/bin/perl

    use strict;
    use warnings;

    sub get_dir_entries;
    sub read_file;

    my $txt_dir='/path/to/dir';

    for (get_dir_entries $txt_dir) {
    next unless /\.txt$/;
    for (read_file "$txt_dir/$_") {
    # ...
    }
    }

    sub get_dir_entries {
    my $dir=shift;
    opendir my $dh, $dir or
    die "Could not open dir `$dir': $!\n";
    readdir $dh;
    }

    sub read_file{
    my $file=shift;
    open my $fh, '<', $file or
    die "Could not open file `$file': $!\n";
    <$fh>;
    }

    __END__


    HTH,
    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
     
    Michele Dondi, Dec 11, 2004
    #15
  16. On Sat, 11 Dec 2004 02:07:21 GMT, "Billy" <>
    wrote:

    >All how about:
    >
    >#!/usr/bin/perl
    >use CGI::Carp qw(fatalsToBrowser);


    Oh, it was a CGI thing then... ;-)

    >use strict;


    While you're there do a favour to yourself and

    use warnings; # too

    >my @files = <*.txt>;


    Why?!? Do you realize that you do *not* use @files at all,
    subsequently?

    >while (<*.txt>) {


    A faq entry warns about potential issues with glob() in scalar
    context. Also, since C<< while (<$fh>) >> is Perl's typical idiom for
    "iterate over filehandle", this could be confusing at best. I'd do

    for (<*.txt>) { # instead.

    > open (HANDLE, $_) || die ("Can't open $_: $!");


    Maybe it's just me but I'd use a lexical FH anyway:

    open my $fh, '<', $_ or die ...

    >Anything wrong with this approach?


    Apart the cmts above, I would say: no. Still, if the script is really
    that simple I'd use @ARGV's magic as of my previous post instead:


    #!/usr/bin/perl

    use strict;
    use warnings;
    # use CGI::Carp qw(fatalsToBrowser);

    @ARGV=<*.txt>;
    while (<>) {
    print; # or something else
    }

    __END__


    HTH,
    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
     
    Michele Dondi, Dec 11, 2004
    #16
  17. On 10 Dec 2004 18:02:13 -0800, "yapp" <> wrote:

    >My sincere apologies for the response. This is the first time I'm
    >actually posting on a discussion group. I should have read the rules.
    >I just realized all the flaws in my code and appreciate the critical
    >comments.


    Well, then another "rule" you should get familiar with is that you'd
    better quote some relevant context...


    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
     
    Michele Dondi, Dec 11, 2004
    #17
  18. On Sat, 11 Dec 2004 03:18:36 GMT, "Billy" <>
    wrote:

    >I found a problem right off... my @files = <*.txt>; only works if I have it
    >in the same file directory.


    Please note that nothing prevents you to from doing

    my @files = <path/to/dir/*.txt>;


    HTH,
    Michele
    --
    {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
    (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
    ..'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
    256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
     
    Michele Dondi, Dec 11, 2004
    #18
  19. Billy

    Billy Guest

    > Apart the cmts above, I would say: no. Still, if the script is really
    > that simple I'd use @ARGV's magic as of my previous post instead:
    > #!/usr/bin/perl
    >
    > use strict;
    > use warnings;
    > # use CGI::Carp qw(fatalsToBrowser);
    >
    > @ARGV=<*.txt>;
    > while (<>) {
    > print; # or something else
    > }
    >
    > __END__
    >
    > HTH,
    > Michele


    Thank you for all the replies... Since I don't know enough of perl yet I
    don't know the construct above. Plus I don't know what this will turn into,
    I prefer to keep it in a form I can learn from then refine later.

    Thank you again....

    Billy
     
    Billy, Dec 11, 2004
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. SJ

    Not open all solution files?

    SJ, Dec 9, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    433
  2. Alexandros

    Open all files in a directory

    Alexandros, Sep 17, 2003, in forum: C++
    Replies:
    1
    Views:
    2,267
    Attila Feher
    Sep 17, 2003
  3. rbt
    Replies:
    3
    Views:
    369
    Fredrik Lundh
    Feb 14, 2005
  4. PokerMan
    Replies:
    3
    Views:
    345
    Mark Rae
    Apr 9, 2007
  5. Tamara
    Replies:
    2
    Views:
    139
    Michele Dondi
    Apr 7, 2004
Loading...

Share This Page