FAQ 5.29 How can I read in an entire file all at once?

Discussion in 'Perl Misc' started by PerlFAQ Server, Mar 16, 2011.

  1. This is an excerpt from the latest version perlfaq5.pod, which
    comes with the standard Perl distribution. These postings aim to
    reduce the number of repeated questions as well as allow the community
    to review and update the answers. The latest version of the complete
    perlfaq is at http://faq.perl.org .


    5.29: How can I read in an entire file all at once?

    Are you sure you want to read the entire file and store it in memory? If
    you mmap the file, you can virtually load the entire file into a string
    without actually storing it in memory:

    use File::Map qw(map_file);

    map_file my $string, $filename;

    Once mapped, you can treat $string as you would any other string. Since
    you don't necessarily have to load the data, mmap-ing can be very fast
    and may not increase your memory footprint.

    If you want to load the entire file, you can use the "File::Slurp"
    module to do it in one one simple and efficient step:

    use File::Slurp;

    my $all_of_it = read_file($filename); # entire file in scalar
    my @all_lines = read_file($filename); # one line per element

    The customary Perl approach for processing all the lines in a file is to
    do so one line at a time:

    open my $input, '<', $file or die "can't open $file: $!";
    while (<$input>) {
    # do something with $_
    close $input or die "can't close $file: $!";

    This is tremendously more efficient than reading the entire file into
    memory as an array of lines and then processing it one element at a
    time, which is often--if not almost always--the wrong approach. Whenever
    you see someone do this:

    my @lines = <INPUT>;

    You should think long and hard about why you need everything loaded at
    once. It's just not a scalable solution. You might also find it more fun
    to use the standard "Tie::File" module, or the "DB_File" module's
    $DB_RECNO bindings, which allow you to tie an array to a file so that
    accessing an element the array actually accesses the corresponding line
    in the file.

    You can read the entire filehandle contents into a scalar.

    my $var;
    local $/;
    open my $fh, '<', $file or die "can't open $file: $!";
    $var = <$fh>;

    That temporarily undefs your record separator, and will automatically
    close the file at block exit. If the file is already open, just use

    my $var = do { local $/; <$fh> };

    You can do that one better by using a localized @ARGV so you can
    eliminate the "open":

    my $var = do { local( @ARGV, $/ ) = $file; <> };

    For ordinary files you can also use the "read" function.

    read( $fh, $var, -s $fh );

    That third argument tests the byte size of the data on the "INPUT"
    filehandle and reads that many bytes into the buffer $var.


    The perlfaq-workers, a group of volunteers, maintain the perlfaq. They
    are not necessarily experts in every domain where Perl might show up,
    so please include as much information as possible and relevant in any
    corrections. The perlfaq-workers also don't have access to every
    operating system or platform, so please include relevant details for
    corrections to examples that do not work on particular platforms.
    Working code is greatly appreciated.

    If you'd like to help maintain the perlfaq, see the details in
    PerlFAQ Server, Mar 16, 2011
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Sjaakie Helderhorst

    Postback and process entire datagrid at once

    Sjaakie Helderhorst, Jun 3, 2004, in forum: ASP .Net
    Sjaakie Helderhorst
    Jun 3, 2004
  2. MRW

    Displaying Entire Page at once

    MRW, Dec 10, 2006, in forum: ASP .Net
    Gabriel Lozano-MorĂ¡n
    Dec 11, 2006
  3. Adam Funk
    Adam Funk
    Dec 21, 2007
  4. Chad
  5. PerlFAQ Server
    PerlFAQ Server
    Jan 26, 2011

Share This Page