Readline using foreach and while

Discussion in 'Perl Misc' started by Saurabh Jain, Mar 25, 2008.

  1. Saurabh Jain

    Saurabh Jain Guest

    Hi,
    Is there any difference in reading a file using a while or a
    foreach in perl?

    If I do :
    foreach(<filehandle>) {
    my $local = <filehandle>; # I assumed I will increment the file
    descriptor here
    print " local $local\n";
    }

    But if I do :
    while(<filehandle>) {
    my $local = <filehandle>; # I assumed I will increment the file
    descriptor here
    print " local $local\n";
    }
    It works fine....
    Is there something wrong or some difference in the two operations? Or
    am I missing something?

    Thanks and Regards,
    Saurabh


    Small example to replicate the issue
    my file name is test.pl

    #!/usr/bin/perl

    open (handle,"test.pl")||die "\n $0 Could not open $! \n";

    my $line = <handle>;#read a line till \n or eof
    print " line $line";
    #foreach(<handle>){ # Not as expected
    while(<handle>){ # Works as expected
    $line =<handle>;#read a line till \n or eof

    print " in side $line";
    $line =<handle>;#read a line till \n or eof
    print " in side $line";
    $line =<handle>;#read a line till \n or eof
    }
    close handle;
     
    Saurabh Jain, Mar 25, 2008
    #1
    1. Advertising

  2. Saurabh Jain

    Ben Bullock Guest

    On Mar 25, 4:25 pm, Saurabh Jain <> wrote:
    > Hi,
    > Is there any difference in reading a file using a while or a
    > foreach in perl?


    The foreach version seems to first read the whole of the file into an
    array, and then go through it line by line:

    #!/usr/bin/perl
    #use warnings;
    use strict;
    open (handle,"testangleop.pl") or die "$0 Could not open $!";

    my $line = <handle>; #read a line till \n or eof
    print "0 line $line";
    foreach (<handle>) { # Not as expected
    #while(<handle>){ # Works as expected
    print $_;
    $line =<handle>; #read a line till \n or eof
    print "1 in side $line";
    $line =<handle>; #read a line till \n or eof
    print "2 in side $line";
    $line =<handle>; #read a line till \n or eof
    print "3 in side $line";
    }

    The while seems to increment through the loop.

    See also

    http://www.unix.org.ua/orelly/perl/prog3/ch02_11.htm
     
    Ben Bullock, Mar 25, 2008
    #2
    1. Advertising

  3. Saurabh Jain wrote:
    > Is there any difference in reading a file using a while or a
    > foreach in perl?


    Yes.

    foreach (<FILEHANDLE>)

    reads the whole file at once, and creates a list in memory of all the
    lines, so that method is inefficient and not recommended in most cases.

    while (<FILEHANDLE>)

    reads one line at a time.

    Please study "perldoc perlsyn" for more comprehensive descriptions.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
     
    Gunnar Hjalmarsson, Mar 25, 2008
    #3
  4. Ben Bullock wrote:
    > On Mar 25, 4:25 pm, Saurabh Jain <> wrote:
    >> Hi,
    >> Is there any difference in reading a file using a while or a
    >> foreach in perl?

    >
    > The foreach version seems to first read the whole of the file into an
    > array, and then go through it line by line:


    perldoc -q "What is the difference between a list and an array"


    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
     
    John W. Krahn, Mar 25, 2008
    #4
  5. Saurabh Jain <> wrote:
    > Is there any difference in reading a file using a while or a
    >foreach in perl?


    Yes. Reading a file line by line works only with while. foreach reads the
    whole file at once.

    >If I do :
    > foreach(<filehandle>) {
    > my $local = <filehandle>; # I assumed I will increment the file


    You just tried to read from a file that is at EOF already.

    >descriptor here
    > print " local $local\n";
    >}
    >
    >But if I do :
    >while(<filehandle>) {
    > my $local = <filehandle>; # I assumed I will increment the file


    You are alternating between reading one line into $_ (in the while
    condition) and one line into $local. Is this what you meant to do?

    >descriptor here
    > print " local $local\n";
    >}


    >
    >Small example to replicate the issue
    >my file name is test.pl
    >
    >#!/usr/bin/perl
    >
    >open (handle,"test.pl")||die "\n $0 Could not open $! \n";
    >
    >my $line = <handle>;#read a line till \n or eof
    >print " line $line";
    >#foreach(<handle>){ # Not as expected
    >while(<handle>){ # Works as expected
    >$line =<handle>;#read a line till \n or eof
    >
    >print " in side $line";
    >$line =<handle>;#read a line till \n or eof
    >print " in side $line";
    >$line =<handle>;#read a line till \n or eof


    And here you are reading one line into $_ (in the while condition) and then
    successively three lines into $line. This may make sense if you know that a
    data set has a fixed format of always 4 lines. But in 99% of all cases it's
    a bug.
    As for the foreach version: it already slurps the whole file into a list,
    therefore there is nothing left that could be read into any of the $line.

    jue
     
    Jürgen Exner, Mar 25, 2008
    #5
  6. Saurabh Jain

    Ben Bullock Guest

    On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:

    > Ben Bullock wrote:


    >> The foreach version seems to first read the whole of the file into an
    >> array, and then go through it line by line:

    >
    > perldoc -q "What is the difference between a list and an array"


    Found in /usr/local/lib/perl5/5.10.0/pod/perlfaq4.pod
    What is the difference between a list and an array?

    An array has a changeable length. A list does not.

    If I had written "the foreach version reads the whole of the file into a
    list", I would have contradicted this FAQ entry, which says I can't read
    things into a list, because reading things into a list would change the
    list's length, and "a list does not" have a changeable length.
     
    Ben Bullock, Mar 25, 2008
    #6
  7. Saurabh Jain

    Ben Morrow Guest

    Quoth Ben Bullock <>:
    > On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
    > > Ben Bullock wrote:

    >
    > >> The foreach version seems to first read the whole of the file into an
    > >> array, and then go through it line by line:

    > >
    > > perldoc -q "What is the difference between a list and an array"

    >
    > Found in /usr/local/lib/perl5/5.10.0/pod/perlfaq4.pod
    > What is the difference between a list and an array?
    >
    > An array has a changeable length. A list does not.
    >
    > If I had written "the foreach version reads the whole of the file into a
    > list", I would have contradicted this FAQ entry, which says I can't read
    > things into a list, because reading things into a list would change the
    > list's length, and "a list does not" have a changeable length.


    No, you're misunderstanding. A list is immutable *after it has been
    created*. Obviously you can create lists with any contents, otherwise
    you would be limited to using only lists compiled into perl. foreach
    accepts a list as argument and iterates over it; <> in list context
    (which is the real problem here) reads the entire file, splits it on
    newline (or rather $/), and returns a (newly created) list with the
    results. You can't modify the list after that: for an example where you
    can, see Tie::File, which reads a file into an *array* instead.

    Ben
     
    Ben Morrow, Mar 25, 2008
    #7
  8. Saurabh Jain

    Ben Morrow Guest

    Quoth Frank Seitz <>:
    > Ben Morrow wrote:
    > > Quoth Ben Bullock <>:
    > >>On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
    > >>>Ben Bullock wrote:
    > >>>>
    > >>>>The foreach version seems to first read the whole of the file into an
    > >>>>array, and then go through it line by line:
    > >>>
    > >>>perldoc -q "What is the difference between a list and an array"

    >
    > Hm. Why is this distinction relevant here?


    I'm not at all sure it is, except in the 'variable vs. value' sense.

    > > A list is immutable *after it has been
    > > created*. Obviously you can create lists with any contents, otherwise
    > > you would be limited to using only lists compiled into perl. foreach
    > > accepts a list as argument and iterates over it;

    >
    > use strict;
    > use warnings;
    >
    > my @a = qw/a b c/;
    > for my $v (@a) {
    > push @a,'d' if $v eq 'c';
    > print "$v\n";
    > }


    Good point. for is a little weird in this respect...

    Ben
     
    Ben Morrow, Mar 25, 2008
    #8
  9. On Mar 25, 1:14 pm, Frank Seitz <> wrote:
    > Ben Morrow wrote:
    > > Quoth Ben Bullock <>:
    > >>On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
    > >>>Ben Bullock wrote:

    >
    > >>>>The foreach version seems to first read the whole of the file into an
    > >>>>array, and then go through it line by line:

    >
    > >>>perldoc -q "What is the difference between a list and an array"

    >
    > Hm. Why is this distinction relevant here?
    >
    > > A list is immutable *after it has been
    > > created*. Obviously you can create lists with any contents, otherwise
    > > you would be limited to using only lists compiled into perl. foreach
    > > accepts a list as argument and iterates over it;

    >
    > use strict;
    > use warnings;
    >
    > my @a = qw/a b c/;
    > for my $v (@a) {
    >     push @a,'d' if $v eq 'c';
    >     print "$v\n";}
    >
    > __END__
    > a
    > b
    > c
    > d
    >


    http://groups.google.com/group/comp.lang.perl.misc/msg/9f2ebca559578bcf
     
    nolo contendere, Mar 25, 2008
    #9
  10. On Mar 25, 1:50 pm, nolo contendere <> wrote:
    > On Mar 25, 1:14 pm, Frank Seitz <> wrote:
    >
    >
    >
    > > Ben Morrow wrote:
    > > > Quoth Ben Bullock <>:
    > > >>On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
    > > >>>Ben Bullock wrote:

    >
    > > >>>>The foreach version seems to first read the whole of the file into an
    > > >>>>array, and then go through it line by line:

    >
    > > >>>perldoc -q "What is the difference between a list and an array"

    >
    > > Hm. Why is this distinction relevant here?

    >
    > > > A list is immutable *after it has been
    > > > created*. Obviously you can create lists with any contents, otherwise
    > > > you would be limited to using only lists compiled into perl. foreach
    > > > accepts a list as argument and iterates over it;

    >
    > > use strict;
    > > use warnings;

    >
    > > my @a = qw/a b c/;
    > > for my $v (@a) {
    > >     push @a,'d' if $v eq 'c';
    > >     print "$v\n";}

    >
    > > __END__
    > > a
    > > b
    > > c
    > > d

    >
    > http://groups.google.com/group/comp.lang.perl.misc/msg/9f2ebca559578bcf


    also, see 'Perl: modifying an array in a loop'

    http://blog.plover.com/prog/perl/undefined.html#3
     
    nolo contendere, Mar 25, 2008
    #10
  11. Frank Seitz wrote:
    > Ben Morrow wrote:
    >> Quoth Ben Bullock <>:
    >>> On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
    >>>> Ben Bullock wrote:
    >>>>> The foreach version seems to first read the whole of the file into an
    >>>>> array, and then go through it line by line:
    >>>> perldoc -q "What is the difference between a list and an array"

    >
    > Hm. Why is this distinction relevant here?
    >
    >> A list is immutable *after it has been
    >> created*. Obviously you can create lists with any contents, otherwise
    >> you would be limited to using only lists compiled into perl. foreach
    >> accepts a list as argument and iterates over it;

    >
    > use strict;
    > use warnings;
    >
    > my @a = qw/a b c/;
    > for my $v (@a) {


    We were talking about using a *list* in a foreach loop so that should be:

    for my $v ( qw/a b c/ ) {

    > push @a,'d' if $v eq 'c';


    You are modifying an *array*, not a list.

    > print "$v\n";
    > }
    > __END__
    > a
    > b
    > c
    > d



    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
     
    John W. Krahn, Mar 25, 2008
    #11
  12. Saurabh Jain

    szr Guest

    John W. Krahn wrote:
    > Ben Bullock wrote:
    >> On Mar 25, 4:25 pm, Saurabh Jain <> wrote:
    >>> Hi,
    >>> Is there any difference in reading a file using a while or a
    >>> foreach in perl?

    >>
    >> The foreach version seems to first read the whole of the file into an
    >> array, and then go through it line by line:

    >
    > perldoc -q "What is the difference between a list and an array"


    Array is the variable type, List is the type of valve an Array
    takes/holds.

    > Perl isn't a toolbox, but a small machine shop where you
    > can special-order certain sorts of tools at low cost and
    > in short order. -- Larry Wall


    Amen. I always loved this quote.

    --
    szr
     
    szr, Mar 25, 2008
    #12
  13. Ben Bullock wrote:
    > On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
    >
    >> Ben Bullock wrote:

    >
    >>> The foreach version seems to first read the whole of the file into an
    >>> array, and then go through it line by line:

    >> perldoc -q "What is the difference between a list and an array"

    >
    > Found in /usr/local/lib/perl5/5.10.0/pod/perlfaq4.pod
    > What is the difference between a list and an array?
    >
    > An array has a changeable length. A list does not.
    >
    > If I had written "the foreach version reads the whole of the file into a
    > list", I would have contradicted this FAQ entry, which says I can't read
    > things into a list, because reading things into a list would change the
    > list's length, and "a list does not" have a changeable length.


    The FAQ entry does not contain the phrase "read things into a list".

    print reverse grep /a/, readdir DIR;

    In the above example, where does grep() get its list from? Is the list
    that grep() returns the same length as the list it gets? Where does
    reverse() get its list from, and is it the same list that grep() gets?
    Where does print() get its list from, and is it the same list that
    reverse() gets?


    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
     
    John W. Krahn, Mar 25, 2008
    #13
  14. On Mar 25, 1:57 pm, "szr" <> wrote:
    > John W. Krahn wrote:
    > > Ben Bullock wrote:
    > >> On Mar 25, 4:25 pm, Saurabh Jain <> wrote:
    > >>> Hi,
    > >>> Is there any difference in reading a file using a while or a
    > >>> foreach in perl?

    >
    > >> The foreach version seems to first read the whole of the file into an
    > >> array, and then go through it line by line:

    >
    > > perldoc -q "What is the difference between a list and an array"

    >
    > Array is the variable type, List is the type of valve an Array
    > takes/holds.
    >


    Actually, an array just holds scalars... or references which is a
    special type of scalar.

    An array can be populated from a list though.

    --
    Charles DeRykus
     
    comp.llang.perl.moderated, Mar 25, 2008
    #14
  15. Saurabh Jain

    Ben Bullock Guest

    On Mar 26, 1:37 am, Ben Morrow <> wrote:
    > Quoth Ben Bullock <>:
    >
    >
    >
    > > On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
    > > > Ben Bullock wrote:

    >
    > > >> The foreach version seems to first read the whole of the file into an
    > > >> array, and then go through it line by line:

    >
    > > > perldoc -q "What is the difference between a list and an array"

    >
    > > Found in /usr/local/lib/perl5/5.10.0/pod/perlfaq4.pod
    > > What is the difference between a list and an array?

    >
    > > An array has a changeable length. A list does not.

    >
    > > If I had written "the foreach version reads the whole of the file into a
    > > list", I would have contradicted this FAQ entry, which says I can't read
    > > things into a list, because reading things into a list would change the
    > > list's length, and "a list does not" have a changeable length.

    >
    > No, you're misunderstanding. A list is immutable *after it has been
    > created*. Obviously you can create lists with any contents, otherwise
    > you would be limited to using only lists compiled into perl. foreach
    > accepts a list as argument and iterates over it; <> in list context
    > (which is the real problem here) reads the entire file, splits it on
    > newline (or rather $/), and returns a (newly created) list with the
    > results. You can't modify the list after that: for an example where you
    > can, see Tie::File, which reads a file into an *array* instead.
    >
    > Ben
     
    Ben Bullock, Mar 25, 2008
    #15
  16. Saurabh Jain

    Ben Bullock Guest

    On Mar 26, 5:56 am, "John W. Krahn" <> wrote:

    > We were talking about using a *list* in a foreach loop


    You might have been, but I wasn't. I was talking about Perl reading in
    data from a file.
     
    Ben Bullock, Mar 25, 2008
    #16
  17. Saurabh Jain

    Ben Bullock Guest

    On Mar 26, 1:37 am, Ben Morrow <> wrote:

    > No, you're misunderstanding. A list is immutable *after it has been
    > created*. Obviously you can create lists with any contents, otherwise
    > you would be limited to using only lists compiled into perl.


    If we can create lists, then there must be a data structures inside
    Perl which represents those created lists. Let's call it the "newly-
    created list" data structure. There is also a data structure for
    arrays, of course.

    Can you tell me the difference between the "newly-created lists" data
    structure and the array data structure? Is there an example of Perl
    code where we can see how the "newly-created lists" differ from
    arrays? Or are these actually the same thing in practice?

    > foreach
    > accepts a list as argument and iterates over it; <> in list context
    > (which is the real problem here) reads the entire file, splits it on
    > newline (or rather $/), and returns a (newly created) list with the
    > results. You can't modify the list after that


    But I wasn't talking about modifying the list after that.

    : for an example where you
    > can, see Tie::File, which reads a file into an *array* instead.


    OK:

    http://perldoc.perl.org/Tie/File.html

    But it says very shortly into the documentation "The file is not
    loaded into memory" and "Changes to the array are reflected in the
    file immediately". So your statement about "reads a file into an
    array" is wrong. It is not reading the file into an array. It is
    making a file appear to be an array. To read a file into an array use:

    my @ary = <handle>;

    Then one can loop over the array using

    foreach (@ary)
     
    Ben Bullock, Mar 25, 2008
    #17
  18. Ben Bullock wrote:
    > If we can create lists, then there must be a data structures inside
    > Perl which represents those created lists. Let's call it the "newly-
    > created list" data structure. There is also a data structure for
    > arrays, of course.
    >
    > Can you tell me the difference between the "newly-created lists" data
    > structure and the array data structure? Is there an example of Perl
    > code where we can see how the "newly-created lists" differ from
    > arrays? Or are these actually the same thing in practice?


    C:\home>type test.pl
    use warnings;
    $var1 = qw/A B C/;
    print "$var1\n";
    @array = qw/A B C/;
    $var2 = @array;
    print "$var2\n";

    C:\home>perl test.pl
    Useless use of a constant in void context at test.pl line 2.
    Useless use of a constant in void context at test.pl line 2.
    C
    3

    C:\home>

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
     
    Gunnar Hjalmarsson, Mar 25, 2008
    #18
  19. Saurabh Jain

    Ben Bullock Guest

    On Mar 26, 8:24 am, Gunnar Hjalmarsson <> wrote:
    > Ben Bullock wrote:
    > > If we can create lists, then there must be a data structures inside
    > > Perl which represents those created lists. Let's call it the "newly-
    > > created list" data structure. There is also a data structure for
    > > arrays, of course.

    >
    > > Can you tell me the difference between the "newly-created lists" data
    > > structure and the array data structure? Is there an example of Perl
    > > code where we can see how the "newly-created lists" differ from
    > > arrays? Or are these actually the same thing in practice?

    >
    > C:\home>type test.pl
    > use warnings;
    > $var1 = qw/A B C/;
    > print "$var1\n";
    > @array = qw/A B C/;
    > $var2 = @array;
    > print "$var2\n";


    Thank you for making this example, but I'm interested in the
    distinction between Ben Morrow's "newly-created lists" and arrays, not
    the distinction between a compile-time list and an array. Your program
    is merely an example of putting a compile-time list and an array into
    scalar context.
     
    Ben Bullock, Mar 26, 2008
    #19
  20. Saurabh Jain

    Ben Morrow Guest

    Quoth Ben Bullock <>:
    > On Mar 26, 1:37 am, Ben Morrow <> wrote:
    >
    > > No, you're misunderstanding. A list is immutable *after it has been
    > > created*. Obviously you can create lists with any contents, otherwise
    > > you would be limited to using only lists compiled into perl.

    >
    > If we can create lists, then there must be a data structures inside
    > Perl which represents those created lists. Let's call it the "newly-
    > created list" data structure.


    Let's call it 'the stack'. Lists are ephemeral in Perl: they have no
    existence beyond the end of the expression they are part of (counting
    sub return and the corresponding call as two parts of a single
    expression). If you want to keep them you have to assign them into an
    array.

    > There is also a data structure for arrays, of course.


    Arrays are a real data structure, allocated somewhere on the heap, with
    a normal ref-counted lifetime. This is, fundamentally, the difference
    between the two.

    > Can you tell me the difference between the "newly-created lists" data
    > structure and the array data structure? Is there an example of Perl
    > code where we can see how the "newly-created lists" differ from
    > arrays? Or are these actually the same thing in practice?


    Apart from the boring implementation details above, lists are read-only,
    even when built from read-write values:

    ~% perl
    my @ary = qw/a b c/;
    $ary[0] = "f";
    my ($x, $y, $z) = qw/a b c/;
    ($x, $y, $z)[0] = "f";
    Can't modify list slice in scalar assignment at - line 4, near ""f";"
    Execution of - aborted due to compilation errors.
    ~%

    List assignment is an exception to this, of course. The only constant in
    Perl is that every rule has an exception :).

    > : for an example where you
    > > can, see Tie::File, which reads a file into an *array* instead.

    >
    > OK:
    >
    > http://perldoc.perl.org/Tie/File.html
    >
    > But it says very shortly into the documentation "The file is not
    > loaded into memory" and "Changes to the array are reflected in the
    > file immediately". So your statement about "reads a file into an
    > array" is wrong. It is not reading the file into an array. It is
    > making a file appear to be an array.


    Yes, of course. However, it is a closer fit to 'put this file into an
    array' than @ary = <HANDLE>, because it is modifiable, and the
    modifications turn up in the file just as you would expect.

    > To read a file into an array use:
    >
    > my @ary = <handle>;


    No. This reads the contents of the file into a list, and then assigns
    that list to an array. The contents of the array are no longer anything
    to do with the file. For instance,

    my ($x, $y, $z, @foo) = <HANDLE>, qw/a b c/;

    works perfectly well, or

    my @random = List::Util::shuffle <HANDLE>;

    > Then one can loop over the array using
    >
    > foreach (@ary)


    Yes, if you must. However (and this was the original point), by this
    time you've already read the whole file. If you're just going to process
    it line-by-line that's (probably) a waste of time.

    Ben
     
    Ben Morrow, Mar 26, 2008
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Arjun
    Replies:
    2
    Views:
    1,132
    Cameron Simpson
    Feb 27, 2010
  2. gavino
    Replies:
    4
    Views:
    557
    gavino
    Sep 20, 2010
  3. Jean-Michel
    Replies:
    0
    Views:
    383
    Jean-Michel
    Dec 22, 2007
  4. Replies:
    11
    Views:
    216
    John W. Krahn
    Feb 15, 2006
  5. Andrew DeFaria
    Replies:
    1
    Views:
    229
    Ben Morrow
    Jan 30, 2008
Loading...

Share This Page