About open file for Read

Discussion in 'Python' started by moonhkt, Dec 10, 2012.

  1. moonhkt

    moonhkt Guest

    Hi All

    I am new in Python. When using open and then for line in f .

    Does it read all the data into f object ? or read line by line ?


    f=open(file, 'r')
    for line in f:
    if userstring in line:
    print "file: " + os.path.join(root,file)
    break
    f.close()


    moonhk
     
    moonhkt, Dec 10, 2012
    #1
    1. Advertising

  2. moonhkt

    Dave Angel Guest

    On 12/10/2012 11:36 AM, moonhkt wrote:
    > Hi All
    >
    > I am new in Python. When using open and then for line in f .
    >
    > Does it read all the data into f object ? or read line by line ?
    >
    >
    > f=open(file, 'r')
    > for line in f:
    > if userstring in line:
    > print "file: " + os.path.join(root,file)
    > break
    > f.close()
    >
    >
    > moonhk


    open() does not read the whole file into any object. There is buffering
    that goes on in the C libraries that open() calls, but that should be
    transparent to you for regular files.

    When you ask for a line, it'll read enough to fulfill that request, and
    maybe some extra that'll get held somewhere in the C runtime library.

    You should look into the 'with' statement, to avoid that f.close().
    That way the file will be closed, regardless of whether you get an
    exception or not.

    http://docs.python.org/2/reference/compound_stmts.html#index-15

    with open(file,. "r") as f:
    for line in f:
    etc.

    BTW, since you're in version 2.x, you should avoid hiding the builtin
    file object. Call it something like file_name, or infile_name.

    --

    DaveA
     
    Dave Angel, Dec 10, 2012
    #2
    1. Advertising

  3. moonhkt

    Peter Otten Guest

    Dave Angel wrote:

    > On 12/10/2012 11:36 AM, moonhkt wrote:
    >> Hi All
    >>
    >> I am new in Python. When using open and then for line in f .
    >>
    >> Does it read all the data into f object ? or read line by line ?
    >>
    >>
    >> f=open(file, 'r')
    >> for line in f:
    >> if userstring in line:
    >> print "file: " + os.path.join(root,file)
    >> break
    >> f.close()
    >>
    >>
    >> moonhk

    >
    > open() does not read the whole file into any object. There is buffering
    > that goes on in the C libraries that open() calls, but that should be
    > transparent to you for regular files.
    >
    > When you ask for a line, it'll read enough to fulfill that request, and
    > maybe some extra that'll get held somewhere in the C runtime library.
    >
    > You should look into the 'with' statement, to avoid that f.close().
    > That way the file will be closed, regardless of whether you get an
    > exception or not.
    >
    > http://docs.python.org/2/reference/compound_stmts.html#index-15
    >
    > with open(file,. "r") as f:
    > for line in f:
    > etc.
    >
    > BTW, since you're in version 2.x, you should avoid hiding the builtin
    > file object. Call it something like file_name, or infile_name.
    >


    Python does a bit of buffering on its own (which is why you cannot mix file
    iteration and .readline() calls):

    >>> with open("tmp.txt", "w") as f: f.writelines("%s\n" % i for i in

    range(10**6))
    ....
    >>> f = open("tmp.txt")
    >>> f.readline()

    '0\n'
    >>> f.tell()

    2
    >>> f.readline()

    '1\n'
    >>> f.tell()

    4
    >>> next(f) # a for-loop does this implicitly

    '2\n'
    >>> f.tell()

    8196 # after a next() call or the first loop iteration
    # part of the file is now in a buffer.
    >>> f.readline()

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    ValueError: Mixing iteration and read methods would lose data
    >>> f.seek(0, 2)
    >>> f.tell()

    6888890


    This is Python 2, in Python 3 f.tell() would fail after a next(f) call, but
    f.readline() continues to work.
     
    Peter Otten, Dec 10, 2012
    #3
  4. On Mon, 10 Dec 2012 08:36:22 -0800, moonhkt wrote:

    > Hi All
    >
    > I am new in Python. When using open and then for line in f .
    >
    > Does it read all the data into f object ? or read line by line ?


    Have you read the Fine Manual?

    http://docs.python.org/2/library/stdtypes.html#file-objects

    If you have read it, and the answer is still not clear, then please tell
    us so we can improve the documentation.

    `for line in open(file, "r"):` does not read the entire file into memory
    at once, it iterates over the file reading one line at a time.


    --
    Steven
     
    Steven D'Aprano, Dec 10, 2012
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ferbar
    Replies:
    9
    Views:
    375
    Keith Thompson
    Sep 25, 2005
  2. Sreejith K
    Replies:
    24
    Views:
    1,065
    Sreejith K
    Mar 24, 2009
  3. Sniper Abandon
    Replies:
    3
    Views:
    249
    Robert Klemme
    Nov 19, 2009
  4. Alex Dowad
    Replies:
    4
    Views:
    298
    Michel Demazure
    May 1, 2010
  5. mike
    Replies:
    3
    Views:
    168
Loading...

Share This Page