Closing files

Discussion in 'Python' started by Henrik Holm, Nov 29, 2004.

  1. Henrik Holm

    Henrik Holm Guest

    I have recently started playing around with Python. Some of the things
    I have done have involved reading files. The way I do this is along the
    lines of

    f = file('file.txt')
    lines = f.readlines()
    f.close()

    I have noticed that it is possible to do this in one line:

    lines = file('file.txt').readlines()

    My question is: does the file get closed if I do the reading in this
    manner?

    Similarly, for reading the output from other programs or system
    commands, I would do:

    o = popen('executable')
    lines = o.readlines()
    o.close()

    Is it OK to do this with a one-liner as well, with

    lines = popen('executable').readlines()

    without closing the file object?

    Thanks,
    Henrik Holm
     
    Henrik Holm, Nov 29, 2004
    #1
    1. Advertisements

  2. (Henrik Holm) writes:

    [+]
    Verbose and readable. A typical approach.

    [+]
    It appears to be closed on my Linux system as by using this shorthand
    approach, you are not saving any reference to the file object.

    Evidently, it is closed immediatly after the statement executes.
    There's no point in the file staying open as you'd have no way to
    refer to it any more.

    [+]
    My guess is that this is probably OK too. At least for simple cases.

    My experience has been, that by going for maximum terseness, you
    eventually get into trouble somehow.

    YMMV

    [+]
    If you are going to have one or more references to the file or pipe
    object, you must either close them or delete the reference to the
    object using del().

    If you are running on Unix, do some experimentation with the 'fuser'
    or 'lsof' commands to see if the system is reporting the files open or
    closed.

    Or why not for fun;

    while True:
    file('/tmp/foo')

    I bet this runs forever and you don't run out of open file descriptors
    :)


    [+]
     
    Jerry Sievers, Nov 29, 2004
    #2
    1. Advertisements

  3. The file gets closed in CPython because file objects are closed when the
    last reference to them gets deleted. (I guess when the .readlines ()
    returns)

    When you use Jython or IronPython, the file will be closed sometime
    later when the file object gets garbage collected. This can lead to
    problems becaause the process might run out of file handles before the
    garbage collection and file locks are held a lot longer.

    This led to two camps.

    One camp thinks that the correct way would be
    f = file('file.txt')
    try:
    lines = f.readlines()
    finally:
    f.close()

    The other camp thinks that reference counting should be implemented in
    all Python implementations.

    And then there are a lot of people who think that changing all the
    readlines one liner would be quite easy should the need of a Jython port
    arrive, so why bother about it now?

    Daniel
     
    Daniel Dittmar, Nov 29, 2004
    #3
  4. The problem with this approach is, when the time for the Jython port
    arrives, do you remember to do it?
     
    Timo Virkkala, Nov 29, 2004
    #4
  5. If you can write a unit test now that would fail using Jython, then
    you're going to be reminded quickly.

    And perhaps there will be changes to Python
    - that make the oppen().readlines () version work in fully garbage
    collected environments; like closing the inner file handle when the end
    of the file is reached
    - that suggest a different solution; like declarations on local
    variables that say "call destructor when object goes out of scope"

    Then, having more complicated code today doesn't buy you anything.

    Daniel
     
    Daniel Dittmar, Nov 29, 2004
    #5
  6. Henrik Holm

    Nick Coghlan Guest

    You may be interested in PEP 310 (reliable acquisition/release pairs):
    http://www.python.org/peps/pep-0310.html

    (Although if that idea gets adopted, it is unlikely to use the 'with' keyword -
    see PEP 3000 for the reason why)

    Regards,
    Nick.
     
    Nick Coghlan, Nov 30, 2004
    #6
  7. I did not follow all of this thread (that precise subject reoccurs
    once in a while, with some regularity), but I merely would like to
    point out that objects never go "out of scope". Only variables do,
    and variables hold references to objects. When the last reference to
    an object disappears, only then the destructor is called. With Python
    (that particular Python that some call C-Python), this is guaranteed.

    [Nick Coghlan]
    Such a PEP might be useful for Jython or other Python-like languages.
    Yet with Python, the real thing, the effect of the PEP is easily
    achieved by relying on the timely finalisation of objects. For example,
    in Pymacs, the `let' and other various `save-' constructs of Emacs Lisp
    are simulated through Let() objects, used like in this example:

    let = Let().push_excursion()
    if True :
    ... user code ...
    del let

    Here, the `if' sole-purpose is to indent the "with" block, and so, to
    make the whole construct more legible.
     
    =?iso-8859-1?Q?Fran=E7ois?= Pinard, Nov 30, 2004
    #7
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.