Closing files

Discussion in 'Python' started by Henrik Holm, Nov 29, 2004.

  1. Henrik Holm

    Henrik Holm Guest

    I have recently started playing around with Python. Some of the things
    I have done have involved reading files. The way I do this is along the
    lines of

    f = file('file.txt')
    lines = f.readlines()
    f.close()

    I have noticed that it is possible to do this in one line:

    lines = file('file.txt').readlines()

    My question is: does the file get closed if I do the reading in this
    manner?

    Similarly, for reading the output from other programs or system
    commands, I would do:

    o = popen('executable')
    lines = o.readlines()
    o.close()

    Is it OK to do this with a one-liner as well, with

    lines = popen('executable').readlines()

    without closing the file object?

    Thanks,
    Henrik Holm
    Henrik Holm, Nov 29, 2004
    #1
    1. Advertising

  2. (Henrik Holm) writes:

    [+]
    > I have recently started playing around with Python. Some of the things
    > I have done have involved reading files. The way I do this is along the
    > lines of
    >
    > f = file('file.txt')
    > lines = f.readlines()
    > f.close()


    Verbose and readable. A typical approach.

    [+]
    >
    > I have noticed that it is possible to do this in one line:
    >
    > lines = file('file.txt').readlines()
    >
    > My question is: does the file get closed if I do the reading in this
    > manner?


    It appears to be closed on my Linux system as by using this shorthand
    approach, you are not saving any reference to the file object.

    Evidently, it is closed immediatly after the statement executes.
    There's no point in the file staying open as you'd have no way to
    refer to it any more.

    [+]
    >
    > Similarly, for reading the output from other programs or system
    > commands, I would do:
    >
    > o = popen('executable')
    > lines = o.readlines()
    > o.close()
    >
    > Is it OK to do this with a one-liner as well, with
    >
    > lines = popen('executable').readlines()


    My guess is that this is probably OK too. At least for simple cases.

    My experience has been, that by going for maximum terseness, you
    eventually get into trouble somehow.

    YMMV

    [+]
    >
    > without closing the file object?


    If you are going to have one or more references to the file or pipe
    object, you must either close them or delete the reference to the
    object using del().

    If you are running on Unix, do some experimentation with the 'fuser'
    or 'lsof' commands to see if the system is reporting the files open or
    closed.

    Or why not for fun;

    while True:
    file('/tmp/foo')

    I bet this runs forever and you don't run out of open file descriptors
    :)


    [+]
    >
    > Thanks,
    > Henrik Holm


    --
    -------------------------------------------------------------------------------
    Jerry Sievers 305 854-3001 (home) WWW ECommerce Consultant
    305 321-1144 (mobile http://www.JerrySievers.com/
    Jerry Sievers, Nov 29, 2004
    #2
    1. Advertising

  3. Henrik Holm wrote:
    > I have recently started playing around with Python. Some of the things
    > I have done have involved reading files. The way I do this is along the
    > lines of
    >
    > f = file('file.txt')
    > lines = f.readlines()
    > f.close()
    >
    > I have noticed that it is possible to do this in one line:
    >
    > lines = file('file.txt').readlines()
    >
    > My question is: does the file get closed if I do the reading in this
    > manner?


    The file gets closed in CPython because file objects are closed when the
    last reference to them gets deleted. (I guess when the .readlines ()
    returns)

    When you use Jython or IronPython, the file will be closed sometime
    later when the file object gets garbage collected. This can lead to
    problems becaause the process might run out of file handles before the
    garbage collection and file locks are held a lot longer.

    This led to two camps.

    One camp thinks that the correct way would be
    f = file('file.txt')
    try:
    lines = f.readlines()
    finally:
    f.close()

    The other camp thinks that reference counting should be implemented in
    all Python implementations.

    And then there are a lot of people who think that changing all the
    readlines one liner would be quite easy should the need of a Jython port
    arrive, so why bother about it now?

    Daniel
    Daniel Dittmar, Nov 29, 2004
    #3
  4. Daniel Dittmar wrote:
    > And then there are a lot of people who think that changing all the
    > readlines one liner would be quite easy should the need of a Jython port
    > arrive, so why bother about it now?


    The problem with this approach is, when the time for the Jython port
    arrives, do you remember to do it?

    --
    Timo Virkkala
    Timo Virkkala, Nov 29, 2004
    #4
  5. Timo Virkkala wrote:
    > Daniel Dittmar wrote:
    >> And then there are a lot of people who think that changing all the
    >> readlines one liner would be quite easy should the need of a Jython
    >> port arrive, so why bother about it now?

    >
    > The problem with this approach is, when the time for the Jython port
    > arrives, do you remember to do it?


    If you can write a unit test now that would fail using Jython, then
    you're going to be reminded quickly.

    And perhaps there will be changes to Python
    - that make the oppen().readlines () version work in fully garbage
    collected environments; like closing the inner file handle when the end
    of the file is reached
    - that suggest a different solution; like declarations on local
    variables that say "call destructor when object goes out of scope"

    Then, having more complicated code today doesn't buy you anything.

    Daniel
    Daniel Dittmar, Nov 29, 2004
    #5
  6. Henrik Holm

    Nick Coghlan Guest

    Daniel Dittmar wrote:
    > - that suggest a different solution; like declarations on local
    > variables that say "call destructor when object goes out of scope"


    You may be interested in PEP 310 (reliable acquisition/release pairs):
    http://www.python.org/peps/pep-0310.html

    (Although if that idea gets adopted, it is unlikely to use the 'with' keyword -
    see PEP 3000 for the reason why)

    Regards,
    Nick.
    Nick Coghlan, Nov 30, 2004
    #6
  7. > Daniel Dittmar wrote:
    > >- that suggest a different solution; like declarations on local
    > >variables that say "call destructor when object goes out of scope"


    I did not follow all of this thread (that precise subject reoccurs
    once in a while, with some regularity), but I merely would like to
    point out that objects never go "out of scope". Only variables do,
    and variables hold references to objects. When the last reference to
    an object disappears, only then the destructor is called. With Python
    (that particular Python that some call C-Python), this is guaranteed.

    [Nick Coghlan]
    > You may be interested in PEP 310 (reliable acquisition/release pairs):
    > http://www.python.org/peps/pep-0310.html


    Such a PEP might be useful for Jython or other Python-like languages.
    Yet with Python, the real thing, the effect of the PEP is easily
    achieved by relying on the timely finalisation of objects. For example,
    in Pymacs, the `let' and other various `save-' constructs of Emacs Lisp
    are simulated through Let() objects, used like in this example:

    let = Let().push_excursion()
    if True :
    ... user code ...
    del let

    Here, the `if' sole-purpose is to indent the "with" block, and so, to
    make the whole construct more legible.

    --
    Fran├žois Pinard http://pinard.progiciels-bpi.ca
    =?iso-8859-1?Q?Fran=E7ois?= Pinard, Nov 30, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. E Tepp

    Closing Files

    E Tepp, Feb 2, 2004, in forum: Java
    Replies:
    1
    Views:
    354
    Mike Schilling
    Feb 2, 2004
  2. Sandra-24

    Opening files without closing them

    Sandra-24, Mar 5, 2006, in forum: Python
    Replies:
    15
    Views:
    513
    3c273
    Mar 8, 2006
  3. =?Utf-8?B?Vk1J?=

    Closing popup window when closing parent window?

    =?Utf-8?B?Vk1J?=, Feb 14, 2007, in forum: ASP .Net
    Replies:
    3
    Views:
    642
    Thomas Hansen
    Feb 15, 2007
  4. Matt
    Replies:
    35
    Views:
    1,352
    J. Davidson
    Nov 21, 2008
  5. thomas
    Replies:
    0
    Views:
    217
    thomas
    Oct 23, 2003
Loading...

Share This Page