Closing files

[Henrik Holm]

I have recently started playing around with Python. Some of the things
I have done have involved reading files. The way I do this is along the
lines of

f = file('file.txt')
lines = f.readlines()
f.close()

I have noticed that it is possible to do this in one line:

lines = file('file.txt').readlines()

My question is: does the file get closed if I do the reading in this
manner?

Similarly, for reading the output from other programs or system
commands, I would do:

o = popen('executable')
lines = o.readlines()
o.close()

Is it OK to do this with a one-liner as well, with

lines = popen('executable').readlines()

without closing the file object?

Thanks,
Henrik Holm

 

[Jerry Sievers]

(e-mail address removed) (Henrik Holm) writes:

[+]

I have recently started playing around with Python. Some of the things
I have done have involved reading files. The way I do this is along the
lines of

f = file('file.txt')
lines = f.readlines()
f.close()

Verbose and readable. A typical approach.

[+]

I have noticed that it is possible to do this in one line:

lines = file('file.txt').readlines()

My question is: does the file get closed if I do the reading in this
manner?

It appears to be closed on my Linux system as by using this shorthand
approach, you are not saving any reference to the file object.

Evidently, it is closed immediatly after the statement executes.
There's no point in the file staying open as you'd have no way to
refer to it any more.

[+]

Similarly, for reading the output from other programs or system
commands, I would do:

o = popen('executable')
lines = o.readlines()
o.close()

Is it OK to do this with a one-liner as well, with

lines = popen('executable').readlines()

My guess is that this is probably OK too. At least for simple cases.

My experience has been, that by going for maximum terseness, you
eventually get into trouble somehow.

YMMV

[+]

without closing the file object?

If you are going to have one or more references to the file or pipe
object, you must either close them or delete the reference to the
object using del().

If you are running on Unix, do some experimentation with the 'fuser'
or 'lsof' commands to see if the system is reporting the files open or
closed.

Or why not for fun;

while True:
file('/tmp/foo')

I bet this runs forever and you don't run out of open file descriptors



[+]

 

[Daniel Dittmar]

I have recently started playing around with Python. Some of the things
I have done have involved reading files. The way I do this is along the
lines of

f = file('file.txt')
lines = f.readlines()
f.close()

I have noticed that it is possible to do this in one line:

lines = file('file.txt').readlines()

My question is: does the file get closed if I do the reading in this
manner?

The file gets closed in CPython because file objects are closed when the
last reference to them gets deleted. (I guess when the .readlines ()
returns)

When you use Jython or IronPython, the file will be closed sometime
later when the file object gets garbage collected. This can lead to
problems becaause the process might run out of file handles before the
garbage collection and file locks are held a lot longer.

This led to two camps.

One camp thinks that the correct way would be
f = file('file.txt')
try:
lines = f.readlines()
finally:
f.close()

The other camp thinks that reference counting should be implemented in
all Python implementations.

And then there are a lot of people who think that changing all the
readlines one liner would be quite easy should the need of a Jython port
arrive, so why bother about it now?

Daniel

 

[Timo Virkkala]

And then there are a lot of people who think that changing all the
readlines one liner would be quite easy should the need of a Jython port
arrive, so why bother about it now?

The problem with this approach is, when the time for the Jython port
arrives, do you remember to do it?

 

[Daniel Dittmar]

The problem with this approach is, when the time for the Jython port
arrives, do you remember to do it?

If you can write a unit test now that would fail using Jython, then
you're going to be reminded quickly.

And perhaps there will be changes to Python
- that make the oppen().readlines () version work in fully garbage
collected environments; like closing the inner file handle when the end
of the file is reached
- that suggest a different solution; like declarations on local
variables that say "call destructor when object goes out of scope"

Then, having more complicated code today doesn't buy you anything.

Daniel

 

[Nick Coghlan]

- that suggest a different solution; like declarations on local
variables that say "call destructor when object goes out of scope"

You may be interested in PEP 310 (reliable acquisition/release pairs):
http://www.python.org/peps/pep-0310.html

(Although if that idea gets adopted, it is unlikely to use the 'with' keyword -
see PEP 3000 for the reason why)

Regards,
Nick.

 

[=?iso-8859-1?Q?Fran=E7ois?= Pinard]

I did not follow all of this thread (that precise subject reoccurs
once in a while, with some regularity), but I merely would like to
point out that objects never go "out of scope". Only variables do,
and variables hold references to objects. When the last reference to
an object disappears, only then the destructor is called. With Python
(that particular Python that some call C-Python), this is guaranteed.

[Nick Coghlan]

You may be interested in PEP 310 (reliable acquisition/release pairs):
http://www.python.org/peps/pep-0310.html

Such a PEP might be useful for Jython or other Python-like languages.
Yet with Python, the real thing, the effect of the PEP is easily
achieved by relying on the timely finalisation of objects. For example,
in Pymacs, the `let' and other various `save-' constructs of Emacs Lisp
are simulated through Let() objects, used like in this example:

let = Let().push_excursion()
if True :
... user code ...
del let

Here, the `if' sole-purpose is to indent the "with" block, and so, to
make the whole construct more legible.