Semantics of file.close()

[seerdecker]

Hello,

I'm a Python beginner and I'm trying to open, write and close a file
in a
correct manner. I've RTFM, RTFS, and I've read this thread:
http://groups.google.ca/group/comp....+explicit+close&rnum=1&hl=en#98c731229d86b01d

I still cannot figure out the semantic of file.close(). As far as I
can
tell it is undocumented. Explanations and example follow.

There are two occasions where you have to close a file:

1) At the end of a series of writes to the file, to ensure that all
data
is written correctly to disk. In this case I want file.close() to
throw an exception if the file cannot be written (e.g. when there
is
no more disk space).

2) To clean up after an error occurred during the processing. In that
case I just want to close the file handle cleanly. I do NOT want an
exception to be thrown (e.g. in my finally clause).

Man page of fclose() -- in C:

fclose - close a stream

Upon successful completion 0 is returned. Otherwise, EOF is
returned
and the global variable errno is set to indicate the error. In
either
case any further access (including another call to fclose()) to
the stream results in undefined behaviour.

The man page of fclose() clearly indicates that fclose() may fail.
I've
already tested under Linux that fwrite() indicates success even if the
disk is full; the error is ONLY reported when close() is called.

Consider the following example:

file1 = None;
file2 = None;

try:
file1 = open("foo1.txt", "wb");
file2 = open("foo2.txt", "wb");
file1.close();
file2.close();

finally:
if file1: file1.close();
if file2: file2.close();

How do I ensure that the close() methods in my finally clause do not
throw an exception?

Thanks a lot,
Laurent Birtz

 

[seerdecker]

Sorry for the bad formatting; I've had to post this through the google
groups.

 

[Dan Bishop]

Hello,

I'm a Python beginner and I'm trying to open, write and close a file
in a
correct manner. I've RTFM, RTFS, and I've read this thread:http://groups.google.ca/group/comp.lang.python/browse_thread/thread/7...

I still cannot figure out the semantic of file.close(). As far as I
can
tell it is undocumented.

It's documented. Not necessarily very well, but it's documented.
Type help(file.close) at the interactive prompt.


close(...)
close() -> None or (perhaps) an integer. Close the file.

Sets data attribute .closed to True. A closed file cannot be used
for
further I/O operations. close() may be called more than once
without
error. Some kinds of file objects (for example, opened by
popen())
may return an exit status upon closing.

How do I ensure that the close() methods in my finally clause do not
throw an exception?

def quiet_close(fp):
"""Close a file, silently ignoring errors."""
try:
fp.close()
except IOError:
pass

 

[Matimus]

How do I ensure that the close() methods in my finally clause do not

throw an exception?

You have no choice. If close is going to fail, it will fail.
Fortunately you can catch the exception and continue on.
<code>
try:
try:
file1.write(somestuff)
finally:
file1.close()
except IOError:
pass
</code>
or you could wrap it in the inner scope:
<code>
try:
file1.write(somestuff)
finally:
try:
file1.close()
except IOError:
pass
</code>

This doesn't prevent the exception from happening, but it prevents the
user for seeing it. Also, you don't need to use semicolons in python.

 

[Evan Klitzke]

Hello,

I'm a Python beginner and I'm trying to open, write and close a file
in a
correct manner. I've RTFM, RTFS, and I've read this thread:
http://groups.google.ca/group/comp....+explicit+close&rnum=1&hl=en#98c731229d86b01d

I still cannot figure out the semantic of file.close(). As far as I
can
tell it is undocumented. Explanations and example follow.

There are two occasions where you have to close a file:

1) At the end of a series of writes to the file, to ensure that all
data
is written correctly to disk. In this case I want file.close() to
throw an exception if the file cannot be written (e.g. when there
is
no more disk space).

2) To clean up after an error occurred during the processing. In that
case I just want to close the file handle cleanly. I do NOT want an
exception to be thrown (e.g. in my finally clause).

Man page of fclose() -- in C:

fclose - close a stream

Upon successful completion 0 is returned. Otherwise, EOF is
returned
and the global variable errno is set to indicate the error. In
either
case any further access (including another call to fclose()) to
the stream results in undefined behaviour.

The man page of fclose() clearly indicates that fclose() may fail.
I've
already tested under Linux that fwrite() indicates success even if the
disk is full; the error is ONLY reported when close() is called.

Consider the following example:

file1 = None;
file2 = None;

try:
file1 = open("foo1.txt", "wb");
file2 = open("foo2.txt", "wb");
file1.close();
file2.close();

finally:
if file1: file1.close();
if file2: file2.close();

How do I ensure that the close() methods in my finally clause do not
throw an exception?

Thanks a lot,
Laurent Birtz

You should take a look at the man pages for close(2) and write(2) (not
fclose). Generally you will only get an error in C if you try to close
a file that isn't open. In Python you don't even have to worry about
that -- if you close a regular file object more than once no exception
will be thrown, _unless_ you are using os.close(), which mimics the C
behavior. If you are out of space, in C you will get an error returned
by the call to write (even if the data isn't actually flushed to disk
yet by the kernel). I'm pretty sure Python mimics this behavior, so an
exception would be called on the write, not on the close operation.

 

[Donn Cave]

How do I ensure that the close() methods in my finally clause do not
throw an exception?

You should take a look at the man pages for close(2) and write(2) (not
fclose). Generally you will only get an error in C if you try to close
a file that isn't open. In Python you don't even have to worry about
that -- if you close a regular file object more than once no exception
will be thrown, _unless_ you are using os.close(), which mimics the C
behavior. If you are out of space, in C you will get an error returned
by the call to write (even if the data isn't actually flushed to disk
yet by the kernel). I'm pretty sure Python mimics this behavior, so an
exception would be called on the write, not on the close operation.[/QUOTE]

No, he went to the trouble to test this, and he knows how it works. C
library I/O can return a disk full error on close, because the last of
the buffer will be flushed with write(2), and Python should raise an
exception at this point.

I don't think there's any remedy for it, other than the obvious -
either always flush, or wrap an explicit close in its own exception
handler.

Donn Cave, (e-mail address removed)

 

[Hrvoje Niksic]

You should take a look at the man pages for close(2) and write(2) (not
fclose). Generally you will only get an error in C if you try to close
a file that isn't open. In Python you don't even have to worry about
that -- if you close a regular file object more than once no exception
will be thrown, _unless_ you are using os.close(), which mimics the C
behavior. If you are out of space, in C you will get an error returned
by the call to write (even if the data isn't actually flushed to disk
yet by the kernel). I'm pretty sure Python mimics this behavior, so an
exception would be called on the write, not on the close operation.

But the writes are buffered, and close causes the buffer to be
flushed. file.close can throw an exception just like fclose, but it
will still ensure that the file is closed.

In the general case, you can't. Preferably you'd want to make sure
that both files are closed:

try:
f1 = file(...)
try:
f2 = file(...)
... do something with f1 and f2 ...
finally:
f2.close()
finally:
f1.close()

Now file.close would be called on both files regardless of where an
exception occurs. If you use Python 2.5, this would be a good use
case for the "nested" function from the contextlib module, which allow
you to write the above more elegantly:

from __future__ import with_statement
from contextlib import nested

with nested(file(...), file(...)) as (f1, f2):
... do something with f1 and f2 ...


Finally, most of this applies to files open for writing, where Python
is forced to flush the cache on close. If the file is opened for
reading, you can assume that the exception will not be raised (and you
can safely ignore it with `try: f.close() except IOError: pass' if you
want to be sure; after all, you can't lose data when closing a file
open for reading).

 

[Matthew Woodcraft]

I don't think there's any remedy for it, other than the obvious -
either always flush, or wrap an explicit close in its own exception
handler.

Even if you have flushed, close() can give an error with some filesystems.

-M-

 

[Evan Klitzke]

But the writes are buffered, and close causes the buffer to be
flushed. file.close can throw an exception just like fclose, but it
will still ensure that the file is closed.

Is this buffering being done by Python or the kernel? AFAIK, the
kernel will return a ENOSPC on a write if there is no space left on a
device, even if the device isn't _really_ full but will be full after
the kernel writes in memory buffers to disk. So a close statement (in
C) should never fail due to a lack of space -- that should be seen on
the write syscall, even if the kernel is doing buffering.

 

[Paul Rubin]

Is this buffering being done by Python or the kernel?

I think this refers to buffering done in the C stdio library, which
Python uses.

 

[Hrvoje Niksic]

Is this buffering being done by Python or the kernel?

It is done in the user space, by the C stdio library which Python
currently uses for IO.