file.close()

B

Ben Finney

Use of external resources -- which should be released as soon as
you're done with them -- are best done in try/finally clauses.

It seems that nesting the 'try' clauses doesn't scale well. What if
fifty files are opened? Must the nesting level of the 'try' clauses be
fifty also, to close them promptly?
 
B

Bryan

i'm curious to know how others handle the closing of files. i seem to
always end up with this pattern even though i rarely see others do it.

f1 = file('file1')
try:
# process f1
finally:
f1.close()

which explicitly closes f1

or for multiple files:

f1 = file('file1')
try:
f2 = file('file2')
try:
# process f1 & f2
finally:
f2.close()
finally:
f1.close()

which explicitly closes f1 & f2
any exceptions opening f1 or f2 is handled outside of this structure or is
allowed to fall out of the program. i'm aware that files will automatically
be closed when the process exits. i'm just curious how others do this for
small scripts and larger programs. is what i'm doing is overkill?

thanks,

bryan
 
E

Erik Max Francis

Bryan said:
which explicitly closes f1 & f2
any exceptions opening f1 or f2 is handled outside of this structure
or is
allowed to fall out of the program. i'm aware that files will
automatically
be closed when the process exits. i'm just curious how others do this
for
small scripts and larger programs. is what i'm doing is overkill?

No, not at all; it's safe and portable. Python the language does not
specify when objects get reclaimed, although CPython the implementation
does it promptly. Use of external resources -- which should be released
as soon as you're done with them -- are best done in try/finally
clauses.
 
B

Ben Finney

Bryan said:
f1 = file('file1')
try:
f2 = file('file2')
try:
# process f1 & f2
finally:
f2.close()
finally:
f1.close()


Ben said:
It seems that nesting the 'try' clauses doesn't scale well. What if
fifty files are opened? Must the nesting level of the 'try' clauses
be fifty also, to close them promptly?

If you're manipulating fifty files in one block, presumably you're
doing so in a uniform way:

allFiles = [...]
try:
...
finally:
for eachFile in allFiles:
eachFile.close()

This doesn't match Bryan's nested structure above, which you blessed as
not "overkill" (in his words). It was this that I considered a
poorly-scaling structure, or "overkill" since only one 'try' block is
required. Do you disagree?
 
E

Erik Max Francis

Ben said:
It seems that nesting the 'try' clauses doesn't scale well. What if
fifty files are opened? Must the nesting level of the 'try' clauses
be
fifty also, to close them promptly?

If you're manipulating fifty files in one block, presumably you're doing
so in a uniform way:

allFiles = [...]
try:
...
finally:
for eachFile in allFiles:
eachFile.close()
 
C

Carl Banks

Ben said:
It seems that nesting the 'try' clauses doesn't scale well. What if
fifty files are opened? Must the nesting level of the 'try' clauses be
fifty also, to close them promptly?

If you need 50 open files, you almost certainly want to have a way of
organizing them. Probably they'll be in a list or dictionary. So, if
they're in a list, for example, you can do this:


filelist = []
try:
filelist.append(open(filename[0]))
filelist.append(open(filename[1]))
...
do_something(filelist)
finally:
for f in filelist:
f.close()
 
E

Erik Max Francis

Ben said:
This doesn't match Bryan's nested structure above, which you blessed
as
not "overkill" (in his words). It was this that I considered a
poorly-scaling structure, or "overkill" since only one 'try' block is
required. Do you disagree?

It uses try/finally to secure the closing of many files in a timely
manner. In that sense, it certainly fits the pattern. It doesn't have
the same nested pattern, but try/finally isn't at issue here. If you
had code which opened 50 files and looked like:

fileOne = file(...)
fileTwo = file(...)
fileThree = file(...)
...
fileFortyNine = file(...)
fileFifty = file(...)

I would say you are doing something wrong. A more systematic handling
of many, many files is indicated whether or not you're using the
try/finally idiom to ensure files get closed in a timely manner.
 
B

Bryan

If you need 50 open files, you almost certainly want to have a way of
organizing them. Probably they'll be in a list or dictionary. So, if
they're in a list, for example, you can do this:


filelist = []
try:
filelist.append(open(filename[0]))
filelist.append(open(filename[1]))
...
do_something(filelist)
finally:
for f in filelist:
f.close()


erik, carl... thanks... this is exactly what i was looking for

bryan
 
B

Ben Finney

filelist = []
try:
filelist.append(open(filename[0]))
filelist.append(open(filename[1]))
...
do_something(filelist)
finally:
for f in filelist:
f.close()

erik, carl... thanks... this is exactly what i was looking for

The only substantial difference I see between this and what you
originally posted, is that there is only one 'try...finally' block for
all the file open/close operations. Is this what you were wanting
clarified?
 
B

Ben Finney

It doesn't have the same nested pattern, but try/finally isn't at
issue here.

Judging by Bryan's responses elsewhere in this thread, the multiple
nested 'try...finally' is indeed what he was asking about. The question
seems to be answered now.
 
B

Bryan

Ben Finney said:
filelist = []
try:
filelist.append(open(filename[0]))
filelist.append(open(filename[1]))
...
do_something(filelist)
finally:
for f in filelist:
f.close()

erik, carl... thanks... this is exactly what i was looking for

The only substantial difference I see between this and what you
originally posted, is that there is only one 'try...finally' block for
all the file open/close operations. Is this what you were wanting
clarified?

well, it wasn't exactly clarification as much as seeing another more
scalable solution. i know a lot of people don't explicitly close files in
python, but i always do even for small scripts.

thanks again,

bryan
 
F

Francois Pinard

[Bryan]
I'm curious to know how others handle the closing of files. [...] I'm
aware that files will automatically be closed when the process exits.

For one, I systematically avoid cluttering my code with unneeded `close'.
The advantages are simplicity and legibility, both utterly important to me.

However, I do understand that if I ever have to move a Python script
to Jython, I will have to revise my scripts for adding the clutter I am
sparing today. I'm quite accepting to do that revision if this occurs.
Until then, I prefer keeping my scripts as neat as possible.

For me, explicitely closing a file, for which the only reference is about
to disappear through function exiting, would be very similar to using
`del' on any variable I happened to use in that function: gross overkill...

The only reason to call `close' explicitly is when there is a need to close
prematurely. Absolutely no doubt that such needs exist at times. But
closing all the time "just in case" is symptomatic of unsure programming.
Or else, it is using Python while still thinking in other languages.
 
B

Bryan

Francois Pinard said:
[Bryan]
I'm curious to know how others handle the closing of files. [...] I'm
aware that files will automatically be closed when the process exits.

For one, I systematically avoid cluttering my code with unneeded `close'.
The advantages are simplicity and legibility, both utterly important to me.

However, I do understand that if I ever have to move a Python script
to Jython, I will have to revise my scripts for adding the clutter I am
sparing today. I'm quite accepting to do that revision if this occurs.
Until then, I prefer keeping my scripts as neat as possible.

For me, explicitely closing a file, for which the only reference is about
to disappear through function exiting, would be very similar to using
`del' on any variable I happened to use in that function: gross overkill...

The only reason to call `close' explicitly is when there is a need to close
prematurely. Absolutely no doubt that such needs exist at times. But
closing all the time "just in case" is symptomatic of unsure programming.
Or else, it is using Python while still thinking in other languages.

you are correct this is so awesome... at least for me it is... now i remove
a lot of my clutter too :) i just did some tests on windows by trying to
delete file1 at the command prompt when raw_input is called.

f = file('file1')
raw_input('pause')

### the file is NOT closed


file('file1')
raw_input('pause')

### the file IS closed


f = file('file1')
del f
raw_input('pause')

### the file IS closed


def foo():
f = file('file1')

foo()
raw_input('pause')

### the file IS closed



can you explain to me how the file gets closed? i'm sure that garbage
collection hasn't happed at the point that i call raw_input. it must have
something to do with the reference count of the file object. does python
immediately call close for you when the reference count goes to zero? i
want the dirty details...

thanks,

bryan
 
U

Ulrich Petri

Francois Pinard said:
[Bryan]
I'm curious to know how others handle the closing of files. [...] I'm
aware that files will automatically be closed when the process exits.

For one, I systematically avoid cluttering my code with unneeded `close'.
The advantages are simplicity and legibility, both utterly important to me.

However, I do understand that if I ever have to move a Python script
to Jython, I will have to revise my scripts for adding the clutter I am
sparing today. I'm quite accepting to do that revision if this occurs.
Until then, I prefer keeping my scripts as neat as possible.

For me, explicitely closing a file, for which the only reference is about
to disappear through function exiting, would be very similar to using
`del' on any variable I happened to use in that function: gross overkill...

The only reason to call `close' explicitly is when there is a need to close
prematurely. Absolutely no doubt that such needs exist at times. But
closing all the time "just in case" is symptomatic of unsure programming.
Or else, it is using Python while still thinking in other languages.

fire up your python and type: "import this" read the output and rethink your
codint techniques

Ciao Ulrich
 
E

Erik Max Francis

Bengt said:
1) Is f.done() really necessary? I.e., doesn't an explicit del f take
care of it
if the object has been coded with a __del__ method? I.e., the idea
was to get
the effect of CPython's immediate effective del on ref count going
to zero, right?

It wouldn't if there were circular references at that point. If you're
going to have some kind of `with' structure that constraints lifetimes,
I'd think you'd probably want something more concrete than just object
deletion; you'd want to make sure a "Stop whatever you were doing now"
method were present and called. But maybe that really depends on the
primary thing that the `with' construct would be used for.
 
H

Hallvard B Furuseth

Francois said:
For one, I systematically avoid cluttering my code with unneeded `close'.

What happens if close fails during GC? Will it still raise an
exception? If so, the exception could happen at an unfortunate place in
the code.

Um. Explicit close does raise an exception if it fails, right?
 
P

Paul Rubin

First reaction == +1, but some questions...

1) Is f.done() really necessary? I.e., doesn't an explicit del f
take care of it if the object has been coded with a __del__
method? I.e., the idea was to get the effect of CPython's
immediate effective del on ref count going to zero, right?

The ref count might not be zero. Something inside the "with" block
might make a new reference and leave it around.
2) how about with two files (or other multiple resources)?

with f1,f2 = file('f1'), file('f2'):
[do stuff with f1 & f2]

What is the canonical expansion?

I think f1.done and f2.done should both get called.
Also, what about the attribute version, i.e.,

ob.f = file(frob)
try:
[do stuff with ob.f]
finally:
del ob.f # calls f.__del__, which calls/does f.close()

I.e., ob.f = something could raise an exception (e.g., a read-only property)
*after* file(frob) has succeeded. So I guess the easiest would be to limit
the left hand side to plain names...

The assignment should be inside the try. If I had it on the outside
before, that was an error.
 
B

Bengt Richter

The ref count might not be zero. Something inside the "with" block
might make a new reference and leave it around.
Aha. In that case, is the del just a courtesy default action?
2) how about with two files (or other multiple resources)?

with f1,f2 = file('f1'), file('f2'):
[do stuff with f1 & f2]

What is the canonical expansion?

I think f1.done and f2.done should both get called.
Consistently ;-)
Also, what about the attribute version, i.e.,

ob.f = file(frob)
try:
[do stuff with ob.f]
finally:
del ob.f # calls f.__del__, which calls/does f.close()

I.e., ob.f = something could raise an exception (e.g., a read-only property)
*after* file(frob) has succeeded. So I guess the easiest would be to limit
the left hand side to plain names...

The assignment should be inside the try. If I had it on the outside
before, that was an error.

Really? Don't you want a failing f = file(frob) exception to skip the finally,
since f might not even be bound to anything in that case?

The trouble I was pointing to is having two possible causes for exception, and only
one of them being of relevance to the file resource.

Maybe if you wanted to go to the trouble, it could be split something like

with ob.f = file(frob):
[do stuff with ob.f]

becoming

_tmp = file(frob)
try:
ob.f = _tmp
[ do stuff with ob.f ]
finally:
_tmp.done()
del _tmp
del ob.f

Not sure about the order. Plain ob.f.done() would not guarantee a call to _tmp.done(),
since ob could refuse to produce f, and f could be a wrapper produced during ob.f = _tmp,
and it might not have a __del__ method etc etc. _tmp is just for some internal temp binding.

Regards,
Bengt Richter
 
D

Dennis Lee Bieber

Hallvard B Furuseth fed this fish to the penguins on Friday 25 July
2003 12:51 pm:

Um. Explicit close does raise an exception if it fails, right?

[wulfraed@beastie wulfraed]$ python
Python 2.2 (#1, Nov 5 2002, 15:43:24)
[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)] on linux-i386
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):

Looks like it doesn't care... As long as the file /had/ been opened
first (doesn't look like the interactive interpreter collected f until
the del either).


--
 
E

Erik Max Francis

Dennis said:
Hallvard B Furuseth fed this fish to the penguins on Friday 25 July
2003 12:51 pm:

...
Looks like it doesn't care... As long as the file /had/ been
opened
first (doesn't look like the interactive interpreter collected f until
the del either).

I don't think you've demonstrated that; all you've shown is that builtin
Python file objects make file closing idempotent. You haven't
demonstrated a case where there actually is an I/O error that occurs
when .close gets called. In particular, I _really_ don't know what you
meant to show with this snippet, as it really has nothing to do with
files at all:
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,731
Messages
2,569,432
Members
44,834
Latest member
BuyCannaLabsCBD

Latest Threads

Top