file.close()

Discussion in 'Python' started by Ben Finney, Jul 24, 2003.

  1. Ben Finney

    Ben Finney Guest

    On Wed, 23 Jul 2003 20:29:27 -0700, Erik Max Francis wrote:
    > Bryan wrote:
    >> is what i'm doing is overkill?

    >
    > Use of external resources -- which should be released as soon as
    > you're done with them -- are best done in try/finally clauses.


    It seems that nesting the 'try' clauses doesn't scale well. What if
    fifty files are opened? Must the nesting level of the 'try' clauses be
    fifty also, to close them promptly?

    --
    \ "God forbid that any book should be banned. The practice is as |
    `\ indefensible as infanticide." -- Dame Rebecca West |
    _o__) |
    http://bignose.squidly.org/ 9CFE12B0 791A4267 887F520C B7AC2E51 BD41714B
    Ben Finney, Jul 24, 2003
    #1
    1. Advertising

  2. Ben Finney

    Bryan Guest

    i'm curious to know how others handle the closing of files. i seem to
    always end up with this pattern even though i rarely see others do it.

    f1 = file('file1')
    try:
    # process f1
    finally:
    f1.close()

    which explicitly closes f1

    or for multiple files:

    f1 = file('file1')
    try:
    f2 = file('file2')
    try:
    # process f1 & f2
    finally:
    f2.close()
    finally:
    f1.close()

    which explicitly closes f1 & f2
    any exceptions opening f1 or f2 is handled outside of this structure or is
    allowed to fall out of the program. i'm aware that files will automatically
    be closed when the process exits. i'm just curious how others do this for
    small scripts and larger programs. is what i'm doing is overkill?

    thanks,

    bryan
    Bryan, Jul 24, 2003
    #2
    1. Advertising

  3. Bryan wrote:

    > which explicitly closes f1 & f2
    > any exceptions opening f1 or f2 is handled outside of this structure
    > or is
    > allowed to fall out of the program. i'm aware that files will
    > automatically
    > be closed when the process exits. i'm just curious how others do this
    > for
    > small scripts and larger programs. is what i'm doing is overkill?


    No, not at all; it's safe and portable. Python the language does not
    specify when objects get reclaimed, although CPython the implementation
    does it promptly. Use of external resources -- which should be released
    as soon as you're done with them -- are best done in try/finally
    clauses.

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ It is fatal to enter any war without the will to win it.
    \__/ Douglas MacArthur
    Erik Max Francis, Jul 24, 2003
    #3
  4. Ben Finney

    Ben Finney Guest

    Bryan (original poster) wrote:
    > f1 = file('file1')
    > try:
    > f2 = file('file2')
    > try:
    > # process f1 & f2
    > finally:
    > f2.close()
    > finally:
    > f1.close()



    On Wed, 23 Jul 2003 21:12:34 -0700, Erik Max Francis wrote:
    > Ben Finney wrote:
    >> It seems that nesting the 'try' clauses doesn't scale well. What if
    >> fifty files are opened? Must the nesting level of the 'try' clauses
    >> be fifty also, to close them promptly?

    >
    > If you're manipulating fifty files in one block, presumably you're
    > doing so in a uniform way:
    >
    > allFiles = [...]
    > try:
    > ...
    > finally:
    > for eachFile in allFiles:
    > eachFile.close()


    This doesn't match Bryan's nested structure above, which you blessed as
    not "overkill" (in his words). It was this that I considered a
    poorly-scaling structure, or "overkill" since only one 'try' block is
    required. Do you disagree?

    --
    \ "Too many Indians spoil the golden egg." -- Sir Joh |
    `\ Bjelke-Petersen |
    _o__) |
    http://bignose.squidly.org/ 9CFE12B0 791A4267 887F520C B7AC2E51 BD41714B
    Ben Finney, Jul 24, 2003
    #4
  5. Ben Finney wrote:

    > It seems that nesting the 'try' clauses doesn't scale well. What if
    > fifty files are opened? Must the nesting level of the 'try' clauses
    > be
    > fifty also, to close them promptly?


    If you're manipulating fifty files in one block, presumably you're doing
    so in a uniform way:

    allFiles = [...]
    try:
    ...
    finally:
    for eachFile in allFiles:
    eachFile.close()

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ Man is a hating rather than a loving animal.
    \__/ Rebecca West
    Erik Max Francis, Jul 24, 2003
    #5
  6. Ben Finney

    Carl Banks Guest

    Ben Finney wrote:
    > On Wed, 23 Jul 2003 20:29:27 -0700, Erik Max Francis wrote:
    >> Bryan wrote:
    >>> is what i'm doing is overkill?

    >>
    >> Use of external resources -- which should be released as soon as
    >> you're done with them -- are best done in try/finally clauses.

    >
    > It seems that nesting the 'try' clauses doesn't scale well. What if
    > fifty files are opened? Must the nesting level of the 'try' clauses be
    > fifty also, to close them promptly?


    If you need 50 open files, you almost certainly want to have a way of
    organizing them. Probably they'll be in a list or dictionary. So, if
    they're in a list, for example, you can do this:


    filelist = []
    try:
    filelist.append(open(filename[0]))
    filelist.append(open(filename[1]))
    ...
    do_something(filelist)
    finally:
    for f in filelist:
    f.close()


    --
    CARL BANKS
    Carl Banks, Jul 24, 2003
    #6
  7. Ben Finney wrote:

    > This doesn't match Bryan's nested structure above, which you blessed
    > as
    > not "overkill" (in his words). It was this that I considered a
    > poorly-scaling structure, or "overkill" since only one 'try' block is
    > required. Do you disagree?


    It uses try/finally to secure the closing of many files in a timely
    manner. In that sense, it certainly fits the pattern. It doesn't have
    the same nested pattern, but try/finally isn't at issue here. If you
    had code which opened 50 files and looked like:

    fileOne = file(...)
    fileTwo = file(...)
    fileThree = file(...)
    ...
    fileFortyNine = file(...)
    fileFifty = file(...)

    I would say you are doing something wrong. A more systematic handling
    of many, many files is indicated whether or not you're using the
    try/finally idiom to ensure files get closed in a timely manner.

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ Together we can take this one day at a time
    \__/ Sweetbox
    Erik Max Francis, Jul 24, 2003
    #7
  8. Ben Finney

    Bryan Guest

    >
    > If you need 50 open files, you almost certainly want to have a way of
    > organizing them. Probably they'll be in a list or dictionary. So, if
    > they're in a list, for example, you can do this:
    >
    >
    > filelist = []
    > try:
    > filelist.append(open(filename[0]))
    > filelist.append(open(filename[1]))
    > ...
    > do_something(filelist)
    > finally:
    > for f in filelist:
    > f.close()
    >



    erik, carl... thanks... this is exactly what i was looking for

    bryan
    Bryan, Jul 24, 2003
    #8
  9. Ben Finney

    Ben Finney Guest

    On Thu, 24 Jul 2003 05:20:15 GMT, Bryan wrote:
    >> filelist = []
    >> try:
    >> filelist.append(open(filename[0]))
    >> filelist.append(open(filename[1]))
    >> ...
    >> do_something(filelist)
    >> finally:
    >> for f in filelist:
    >> f.close()

    >
    > erik, carl... thanks... this is exactly what i was looking for


    The only substantial difference I see between this and what you
    originally posted, is that there is only one 'try...finally' block for
    all the file open/close operations. Is this what you were wanting
    clarified?

    --
    \ "I spent all my money on a FAX machine. Now I can only FAX |
    `\ collect." -- Steven Wright |
    _o__) |
    http://bignose.squidly.org/ 9CFE12B0 791A4267 887F520C B7AC2E51 BD41714B
    Ben Finney, Jul 24, 2003
    #9
  10. Ben Finney

    Ben Finney Guest

    On Wed, 23 Jul 2003 22:19:07 -0700, Erik Max Francis wrote:
    > Ben Finney wrote:
    >> This doesn't match Bryan's nested structure above, which you blessed
    >> as not "overkill" (in his words).

    > It doesn't have the same nested pattern, but try/finally isn't at
    > issue here.


    Judging by Bryan's responses elsewhere in this thread, the multiple
    nested 'try...finally' is indeed what he was asking about. The question
    seems to be answered now.

    --
    \ "Those who will not reason, are bigots, those who cannot, are |
    `\ fools, and those who dare not, are slaves." -- "Lord" George |
    _o__) Gordon Noel Byron |
    http://bignose.squidly.org/ 9CFE12B0 791A4267 887F520C B7AC2E51 BD41714B
    Ben Finney, Jul 24, 2003
    #10
  11. Ben Finney

    Bryan Guest

    "Ben Finney" <> wrote in
    message news:...
    > On Thu, 24 Jul 2003 05:20:15 GMT, Bryan wrote:
    > >> filelist = []
    > >> try:
    > >> filelist.append(open(filename[0]))
    > >> filelist.append(open(filename[1]))
    > >> ...
    > >> do_something(filelist)
    > >> finally:
    > >> for f in filelist:
    > >> f.close()

    > >
    > > erik, carl... thanks... this is exactly what i was looking for

    >
    > The only substantial difference I see between this and what you
    > originally posted, is that there is only one 'try...finally' block for
    > all the file open/close operations. Is this what you were wanting
    > clarified?
    >


    well, it wasn't exactly clarification as much as seeing another more
    scalable solution. i know a lot of people don't explicitly close files in
    python, but i always do even for small scripts.

    thanks again,

    bryan
    Bryan, Jul 24, 2003
    #11
  12. [Bryan]

    > I'm curious to know how others handle the closing of files. [...] I'm
    > aware that files will automatically be closed when the process exits.


    For one, I systematically avoid cluttering my code with unneeded `close'.
    The advantages are simplicity and legibility, both utterly important to me.

    However, I do understand that if I ever have to move a Python script
    to Jython, I will have to revise my scripts for adding the clutter I am
    sparing today. I'm quite accepting to do that revision if this occurs.
    Until then, I prefer keeping my scripts as neat as possible.

    For me, explicitely closing a file, for which the only reference is about
    to disappear through function exiting, would be very similar to using
    `del' on any variable I happened to use in that function: gross overkill...

    The only reason to call `close' explicitly is when there is a need to close
    prematurely. Absolutely no doubt that such needs exist at times. But
    closing all the time "just in case" is symptomatic of unsure programming.
    Or else, it is using Python while still thinking in other languages.

    --
    François Pinard http://www.iro.umontreal.ca/~pinard
    Francois Pinard, Jul 24, 2003
    #12
  13. Ben Finney

    Bryan Guest

    "Francois Pinard" <> wrote in message
    news:...
    > [Bryan]
    >
    > > I'm curious to know how others handle the closing of files. [...] I'm
    > > aware that files will automatically be closed when the process exits.

    >
    > For one, I systematically avoid cluttering my code with unneeded `close'.
    > The advantages are simplicity and legibility, both utterly important to

    me.
    >
    > However, I do understand that if I ever have to move a Python script
    > to Jython, I will have to revise my scripts for adding the clutter I am
    > sparing today. I'm quite accepting to do that revision if this occurs.
    > Until then, I prefer keeping my scripts as neat as possible.
    >
    > For me, explicitely closing a file, for which the only reference is about
    > to disappear through function exiting, would be very similar to using
    > `del' on any variable I happened to use in that function: gross

    overkill...
    >
    > The only reason to call `close' explicitly is when there is a need to

    close
    > prematurely. Absolutely no doubt that such needs exist at times. But
    > closing all the time "just in case" is symptomatic of unsure programming.
    > Or else, it is using Python while still thinking in other languages.
    >
    > --
    > François Pinard http://www.iro.umontreal.ca/~pinard
    >


    you are correct this is so awesome... at least for me it is... now i remove
    a lot of my clutter too :) i just did some tests on windows by trying to
    delete file1 at the command prompt when raw_input is called.

    f = file('file1')
    raw_input('pause')

    ### the file is NOT closed


    file('file1')
    raw_input('pause')

    ### the file IS closed


    f = file('file1')
    del f
    raw_input('pause')

    ### the file IS closed


    def foo():
    f = file('file1')

    foo()
    raw_input('pause')

    ### the file IS closed



    can you explain to me how the file gets closed? i'm sure that garbage
    collection hasn't happed at the point that i call raw_input. it must have
    something to do with the reference count of the file object. does python
    immediately call close for you when the reference count goes to zero? i
    want the dirty details...

    thanks,

    bryan
    Bryan, Jul 25, 2003
    #13
  14. Ben Finney

    Ulrich Petri Guest

    "Francois Pinard" <> schrieb im Newsbeitrag
    news:...
    > [Bryan]
    >
    > > I'm curious to know how others handle the closing of files. [...] I'm
    > > aware that files will automatically be closed when the process exits.

    >
    > For one, I systematically avoid cluttering my code with unneeded `close'.
    > The advantages are simplicity and legibility, both utterly important to

    me.
    >
    > However, I do understand that if I ever have to move a Python script
    > to Jython, I will have to revise my scripts for adding the clutter I am
    > sparing today. I'm quite accepting to do that revision if this occurs.
    > Until then, I prefer keeping my scripts as neat as possible.
    >
    > For me, explicitely closing a file, for which the only reference is about
    > to disappear through function exiting, would be very similar to using
    > `del' on any variable I happened to use in that function: gross

    overkill...
    >
    > The only reason to call `close' explicitly is when there is a need to

    close
    > prematurely. Absolutely no doubt that such needs exist at times. But
    > closing all the time "just in case" is symptomatic of unsure programming.
    > Or else, it is using Python while still thinking in other languages.
    >


    fire up your python and type: "import this" read the output and rethink your
    codint techniques

    Ciao Ulrich
    Ulrich Petri, Jul 25, 2003
    #14
  15. Bengt Richter wrote:

    > 1) Is f.done() really necessary? I.e., doesn't an explicit del f take
    > care of it
    > if the object has been coded with a __del__ method? I.e., the idea
    > was to get
    > the effect of CPython's immediate effective del on ref count going
    > to zero, right?


    It wouldn't if there were circular references at that point. If you're
    going to have some kind of `with' structure that constraints lifetimes,
    I'd think you'd probably want something more concrete than just object
    deletion; you'd want to make sure a "Stop whatever you were doing now"
    method were present and called. But maybe that really depends on the
    primary thing that the `with' construct would be used for.

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ Love is when you wake up in the morning and have a big smile.
    \__/ Anggun
    Erik Max Francis, Jul 25, 2003
    #15
  16. Francois Pinard wrote:
    > For one, I systematically avoid cluttering my code with unneeded `close'.


    What happens if close fails during GC? Will it still raise an
    exception? If so, the exception could happen at an unfortunate place in
    the code.

    Um. Explicit close does raise an exception if it fails, right?

    --
    Hallvard
    Hallvard B Furuseth, Jul 25, 2003
    #16
  17. Ben Finney

    Paul Rubin Guest

    (Bengt Richter) writes:
    > >"done" here is a generic method that gets called on exiting a "with"
    > >block.

    > First reaction == +1, but some questions...
    >
    > 1) Is f.done() really necessary? I.e., doesn't an explicit del f
    > take care of it if the object has been coded with a __del__
    > method? I.e., the idea was to get the effect of CPython's
    > immediate effective del on ref count going to zero, right?


    The ref count might not be zero. Something inside the "with" block
    might make a new reference and leave it around.

    > 2) how about with two files (or other multiple resources)?
    >
    > with f1,f2 = file('f1'), file('f2'):
    > [do stuff with f1 & f2]
    >
    > What is the canonical expansion?


    I think f1.done and f2.done should both get called.

    > Also, what about the attribute version, i.e.,
    >
    > ob.f = file(frob)
    > try:
    > [do stuff with ob.f]
    > finally:
    > del ob.f # calls f.__del__, which calls/does f.close()
    >
    > I.e., ob.f = something could raise an exception (e.g., a read-only property)
    > *after* file(frob) has succeeded. So I guess the easiest would be to limit
    > the left hand side to plain names...


    The assignment should be inside the try. If I had it on the outside
    before, that was an error.
    Paul Rubin, Jul 25, 2003
    #17
  18. On 25 Jul 2003 13:35:49 -0700, Paul Rubin <http://> wrote:

    > (Bengt Richter) writes:
    >> >"done" here is a generic method that gets called on exiting a "with"
    >> >block.

    >> First reaction == +1, but some questions...
    >>
    >> 1) Is f.done() really necessary? I.e., doesn't an explicit del f
    >> take care of it if the object has been coded with a __del__
    >> method? I.e., the idea was to get the effect of CPython's
    >> immediate effective del on ref count going to zero, right?

    >
    >The ref count might not be zero. Something inside the "with" block
    >might make a new reference and leave it around.
    >

    Aha. In that case, is the del just a courtesy default action?

    >> 2) how about with two files (or other multiple resources)?
    >>
    >> with f1,f2 = file('f1'), file('f2'):
    >> [do stuff with f1 & f2]
    >>
    >> What is the canonical expansion?

    >
    >I think f1.done and f2.done should both get called.

    Consistently ;-)

    >
    >> Also, what about the attribute version, i.e.,
    >>
    >> ob.f = file(frob)
    >> try:
    >> [do stuff with ob.f]
    >> finally:
    >> del ob.f # calls f.__del__, which calls/does f.close()
    >>
    >> I.e., ob.f = something could raise an exception (e.g., a read-only property)
    >> *after* file(frob) has succeeded. So I guess the easiest would be to limit
    >> the left hand side to plain names...

    >
    >The assignment should be inside the try. If I had it on the outside
    >before, that was an error.


    Really? Don't you want a failing f = file(frob) exception to skip the finally,
    since f might not even be bound to anything in that case?

    The trouble I was pointing to is having two possible causes for exception, and only
    one of them being of relevance to the file resource.

    Maybe if you wanted to go to the trouble, it could be split something like

    with ob.f = file(frob):
    [do stuff with ob.f]

    becoming

    _tmp = file(frob)
    try:
    ob.f = _tmp
    [ do stuff with ob.f ]
    finally:
    _tmp.done()
    del _tmp
    del ob.f

    Not sure about the order. Plain ob.f.done() would not guarantee a call to _tmp.done(),
    since ob could refuse to produce f, and f could be a wrapper produced during ob.f = _tmp,
    and it might not have a __del__ method etc etc. _tmp is just for some internal temp binding.

    Regards,
    Bengt Richter
    Bengt Richter, Jul 25, 2003
    #18
  19. Hallvard B Furuseth fed this fish to the penguins on Friday 25 July
    2003 12:51 pm:


    > Um. Explicit close does raise an exception if it fails, right?
    >


    [wulfraed@beastie wulfraed]$ python
    Python 2.2 (#1, Nov 5 2002, 15:43:24)
    [GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)] on linux-i386
    Type "help", "copyright", "credits" or "license" for more information.
    >>> f = file("t.t","w")
    >>> f.close()
    >>> f.close()
    >>> f

    <closed file 't.t', mode 'w' at 0x809dc88>
    >>> del f
    >>> f.close()

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    NameError: name 'f' is not defined
    >>>


    Looks like it doesn't care... As long as the file /had/ been opened
    first (doesn't look like the interactive interpreter collected f until
    the del either).


    --
    > ============================================================== <
    > | Wulfraed Dennis Lee Bieber KD6MOG <
    > | Bestiaria Support Staff <
    > ============================================================== <
    > Bestiaria Home Page: http://www.beastie.dm.net/ <
    > Home Page: http://www.dm.net/~wulfraed/ <
    Dennis Lee Bieber, Jul 26, 2003
    #19
  20. Dennis Lee Bieber wrote:

    > Hallvard B Furuseth fed this fish to the penguins on Friday 25 July
    > 2003 12:51 pm:
    >
    > > Um. Explicit close does raise an exception if it fails, right?

    >

    ...
    > Looks like it doesn't care... As long as the file /had/ been
    > opened
    > first (doesn't look like the interactive interpreter collected f until
    > the del either).


    I don't think you've demonstrated that; all you've shown is that builtin
    Python file objects make file closing idempotent. You haven't
    demonstrated a case where there actually is an I/O error that occurs
    when .close gets called. In particular, I _really_ don't know what you
    meant to show with this snippet, as it really has nothing to do with
    files at all:

    > >>> f

    > <closed file 't.t', mode 'w' at 0x809dc88>
    > >>> del f
    > >>> f.close()

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > NameError: name 'f' is not defined


    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ He who knows how to be poor knows everything.
    \__/ Jules Michelet
    Erik Max Francis, Jul 28, 2003
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Denon
    Replies:
    1
    Views:
    563
    Saravana [MVP]
    Nov 14, 2003
  2. Daniel Albisser
    Replies:
    1
    Views:
    1,057
    GaryM
    Apr 7, 2004
  3. Dani
    Replies:
    3
    Views:
    469
  4. Iñaki Baz Castillo
    Replies:
    7
    Views:
    821
    Iñaki Baz Castillo
    Jan 12, 2010
  5. Iulian Ilea
    Replies:
    1
    Views:
    292
    pcx99
    Dec 21, 2006
Loading...

Share This Page