Safely dealing with arbitrary file objects

Discussion in 'Python' started by Steven D'Aprano, Oct 14, 2007.

  1. I have found myself writing functions rather like these:

    def openfile(filename):
    if filename == '-':
    # convention for shell scripts in Unix-land is to use
    # '-' for stdin/stdout for reading/writing.
    outfile = sys.stdout
    if filename == '2-':
    outfile = sys.stderr
    else:
    outfile = file(filename, 'w')
    return outfile

    def closefile(fileobj):
    # don't close standard file objects, or their replacements
    if not fileobj in (sys.stdout, sys.stderr, sys.stdin,
    sys.__stdout__, sys.__stderr__, sys.__stdin__):
    fileobj.close()


    def processfile(filename):
    outfile = openfile(filename)
    try:
    # lots of processing here, which may raise exceptions
    var = "stuff happens"
    outfile.write(var)
    finally:
    closefile(outfile)



    A question:

    I know I'm being paranoid about not closing files I shouldn't close, but
    am I being excessively paranoid, or not paranoid enough?

    Suggestions for improvements welcome; while I'm happy to read suggestions
    using the new with statement, I can't yet rely on having Python 2.5 or
    better so I have to stick to try...finally.


    --
    Steven.
    Steven D'Aprano, Oct 14, 2007
    #1
    1. Advertising

  2. Steven D'Aprano wrote:
    > I have found myself writing functions rather like these:
    >
    > def openfile(filename):
    > if filename == '-':
    > # convention for shell scripts in Unix-land is to use
    > # '-' for stdin/stdout for reading/writing.
    > outfile = sys.stdout
    > if filename == '2-':
    > outfile = sys.stderr
    > else:
    > outfile = file(filename, 'w')
    > return outfile
    >
    > def closefile(fileobj):
    > # don't close standard file objects, or their replacements
    > if not fileobj in (sys.stdout, sys.stderr, sys.stdin,
    > sys.__stdout__, sys.__stderr__, sys.__stdin__):
    > fileobj.close()
    >
    >
    > def processfile(filename):
    > outfile = openfile(filename)
    > try:
    > # lots of processing here, which may raise exceptions
    > var = "stuff happens"
    > outfile.write(var)
    > finally:
    > closefile(outfile)
    >
    >
    >
    > A question:
    >
    > I know I'm being paranoid about not closing files I shouldn't close, but
    > am I being excessively paranoid, or not paranoid enough?
    >
    > Suggestions for improvements welcome; while I'm happy to read suggestions
    > using the new with statement, I can't yet rely on having Python 2.5 or
    > better so I have to stick to try...finally.


    How about a wrapper that passes along all but 'close', so you don't
    have to worry about closing something you means to leave open.
    Something like:

    class DontCloseOutput(file):
    '''subclass file only to allow isinstance checks to work out'''
    def __init__(self, referent):
    self._actual = referent
    self.closed = False

    def close(self):
    self.closed = True

    def write(self, data):
    if self.closed:
    raise IOError('Tried to write closed pseudo-file')
    return self._actual.write(data)


    def writeopen(filename):
    if filename == '-':
    # convention for shell scripts in Unix-land is to use
    # '-' for stdin/stdout for reading/writing.
    return DontCloseOutput(sys.stdout)
    if filename == '2-':
    return DontCloseOutput(sys.stderr)
    else:
    return open(filename, 'w') # BTW, open is the preferred way
    # file was for a short time.

    You of course can do similar things (probably forwarding more message)
    with readopen.

    -Scott David Daniels
    Scott David Daniels, Oct 14, 2007
    #2
    1. Advertising

  3. Steven D'Aprano

    Paul Hankin Guest

    On Oct 14, 5:01 am, Steven D'Aprano <st...@REMOVE-THIS-
    cybersource.com.au> wrote:
    > I have found myself writing functions rather like these:
    >
    > def openfile(filename):
    > if filename == '-':
    > # convention for shell scripts in Unix-land is to use
    > # '-' for stdin/stdout for reading/writing.
    > outfile = sys.stdout
    > if filename == '2-':
    > outfile = sys.stderr
    > else:
    > outfile = file(filename, 'w')
    > return outfile
    >
    > def closefile(fileobj):
    > # don't close standard file objects, or their replacements
    > if not fileobj in (sys.stdout, sys.stderr, sys.stdin,
    > sys.__stdout__, sys.__stderr__, sys.__stdin__):
    > fileobj.close()
    >
    > def processfile(filename):
    > outfile = openfile(filename)
    > try:
    > # lots of processing here, which may raise exceptions
    > var = "stuff happens"
    > outfile.write(var)
    > finally:
    > closefile(outfile)
    >
    > A question:
    >
    > I know I'm being paranoid about not closing files I shouldn't close, but
    > am I being excessively paranoid, or not paranoid enough?
    >
    > Suggestions for improvements welcome; while I'm happy to read suggestions
    > using the new with statement, I can't yet rely on having Python 2.5 or
    > better so I have to stick to try...finally.


    With with:

    import sys

    class leave_open(object):
    def __init__(self, obj):
    self.obj = obj
    def __enter__(self):
    return self.obj
    def __exit__(self, *exc_info):
    pass

    def open_file(name, special={'-':sys.stdout, '2-':sys.stderr}):
    if name in special:
    return leave_open(special[name])
    else:
    return open(name, 'w')

    def process_file(filename):
    with open_file(filename) as outfile:
    outfile.write("stuff happens")


    Without with:

    import sys

    def open_file(name, special={'-':sys.stdout, '2-':sys.stderr}):
    if name in special:
    return False, special[name]
    return True, open(name, 'w')

    def process_file(filename):
    should_close, outfile = open_file(filename)
    try:
    outfile.write('stuff happens')
    finally:
    if should_close:
    outfile.close()

    --
    Paul Hankin
    Paul Hankin, Oct 14, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Honestmath
    Replies:
    5
    Views:
    553
    Honestmath
    Dec 13, 2004
  2. Steven D'Aprano

    Safely renaming a file without overwriting

    Steven D'Aprano, Oct 28, 2006, in forum: Python
    Replies:
    11
    Views:
    461
    Wolfgang Draxinger
    Oct 29, 2006
  3. Steven D'Aprano
    Replies:
    3
    Views:
    1,823
    Jeffrey Straszheim
    Dec 6, 2008
  4. Verweij, Arjen
    Replies:
    0
    Views:
    203
    Verweij, Arjen
    Nov 6, 2009
  5. Replies:
    3
    Views:
    519
    Steven D'Aprano
    Jul 1, 2011
Loading...

Share This Page