Dealing with exceptions

Discussion in 'Python' started by bvdp, Mar 2, 2013.

  1. bvdp

    bvdp Guest

    Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain when Icreate try blocks.

    For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:

    try:
    shutil.copy(s, os.path.join(usbpath, songname))
    except ...

    now, I need to figure out just what exceptions to handle. Let's see:

    IOError that means that the disk is full or otherwise buggered. Better dump out of the loop.

    But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalid when writing to a FAT32 filesystem). And, so what exception is that? Withoutactually creating the error, I can't figure it out.

    In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.

    Maybe I'm just venting about FAT32 filesystems :)
     
    bvdp, Mar 2, 2013
    #1
    1. Advertising

  2. bvdp

    Kwpolska Guest

    On Sat, Mar 2, 2013 at 6:40 PM, bvdp <> wrote:
    > Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain whenI create try blocks.
    >
    > For example, I'm writing a little program do copy specific files to a USBstick. To do the actual copy I'm using:
    >
    > try:
    > shutil.copy(s, os.path.join(usbpath, songname))
    > except ...
    >
    > now, I need to figure out just what exceptions to handle. Let's see:
    >
    > IOError that means that the disk is full or otherwise buggered. Betterdump out of the loop.
    >
    > But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalidwhen writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out.
    >
    > In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.
    >
    > Maybe I'm just venting about FAT32 filesystems :)
    >
    > --
    > http://mail.python.org/mailman/listinfo/python-list


    IOError and OSError should cover all copy problems, I think.

    Also, you can do `except:` for a catch-all, but it is discouraged
    unless you have REALLY good reasons to do this. And, most of the
    time, you don’t.

    --
    Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16
    stop html mail | always bottom-post
    http://asciiribbon.org | http://caliburn.nl/topposting.html
     
    Kwpolska, Mar 2, 2013
    #2
    1. Advertising

  3. bvdp

    Ian Kelly Guest

    On Sat, Mar 2, 2013 at 10:40 AM, bvdp <> wrote:
    > Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain whenI create try blocks.
    >
    > For example, I'm writing a little program do copy specific files to a USBstick. To do the actual copy I'm using:
    >
    > try:
    > shutil.copy(s, os.path.join(usbpath, songname))
    > except ...
    >
    > now, I need to figure out just what exceptions to handle. Let's see:
    >
    > IOError that means that the disk is full or otherwise buggered. Betterdump out of the loop.
    >
    > But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalidwhen writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out.


    OSError. In Python 3, I expect it would more specifically be a
    FileNotFoundError, which is a subclass of OSError.

    > In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.


    OSError will cover a wide swath of possible exceptions here.
     
    Ian Kelly, Mar 2, 2013
    #3
  4. bvdp

    Ian Kelly Guest

    On Sat, Mar 2, 2013 at 10:52 AM, Kwpolska <> wrote:
    > IOError and OSError should cover all copy problems, I think.


    And it may be worth pointing out here that as of Python 3.3, IOError
    is just a synonym for OSError.
     
    Ian Kelly, Mar 2, 2013
    #4
  5. On Sun, Mar 3, 2013 at 4:40 AM, bvdp <> wrote:
    > For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:
    >
    > try:
    > shutil.copy(s, os.path.join(usbpath, songname))
    > except ...
    >
    > now, I need to figure out just what exceptions to handle.


    Here's a bit of a left-field thought: Maybe none of them.

    What are you actually doing when you get an exception? Can you
    plausibly recover? If not - that is, if you're going to abort the
    whole operation anyway - then save yourself the trouble of writing the
    try/catch, and just let the exception propagate up (to the console, if
    nowhere else).

    On the other hand, if you want to simply report the error and continue
    on (meaning you get as many songs copied as possible), then do what
    others have recommended and catch OSError.

    ChrisA
     
    Chris Angelico, Mar 2, 2013
    #5
  6. On 02/03/2013 17:58, Ian Kelly wrote:
    > On Sat, Mar 2, 2013 at 10:40 AM, bvdp <> wrote:
    >> Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain when I create try blocks.
    >>
    >> For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using:
    >>
    >> try:
    >> shutil.copy(s, os.path.join(usbpath, songname))
    >> except ...
    >>
    >> now, I need to figure out just what exceptions to handle. Let's see:
    >>
    >> IOError that means that the disk is full or otherwise buggered. Better dump out of the loop.
    >>
    >> But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalid when writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out.

    >
    > OSError. In Python 3, I expect it would more specifically be a
    > FileNotFoundError, which is a subclass of OSError.


    This wasn't introduced until Python 3.3 see
    http://docs.python.org/3/whatsnew/3.3.html#pep-3151-reworking-the-os-and-io-exception-hierarchy

    >
    >> In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands.

    >
    > OSError will cover a wide swath of possible exceptions here.
    >



    --
    Cheers.

    Mark Lawrence
     
    Mark Lawrence, Mar 2, 2013
    #6
  7. On Sat, Mar 2, 2013 at 1:21 PM, Chris Angelico <> wrote:
    >> now, I need to figure out just what exceptions to handle.

    >
    > Here's a bit of a left-field thought: Maybe none of them.
    >
    > What are you actually doing when you get an exception? Can you
    > plausibly recover? If not - that is, if you're going to abort the
    > whole operation anyway - then save yourself the trouble of writing the
    > try/catch, and just let the exception propagate up (to the console, if
    > nowhere else).


    He can't know if he should handle the errors if he doesn't know what
    those errors are. Thus the question.

    -- Devin
     
    Devin Jeanpierre, Mar 2, 2013
    #7
  8. bvdp

    bvdp Guest


    >
    > IOError and OSError should cover all copy problems, I think.


    How do you know that? I can figure it out as well by running the program, but I'd like to make the determination of what to catch when I'm writing the code.
     
    bvdp, Mar 2, 2013
    #8
  9. bvdp

    bvdp Guest


    >
    > IOError and OSError should cover all copy problems, I think.


    How do you know that? I can figure it out as well by running the program, but I'd like to make the determination of what to catch when I'm writing the code.
     
    bvdp, Mar 2, 2013
    #9
  10. bvdp

    bvdp Guest


    >
    > Here's a bit of a left-field thought: Maybe none of them.
    >


    Not far left at all :)
    >
    > What are you actually doing when you get an exception? Can you
    >
    > plausibly recover? If not - that is, if you're going to abort the
    >
    > whole operation anyway - then save yourself the trouble of writing the
    >
    > try/catch, and just let the exception propagate up (to the console, if
    >
    > nowhere else).
    >


    My first cut of the program did exactly that. Just abort the whole thing, figuring that the disk was full or buggered.

    What I ran into was that half way through the process I ended up with a filename which the FAT32 didn't like. So, my brain-dead idea was to catch those (and ignore them) and continue on. But then I have to distinguish betweena bad filename and "real" errors.

    But, you are probably correct if you say I should check the filename first :) Is there a module for that?
     
    bvdp, Mar 2, 2013
    #10
  11. bvdp

    bvdp Guest


    >
    > Here's a bit of a left-field thought: Maybe none of them.
    >


    Not far left at all :)
    >
    > What are you actually doing when you get an exception? Can you
    >
    > plausibly recover? If not - that is, if you're going to abort the
    >
    > whole operation anyway - then save yourself the trouble of writing the
    >
    > try/catch, and just let the exception propagate up (to the console, if
    >
    > nowhere else).
    >


    My first cut of the program did exactly that. Just abort the whole thing, figuring that the disk was full or buggered.

    What I ran into was that half way through the process I ended up with a filename which the FAT32 didn't like. So, my brain-dead idea was to catch those (and ignore them) and continue on. But then I have to distinguish betweena bad filename and "real" errors.

    But, you are probably correct if you say I should check the filename first :) Is there a module for that?
     
    bvdp, Mar 2, 2013
    #11
  12. bvdp

    Rick Johnson Guest

    On Saturday, March 2, 2013 11:40:11 AM UTC-6, bvdp wrote:
    > Every time I write a program with exception handling (and
    > I suppose that includes just about every program I write!)
    > I need to scratch my brain when I create try blocks.
    >
    > For example, I'm writing a little program do copy specific
    > files to a USB stick. To do the actual copy I'm using:
    >
    > try:
    > shutil.copy(s, os.path.join(usbpath, songname))
    > except ...
    >
    > now, I need to figure out just what exceptions to handle.
    > Let's see:
    >
    > IOError that means that the disk is full or otherwise
    > buggered. Better dump out of the loop.
    >
    > But, I know there can be other errors as well. Doing some
    > tests, I know that certain filenames are invalid (I think
    > a "?" or unicode char is invalid when writing to a FAT32
    > filesystem). And, so what exception is that? Without
    > actually creating the error, I can't figure it out.


    Well how can you expect an error to be thrown without creating the scenario that will throw one? *wink*

    > In this case, I can run the program an number of times and
    > parse out the errors and write code to catch various
    > things. But, I think I'm missing something completely.
    > Guess what I'm looking for is a list of possible
    > (probable?) errors for the shutil.copy() command. And, in
    > a much bigger manual, for most other commands.


    No. What you are doing is *misunderstanding* that well written methods must follow the values of:

    "doing one thing and doing it well"

    In the case of "shutil.copy", that "one thing" is "coping files"; but that "one thing" does NOT include:

    * resolving file paths
    * validating file path chars on an OS by OS basis
    * ensuring disc space is adequate
    * etc...

    Methods that take inputs, like in this case: "src" and "dst" file paths, should not be responsible for the validity of the inputs. It is the responsibility of the caller to validate these inputs PRIOR to injecting them into the method.

    But even *IF* a convincing argument could be made for methods to validate inputs, then at what point do we draw the line? How many foolish samples of the possible permutation set should we consider? Should we inform the user of such foolish usage as:

    shutil.copy(src=1, dst=2, follow_symlinks=isinstance)
    shutil.copy(src='cat', dst='hat', follow_symlinks=list)

    Now whilst these example are quite absurd, they are in-fact possibilities that must be considered *IF* we intend to follow your "wish" until it's logical conclusion.
     
    Rick Johnson, Mar 2, 2013
    #12
  13. bvdp

    Terry Reedy Guest

    On 3/2/2013 12:40 PM, bvdp wrote:

    > But, I know there can be other errors as well. Doing some tests, I
    > know that certain filenames are invalid (I think a "?" or unicode
    > char is invalid when writing to a FAT32 filesystem). And, so what
    > exception is that? Without actually creating the error, I can't
    > figure it out.


    So use the interactive interpreter (or idle, or ipython) and create the
    error. You should always have it open when editing. Using less time that
    it took you to write the above. 3.3, win7, (idle)

    >>> open('sdjhfjshdfkjsh')

    Traceback (most recent call last):
    File "<pyshell#2>", line 1, in <module>
    open('sdjhfjshdfkjsh')
    FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'

    Now, does shutil pass on FileNotFoundError? I will let you experiment.

    There are error conditions that are hard to generate, but a bad file
    name is not one of them.

    --
    Terry Jan Reedy
     
    Terry Reedy, Mar 2, 2013
    #13
  14. On Sun, Mar 3, 2013 at 8:23 AM, Terry Reedy <> wrote:
    >>>> open('sdjhfjshdfkjsh')

    > Traceback (most recent call last):
    > File "<pyshell#2>", line 1, in <module>
    > open('sdjhfjshdfkjsh')
    > FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'
    >
    > Now, does shutil pass on FileNotFoundError? I will let you experiment.
    >
    > There are error conditions that are hard to generate, but a bad file name is
    > not one of them.


    That's actually a perfectly valid file name, but one that doesn't
    happen to have a corresponding file. However, the same technique will
    work with the OP's description of "a filename which the FAT32 didn't
    like" too.

    ChrisA
     
    Chris Angelico, Mar 2, 2013
    #14
  15. bvdp

    Terry Reedy Guest

    On 3/2/2013 5:16 PM, Chris Angelico wrote:
    > On Sun, Mar 3, 2013 at 8:23 AM, Terry Reedy <> wrote:
    >>>>> open('sdjhfjshdfkjsh')

    >> Traceback (most recent call last):
    >> File "<pyshell#2>", line 1, in <module>
    >> open('sdjhfjshdfkjsh')
    >> FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'
    >>
    >> Now, does shutil pass on FileNotFoundError? I will let you experiment.
    >>
    >> There are error conditions that are hard to generate, but a bad file name is
    >> not one of them.

    >
    > That's actually a perfectly valid file name, but one that doesn't
    > happen to have a corresponding file. However, the same technique will
    > work with the OP's description of "a filename which the FAT32 didn't
    > like" too.


    Yeah, that gives a different error and message.
    >>> open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')

    Traceback (most recent call last):
    File "<pyshell#3>", line 1, in <module>
    open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')
    OSError: [Errno 22] Invalid argument: 'a~`!@#$%^&*()_-+={[}]|\\<,>.?/'


    --
    Terry Jan Reedy
     
    Terry Reedy, Mar 2, 2013
    #15
  16. On Sun, Mar 3, 2013 at 10:08 AM, Terry Reedy <> wrote:
    > On 3/2/2013 5:16 PM, Chris Angelico wrote:
    >>
    >> On Sun, Mar 3, 2013 at 8:23 AM, Terry Reedy <> wrote:
    >>>>>>
    >>>>>> open('sdjhfjshdfkjsh')
    >>>
    >>> Traceback (most recent call last):
    >>> File "<pyshell#2>", line 1, in <module>
    >>> open('sdjhfjshdfkjsh')
    >>> FileNotFoundError: [Errno 2] No such file or directory: 'sdjhfjshdfkjsh'
    >>>
    >>> Now, does shutil pass on FileNotFoundError? I will let you experiment.
    >>>
    >>> There are error conditions that are hard to generate, but a bad file name
    >>> is
    >>> not one of them.

    >>
    >>
    >> That's actually a perfectly valid file name, but one that doesn't
    >> happen to have a corresponding file. However, the same technique will
    >> work with the OP's description of "a filename which the FAT32 didn't
    >> like" too.

    >
    >
    > Yeah, that gives a different error and message.
    >>>> open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')

    >
    > Traceback (most recent call last):
    > File "<pyshell#3>", line 1, in <module>
    > open('a~`!@#$%^&*()_-+={[}]|\<,>.?/')
    > OSError: [Errno 22] Invalid argument: 'a~`!@#$%^&*()_-+={[}]|\\<,>.?/'


    That should be:

    OSError: Profanity not permitted on respectable file systems

    ChrisA
     
    Chris Angelico, Mar 2, 2013
    #16
  17. On Sat, 02 Mar 2013 18:52:19 +0100, Kwpolska wrote:

    > Also, you can do `except:` for a catch-all, but it is discouraged unless
    > you have REALLY good reasons to do this. And, most of the time, you
    > don’t.



    `except Exception` is to be much preferred over a bare except. It
    excludes KeyboardInterruptError (the user hits Ctrl-C) and SystemExit,
    which you normally either want to let through, or handle separately.


    But yes, in general, only catch the minimum you *know* you need to catch
    and can deal with. Anything else is a bug in your code that needs to be
    fixed, and you can't fix it if you never see the exception.



    --
    Steven
     
    Steven D'Aprano, Mar 3, 2013
    #17
  18. On Sun, Mar 3, 2013 at 11:41 AM, Steven D'Aprano
    <> wrote:
    > But yes, in general, only catch the minimum you *know* you need to catch
    > and can deal with. Anything else is a bug in your code that needs to be
    > fixed, and you can't fix it if you never see the exception.


    With the exception (if you'll excuse the expression) of "framework"
    systems, where there's a distinct separation between "inside" and
    "outside". Often then, the "outside" will catch any exception thrown
    by the "inside" and deal with it in some generic way (for instance, a
    web server might log the details and return HTTP 500 to the client,
    then go back and handle the next request). Effectively, this is doing
    the job of the top-level exception handler: log the exception (to the
    console) and terminate.

    ChrisA
     
    Chris Angelico, Mar 3, 2013
    #18
  19. On Sat, 02 Mar 2013 11:35:13 -0800, bvdp wrote:


    >> IOError and OSError should cover all copy problems, I think.

    >
    > How do you know that? I can figure it out as well by running the
    > program, but I'd like to make the determination of what to catch when
    > I'm writing the code.



    In my experience, I would say:

    20% by reading the documentation, 20% by experience, and 60% by
    experimentation at the interactive interpreter.

    Generally if you read the docs, you will get some idea of the exceptions
    you can expect to get. E.g. the docs for open claim:

    "Raise IOError upon failure."

    (call "help(open)" in the interactive interpreter).

    Experience and experimentation come into it in the (unfortunately very
    common case) where the docs don't describe the exceptions you can expect.

    For the specific case of IOError and OSError, another very useful skill
    is googling for the specific errno you can test for:

    try:
    something()
    except IOError as e:
    if e.errno == whatever:
    do_this()
    else:
    raise


    In Python 3.3, this becomes much nicer with individual exceptions for the
    most common errnos.


    --
    Steven
     
    Steven D'Aprano, Mar 3, 2013
    #19
  20. bvdp

    Nobody Guest

    On Sat, 02 Mar 2013 18:52:19 +0100, Kwpolska wrote:

    > Also, you can do `except:` for a catch-all, but it is discouraged unless
    > you have REALLY good reasons to do this. And, most of the time, you
    > don’t.


    Most of the time you probably want to catch either Exception (which
    excludes GeneratorExit, KeyboardInterrupt and SystemExit) or StandardError
    (which excludes the above pluse warnings and StopIteration).

    Only in specific circumstances can you reasonably go finer than that,
    partly due to the set of exceptions a method may throw seldom being
    documented, and partly due to duck typing; a parameter which is intended
    to be a file object could realistically be any object which supports the
    appropriate methods (e.g. .read()), and there's no guarantee that those
    methods will have the same set of possible exceptions as a real file
    object.
     
    Nobody, Mar 3, 2013
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ahmed Moustafa
    Replies:
    5
    Views:
    30,041
    Chris Smith
    Jul 14, 2004
  2. Paul Miller
    Replies:
    3
    Views:
    1,024
    Alex Martelli
    Nov 12, 2003
  3. AndyColeman
    Replies:
    1
    Views:
    811
    Chris Uppal
    Nov 28, 2006
  4. Aaron W. LaFramboise
    Replies:
    4
    Views:
    351
  5. Replies:
    3
    Views:
    614
    Sherm Pendley
    Apr 16, 2007
Loading...

Share This Page