try/except KeyError vs "if name in ..."

Discussion in 'Python' started by Manuel Pégourié-Gonnard, Oct 6, 2012.

  1. Hi,

    I was looking at the example found here [1] which begins with:

    [1] http://docs.python.org/py3k/library/imp.html#examples

    def __import__(name, globals=None, locals=None, fromlist=None):
    # Fast path: see if the module has already been imported.
    try:
    return sys.modules[name]
    except KeyError:
    pass

    I was wondering if the formulation

    if name in sys.modules:
    return sys.modules[name]

    would be equivalent. IOW, is using try/except here only a matter of
    style or a necessity?

    I'm suspecting that maybe, in multithreaded environments, the second
    option may be subject to a race condition, if another thread removes
    name frome sys.modules between the if and the return, but as I'm not
    very familiar (yet) with Python threads, I'm not sure it is a real
    concern here.

    And maybe there are other reasons I'm completely missing for prefering
    EAFP over LBYL here?

    Thanks in advance for your comments.

    --
    Manuel Pégourié-Gonnard - http://people.math.jussieu.fr/~mpg/
    Manuel Pégourié-Gonnard, Oct 6, 2012
    #1
    1. Advertising

  2. On Sat, 06 Oct 2012 08:27:25 +0200, Manuel Pégourié-Gonnard wrote:

    > Hi,
    >
    > I was looking at the example found here [1] which begins with:
    >
    > [1] http://docs.python.org/py3k/library/imp.html#examples
    >
    > def __import__(name, globals=None, locals=None, fromlist=None):
    > # Fast path: see if the module has already been imported. try:
    > return sys.modules[name]
    > except KeyError:
    > pass
    >
    > I was wondering if the formulation
    >
    > if name in sys.modules:
    > return sys.modules[name]
    >
    > would be equivalent. IOW, is using try/except here only a matter of
    > style or a necessity?


    Mostly style, but not entirely.

    If you expect that most of the time the module will be found, the
    try...except version will be faster. If you expect that most of the time
    the module will not be found, the "if name in" version will be faster.

    But see also:

    > I'm suspecting that maybe, in multithreaded environments, the second
    > option may be subject to a race condition, if another thread removes
    > name frome sys.modules between the if and the return, but as I'm not
    > very familiar (yet) with Python threads, I'm not sure it is a real
    > concern here.


    In practice, no, it would be very unusual for another thread to remove
    the name from sys.modules. So don't do that :)

    But in principle, yes, it is a race condition and yes it is a (small)
    concern. Since it is so easy to avoid even this tiny risk, why not use
    the try...except version and avoid it completely?



    --
    Steven
    Steven D'Aprano, Oct 6, 2012
    #2
    1. Advertising

  3. Manuel Pégourié-Gonnard <> wrote:

    >Hi,
    >I was looking at the example found here [1] which begins with:
    >
    >[1] http://docs.python.org/py3k/library/imp.html#examples
    >
    >def __import__(name, globals=None, locals=None, fromlist=None):
    > # Fast path: see if the module has already been imported.
    > try:
    > return sys.modules[name]
    > except KeyError:
    > pass
    >
    >I was wondering if the formulation
    >
    > if name in sys.modules:
    > return sys.modules[name]
    >
    >would be equivalent. IOW, is using try/except here only a matter of
    >style or a necessity?


    Somewhere I read a text regarding 'try:' versus 'if'. If you take the
    probabitility into consideration, how many times the test will fail or
    succeed, there are two possibilities:
    - If the test will fail only on very rare occasions: Use 'try:'. When
    the statement(s) in the try-path succeed, the try:/except: construct
    will not consume additional execution time (not even for the test). The
    statements will just be executed as they are. Only in the (rare) case of
    failure, the exception handling will take additional execution time (a
    considerably big amount). The fact, that in python it is not named
    'error handling', but 'exception handling' reflects this. The failure
    should be the exception, also in matters of occurrence.
    - If the relation between success and failure is not predictable, or if
    the case of failure will be frequent, use 'if'. The failure of a 'try:'
    gives you a penalty in form of consumption of a high amount of execution
    time. So, in these constellations, it is better to accept the relatively
    small amount of execution time taken by the explicit test.

    Obviously, you can use 'try:' only, if there is the possibility to
    produce an exception on failure. In the other cases you must use the
    explicit test by 'if'.


    >I'm suspecting that maybe, in multithreaded environments, the second
    >option may be subject to a race condition, if another thread removes
    >name frome sys.modules between the if and the return, but as I'm not
    >very familiar (yet) with Python threads, I'm not sure it is a real
    >concern here.


    Your idea sounds reasonable, but I also am not yet familiar with threads.


    >And maybe there are other reasons I'm completely missing for prefering
    >EAFP over LBYL here?


    One reason might be what I described above.



    Best regards,

    Günther
    Günther Dietrich, Oct 6, 2012
    #3
  4. Steven D'Aprano scripsit :

    > If you expect that most of the time the module will be found, the
    > try...except version will be faster. If you expect that most of the time
    > the module will not be found, the "if name in" version will be faster.
    >

    Ok.

    In the particular case of __import__, I guess speed is not crucial since
    I doubt import often happen within a program's inner loop. But I'll
    remember that point for other cases anyway.

    >> I'm suspecting that maybe, in multithreaded environments, the second
    >> option may be subject to a race condition, if another thread removes
    >> name frome sys.modules between the if and the return, but as I'm not
    >> very familiar (yet) with Python threads, I'm not sure it is a real
    >> concern here.

    >
    > In practice, no, it would be very unusual for another thread to remove
    > the name from sys.modules. So don't do that :)
    >

    That wasn't my intention. But sometimes other people may be "creative" :)

    > But in principle, yes, it is a race condition and yes it is a (small)
    > concern. Since it is so easy to avoid even this tiny risk, why not use
    > the try...except version and avoid it completely?
    >

    Ok.

    Thanks for your explanations.

    --
    Manuel Pégourié-Gonnard - http://people.math.jussieu.fr/~mpg/
    Manuel Pégourié-Gonnard, Oct 6, 2012
    #4
  5. Günther Dietrich scripsit :

    > Somewhere I read a text regarding 'try:' versus 'if'. If you take the
    > probabitility into consideration, how many times the test will fail or
    > succeed, there are two possibilities: [...]


    Ok, thanks for the details!

    --
    Manuel Pégourié-Gonnard - http://people.math.jussieu.fr/~mpg/
    Manuel Pégourié-Gonnard, Oct 6, 2012
    #5
  6. Manuel Pégourié-Gonnard

    Terry Reedy Guest

    On 10/6/2012 7:36 AM, Dave Angel wrote:

    > The distinction in performance between the success and failure modes of
    > the try/catch isn't nearly as large as one of the other responses might
    > lead you to believe. For example, a for loop generally terminates with
    > a raise (of StopIteration exception), and that doesn't convince us to
    > replace it with a while loop.


    For statement generally loop many times, up to millions of times,
    without an exception being raised, whereas while statements test the
    condition each time around the loop. So the rule 'if failure is rare
    (less than 10-20%) use try', applies here. For if/them versus
    try/except, I don't worry too much about it.

    --
    Terry Jan Reedy
    Terry Reedy, Oct 6, 2012
    #6
  7. On Sunday, 7 October 2012 01:12:56 UTC+5:30, Terry Reedy wrote:
    > On 10/6/2012 7:36 AM, Dave Angel wrote:
    >
    >
    >
    > > The distinction in performance between the success and failure modes of

    >
    > > the try/catch isn't nearly as large as one of the other responses might

    >
    > > lead you to believe. For example, a for loop generally terminates with

    >
    > > a raise (of StopIteration exception), and that doesn't convince us to

    >
    > > replace it with a while loop.

    >
    >
    >
    > For statement generally loop many times, up to millions of times,
    >
    > without an exception being raised, whereas while statements test the
    >
    > condition each time around the loop. So the rule 'if failure is rare
    >
    > (less than 10-20%) use try', applies here. For if/them versus
    >
    > try/except, I don't worry too much about it.
    >
    >
    >
    > --
    >
    > Terry Jan Reedy


    I use try and except when I need to raise exceptions e.g.:

    try:
    value = vm.variables[name]
    except KeyError:
    raise NameError("variable name not defined in VM's variables")
    Ramchandra Apte, Oct 7, 2012
    #7
  8. On Sunday, 7 October 2012 01:12:56 UTC+5:30, Terry Reedy wrote:
    > On 10/6/2012 7:36 AM, Dave Angel wrote:
    >
    >
    >
    > > The distinction in performance between the success and failure modes of

    >
    > > the try/catch isn't nearly as large as one of the other responses might

    >
    > > lead you to believe. For example, a for loop generally terminates with

    >
    > > a raise (of StopIteration exception), and that doesn't convince us to

    >
    > > replace it with a while loop.

    >
    >
    >
    > For statement generally loop many times, up to millions of times,
    >
    > without an exception being raised, whereas while statements test the
    >
    > condition each time around the loop. So the rule 'if failure is rare
    >
    > (less than 10-20%) use try', applies here. For if/them versus
    >
    > try/except, I don't worry too much about it.
    >
    >
    >
    > --
    >
    > Terry Jan Reedy


    I use try and except when I need to raise exceptions e.g.:

    try:
    value = vm.variables[name]
    except KeyError:
    raise NameError("variable name not defined in VM's variables")
    Ramchandra Apte, Oct 7, 2012
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. John Salerno
    Replies:
    20
    Views:
    843
    John Salerno
    Aug 11, 2006
  2. Fabio Z Tessitore

    who is simpler? try/except/else or try/except

    Fabio Z Tessitore, Aug 12, 2007, in forum: Python
    Replies:
    5
    Views:
    367
  3. David House

    try -> except -> else -> except?

    David House, Jul 6, 2009, in forum: Python
    Replies:
    2
    Views:
    332
    Bruno Desthuilliers
    Jul 6, 2009
  4. MRAB
    Replies:
    4
    Views:
    306
  5. gert
    Replies:
    2
    Views:
    593
Loading...

Share This Page