Is there a more elegant way to handle determing fail status?

Discussion in 'Python' started by J, Jan 15, 2013.

  1. J

    J Guest

    Ok, so I have a diagnostic tool, written by someone else. That tool
    runs a series of small tests defined by the user and can simplified
    summary output that can be one of the following:

    FAILED_CRITICAL
    FAILED_HIGH
    FAILED_MEDIUM
    FAILED_LOW
    PASSED

    I also have a wrapper script I wrote to run these tests, summarize the
    results of all tests aggregated and then fail based on a particular
    fail level.

    The idea is that if I run 3 tests with the diagnostic tool and it
    tells me the following:

    testA: PASSED
    testB: FAILED_MEDIUM
    testC: PASSED

    AND I told the wrapper to only fail on HIGH or above, the wrapper will
    tell me that I had a medium failure, but the wrapper will still exit
    with a 0 (success)

    if I get the same results as above, but tell the wrapper to fail on
    LOW, then it will tell me I had that medium failure, but the wrapper
    will exit with a 1 (failure).

    The problem is that my exit determination looks like this:

    if fail_priority == fail_levels['FAILED_CRITICAL']:
    if critical_fails:
    return 1
    if fail_priority == fail_levels['FAILED_HIGH']:
    if critical_fails or high_fails:
    return 1
    if fail_priority == fail_levels['FAILED_MEDIUM']:
    if critical_fails or high_fails or medium_fails:
    return 1
    if fail_priority == fail_levels['FAILED_LOW']:
    if critical_fails or high_fails or medium_fails or low_fails:
    return 1

    return 0

    So, to explain the above... the fail level can be set by the user when
    running the wrapper using -f (or it defaults to 'high')
    the wrapper assigns a number to each level using this:

    # Set correct fail level
    args.fail_level = 'FAILED_%s' % args.fail_level.upper()

    # Get our failure priority and create the priority values
    fail_levels = {'FAILED_CRITICAL':4,
    'FAILED_HIGH':3,
    'FAILED_MEDIUM':2,
    'FAILED_LOW':1}
    fail_priority = fail_levels[args.fail_level]

    the variables critical_fails, high_fails, medium_fails, low_fails are
    all counters that are etiher None, or the number of tests that were
    failed.

    So using this output from the diagnostic tool:

    testA: PASSED
    testB: FAILED_HIGH
    testC: PASSED
    testD: FAILED_MEDIUM
    testE: PASSED

    critical_fails would be None
    high_fails would be 1
    medium_fails would be 1
    low_fails would be None.

    The exit code determination above works, but it just feels inelegant.
    It feels like there's a better way of implementing that, but I can't
    come up with one that still honors the fail level properly (e.g. other
    solutions will fail on medium, but won't fail properly on medium OR
    higher).

    I can provide the full script if necessary, if the above isn't enough
    to point me in a direction that has a better way of doing this...

    Thanks for looking,

    Jeff
    J, Jan 15, 2013
    #1
    1. Advertising

  2. J

    donarb Guest

    On Tuesday, January 15, 2013 3:24:44 PM UTC-8, J wrote:
    > Ok, so I have a diagnostic tool, written by someone else. That tool
    >
    > runs a series of small tests defined by the user and can simplified
    >
    > summary output that can be one of the following:
    >
    >
    >
    > FAILED_CRITICAL
    >
    > FAILED_HIGH
    >
    > FAILED_MEDIUM
    >
    > FAILED_LOW
    >
    > PASSED
    >
    >
    >
    > I also have a wrapper script I wrote to run these tests, summarize the
    >
    > results of all tests aggregated and then fail based on a particular
    >
    > fail level.
    >
    >
    >
    > The idea is that if I run 3 tests with the diagnostic tool and it
    >
    > tells me the following:
    >
    >
    >
    > testA: PASSED
    >
    > testB: FAILED_MEDIUM
    >
    > testC: PASSED
    >
    >
    >
    > AND I told the wrapper to only fail on HIGH or above, the wrapper will
    >
    > tell me that I had a medium failure, but the wrapper will still exit
    >
    > with a 0 (success)
    >
    >
    >
    > if I get the same results as above, but tell the wrapper to fail on
    >
    > LOW, then it will tell me I had that medium failure, but the wrapper
    >
    > will exit with a 1 (failure).
    >
    >
    >
    > The problem is that my exit determination looks like this:
    >
    >
    >
    > if fail_priority == fail_levels['FAILED_CRITICAL']:
    >
    > if critical_fails:
    >
    > return 1
    >
    > if fail_priority == fail_levels['FAILED_HIGH']:
    >
    > if critical_fails or high_fails:
    >
    > return 1
    >
    > if fail_priority == fail_levels['FAILED_MEDIUM']:
    >
    > if critical_fails or high_fails or medium_fails:
    >
    > return 1
    >
    > if fail_priority == fail_levels['FAILED_LOW']:
    >
    > if critical_fails or high_fails or medium_fails or low_fails:
    >
    > return 1
    >
    >
    >
    > return 0
    >
    >
    >
    > So, to explain the above... the fail level can be set by the user when
    >
    > running the wrapper using -f (or it defaults to 'high')
    >
    > the wrapper assigns a number to each level using this:
    >
    >
    >
    > # Set correct fail level
    >
    > args.fail_level = 'FAILED_%s' % args.fail_level.upper()
    >
    >
    >
    > # Get our failure priority and create the priority values
    >
    > fail_levels = {'FAILED_CRITICAL':4,
    >
    > 'FAILED_HIGH':3,
    >
    > 'FAILED_MEDIUM':2,
    >
    > 'FAILED_LOW':1}
    >
    > fail_priority = fail_levels[args.fail_level]
    >
    >
    >
    > the variables critical_fails, high_fails, medium_fails, low_fails are
    >
    > all counters that are etiher None, or the number of tests that were
    >
    > failed.
    >
    >
    >
    > So using this output from the diagnostic tool:
    >
    >
    >
    > testA: PASSED
    >
    > testB: FAILED_HIGH
    >
    > testC: PASSED
    >
    > testD: FAILED_MEDIUM
    >
    > testE: PASSED
    >
    >
    >
    > critical_fails would be None
    >
    > high_fails would be 1
    >
    > medium_fails would be 1
    >
    > low_fails would be None.
    >
    >
    >
    > The exit code determination above works, but it just feels inelegant.
    >
    > It feels like there's a better way of implementing that, but I can't
    >
    > come up with one that still honors the fail level properly (e.g. other
    >
    > solutions will fail on medium, but won't fail properly on medium OR
    >
    > higher).
    >
    >
    >
    > I can provide the full script if necessary, if the above isn't enough
    >
    > to point me in a direction that has a better way of doing this...
    >
    >
    >
    > Thanks for looking,
    >
    >
    >
    > Jeff


    My back of the envelope coding would do it this way. Use an array of fail_counters, with PASSED as the first element all the way up to FAILED_CRITICALas the last element. Then use a simple loop starting from index fail_priority to the end of the list looking for errors. Like this:

    # Array of fail counters
    fail_counters = [
    0, # PASSED
    0, # LOW
    0, # MEDIUM
    0, # HIGH
    0 # CRITICAL
    ]

    .... run tests, accumulate error counts in fail_counters

    for i in range(fail_priority, len(fail_counters)):
    if fail_counters:
    return 1
    return 0
    donarb, Jan 15, 2013
    #2
    1. Advertising

  3. J

    donarb Guest

    On Tuesday, January 15, 2013 3:24:44 PM UTC-8, J wrote:
    > Ok, so I have a diagnostic tool, written by someone else. That tool
    >
    > runs a series of small tests defined by the user and can simplified
    >
    > summary output that can be one of the following:
    >
    >
    >
    > FAILED_CRITICAL
    >
    > FAILED_HIGH
    >
    > FAILED_MEDIUM
    >
    > FAILED_LOW
    >
    > PASSED
    >
    >
    >
    > I also have a wrapper script I wrote to run these tests, summarize the
    >
    > results of all tests aggregated and then fail based on a particular
    >
    > fail level.
    >
    >
    >
    > The idea is that if I run 3 tests with the diagnostic tool and it
    >
    > tells me the following:
    >
    >
    >
    > testA: PASSED
    >
    > testB: FAILED_MEDIUM
    >
    > testC: PASSED
    >
    >
    >
    > AND I told the wrapper to only fail on HIGH or above, the wrapper will
    >
    > tell me that I had a medium failure, but the wrapper will still exit
    >
    > with a 0 (success)
    >
    >
    >
    > if I get the same results as above, but tell the wrapper to fail on
    >
    > LOW, then it will tell me I had that medium failure, but the wrapper
    >
    > will exit with a 1 (failure).
    >
    >
    >
    > The problem is that my exit determination looks like this:
    >
    >
    >
    > if fail_priority == fail_levels['FAILED_CRITICAL']:
    >
    > if critical_fails:
    >
    > return 1
    >
    > if fail_priority == fail_levels['FAILED_HIGH']:
    >
    > if critical_fails or high_fails:
    >
    > return 1
    >
    > if fail_priority == fail_levels['FAILED_MEDIUM']:
    >
    > if critical_fails or high_fails or medium_fails:
    >
    > return 1
    >
    > if fail_priority == fail_levels['FAILED_LOW']:
    >
    > if critical_fails or high_fails or medium_fails or low_fails:
    >
    > return 1
    >
    >
    >
    > return 0
    >
    >
    >
    > So, to explain the above... the fail level can be set by the user when
    >
    > running the wrapper using -f (or it defaults to 'high')
    >
    > the wrapper assigns a number to each level using this:
    >
    >
    >
    > # Set correct fail level
    >
    > args.fail_level = 'FAILED_%s' % args.fail_level.upper()
    >
    >
    >
    > # Get our failure priority and create the priority values
    >
    > fail_levels = {'FAILED_CRITICAL':4,
    >
    > 'FAILED_HIGH':3,
    >
    > 'FAILED_MEDIUM':2,
    >
    > 'FAILED_LOW':1}
    >
    > fail_priority = fail_levels[args.fail_level]
    >
    >
    >
    > the variables critical_fails, high_fails, medium_fails, low_fails are
    >
    > all counters that are etiher None, or the number of tests that were
    >
    > failed.
    >
    >
    >
    > So using this output from the diagnostic tool:
    >
    >
    >
    > testA: PASSED
    >
    > testB: FAILED_HIGH
    >
    > testC: PASSED
    >
    > testD: FAILED_MEDIUM
    >
    > testE: PASSED
    >
    >
    >
    > critical_fails would be None
    >
    > high_fails would be 1
    >
    > medium_fails would be 1
    >
    > low_fails would be None.
    >
    >
    >
    > The exit code determination above works, but it just feels inelegant.
    >
    > It feels like there's a better way of implementing that, but I can't
    >
    > come up with one that still honors the fail level properly (e.g. other
    >
    > solutions will fail on medium, but won't fail properly on medium OR
    >
    > higher).
    >
    >
    >
    > I can provide the full script if necessary, if the above isn't enough
    >
    > to point me in a direction that has a better way of doing this...
    >
    >
    >
    > Thanks for looking,
    >
    >
    >
    > Jeff


    My back of the envelope coding would do it this way. Use an array of fail_counters, with PASSED as the first element all the way up to FAILED_CRITICALas the last element. Then use a simple loop starting from index fail_priority to the end of the list looking for errors. Like this:

    # Array of fail counters
    fail_counters = [
    0, # PASSED
    0, # LOW
    0, # MEDIUM
    0, # HIGH
    0 # CRITICAL
    ]

    .... run tests, accumulate error counts in fail_counters

    for i in range(fail_priority, len(fail_counters)):
    if fail_counters:
    return 1
    return 0
    donarb, Jan 15, 2013
    #3
  4. On Tue, 15 Jan 2013 18:24:44 -0500, J wrote:


    > The problem is that my exit determination looks like this:
    >
    > if fail_priority == fail_levels['FAILED_CRITICAL']:
    > if critical_fails:
    > return 1
    > if fail_priority == fail_levels['FAILED_HIGH']:
    > if critical_fails or high_fails:
    > return 1
    > if fail_priority == fail_levels['FAILED_MEDIUM']:
    > if critical_fails or high_fails or medium_fails:
    > return 1
    > if fail_priority == fail_levels['FAILED_LOW']:
    > if critical_fails or high_fails or medium_fails or low_fails:
    > return 1
    > return 0



    Here's a general solution that doesn't require the failure levels to be
    numbers, or listed in order, or to support any operation except equality
    testing.

    Start with an ordered list of failure severity levels, and an equivalent
    ordered list of failure counts. Find the offset of the requested failure
    level in the FAILURES list. Then inspect the failure counts, up to and
    including that offset, ignoring any failure outside of that range:


    failures = [FAILED_CRITICAL, FAILED_HIGH, FAILED_MEDIUM, FAILED_LOW]
    counts = [critical_fails, high_fails, medium_fails, low_fails]
    i = failures.index(fail_priority)
    if any(counts[:i+1]):
    print "Failed"
    else:
    print "No failures that we care about"


    The actual values for FAILED_CRITICAL etc. can be arbitrary values:
    integers *in any order*, strings, complicated records, anything so long
    as they support equality and are distinct. fail_priority must be set to
    one of those values.


    > the variables critical_fails, high_fails, medium_fails, low_fails are
    > all counters that are etiher None, or the number of tests that were
    > failed.


    Why set them to None when no tests failed? If no tests failed, why not
    just use 0?

    That is, instead of the counters being "the number of failures, or None",
    they could be "the number of failures".



    --
    Steven
    Steven D'Aprano, Jan 16, 2013
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Kamilche
    Replies:
    7
    Views:
    280
    Peter Hansen
    Jun 29, 2004
  2. Lilith
    Replies:
    8
    Views:
    343
    Dave Rahardja
    Sep 2, 2005
  3. Oscar Benjamin
    Replies:
    0
    Views:
    126
    Oscar Benjamin
    Jan 15, 2013
  4. Ian Kelly
    Replies:
    0
    Views:
    140
    Ian Kelly
    Jan 16, 2013
  5. Dr.Ruud
    Replies:
    1
    Views:
    158
    Ted Zlatanov
    Feb 21, 2013
Loading...

Share This Page