idiom for debug code?

Discussion in 'Python' started by Dan Perl, Sep 30, 2004.

  1. Dan Perl

    Dan Perl Guest

    Is there a mechanism or an idiom for adding code for debugging so that it
    can easily be removed in the production code? I am thinking of something
    similar to the C/C++ preprocessor statements with which you can compile an
    application with the debug code or without it (the default).

    Dan
     
    Dan Perl, Sep 30, 2004
    #1
    1. Advertising

  2. Dan Perl

    Larry Bates Guest

    Dan Perl wrote:
    > Is there a mechanism or an idiom for adding code for debugging so that it
    > can easily be removed in the production code? I am thinking of something
    > similar to the C/C++ preprocessor statements with which you can compile an
    > application with the debug code or without it (the default).
    >
    > Dan
    >
    >

    Personally I think you should consider NOT removing the debugging code.
    It never ceases to amaze me how many times I must have a client run
    the application with debug set so that the program logs details about
    the running process and intermediate results. The output logfile can
    then be easily emailed, faxed, etc. to me so that I can determine what
    is REALLY the problem. The overhead of these if _debug: "type"
    statements seems incredibly low compared to the ability to get this
    information when needed. Just some thoughts based on my experience
    of the last 30+ years.

    Larry Bates
    Syscon, Inc.
     
    Larry Bates, Sep 30, 2004
    #2
    1. Advertising

  3. Dan Perl

    Dan Perl Guest

    I was not thinking of actually removing code. The analogy to C++
    preprocessing was just an example. I am looking at something that can be
    determined at run-time, not compile-time.

    And you're right, I'm not really concerned about the overhead of an "if"
    because I will not use this extensively anyway. This problem occurred to me
    when I decided to add an import for win32traceutil. I'm looking for a way
    to have that permanently in the code and enabling/disabling it only at
    run-time.

    I have thought of alternatives like an environment variable or a
    configuration parameter, but I was looking for other ideas. I thought this
    should be a common issue and that there could be an idiom for it.

    Dan

    PS: in the case of importing win32traceutil, I guess that checking the
    environment would be necessary anyway, or the import should be in a try
    statement, ignoring the exceptions.

    "Larry Bates" <> wrote in message
    news:...
    > Dan Perl wrote:
    >> Is there a mechanism or an idiom for adding code for debugging so that it
    >> can easily be removed in the production code? I am thinking of something
    >> similar to the C/C++ preprocessor statements with which you can compile
    >> an application with the debug code or without it (the default).
    >>
    >> Dan

    > Personally I think you should consider NOT removing the debugging code.
    > It never ceases to amaze me how many times I must have a client run
    > the application with debug set so that the program logs details about
    > the running process and intermediate results. The output logfile can
    > then be easily emailed, faxed, etc. to me so that I can determine what
    > is REALLY the problem. The overhead of these if _debug: "type"
    > statements seems incredibly low compared to the ability to get this
    > information when needed. Just some thoughts based on my experience
    > of the last 30+ years.
    >
    > Larry Bates
    > Syscon, Inc.
     
    Dan Perl, Sep 30, 2004
    #3
  4. [Dan Perl]:
    >
    > Is there a mechanism or an idiom for adding code for debugging so that it
    > can easily be removed in the production code? I am thinking of something
    > similar to the C/C++ preprocessor statements with which you can compile an
    > application with the debug code or without it (the default).


    assert is nice, but it's not really what I call debug code -- that's
    code which should stay.

    for "true" debug code, use the logging module and emit debug messages
    with logging.debug("..."). you can raise the debug level if you don't
    need them.

    or define your own debug() function. to disable debug, simply bind a
    null function to that name. this rebinding can be done dynamically,
    of course. (this is what the logging module does with unwanted log
    levels.)

    in either case, you may want to avoid code like:

    debug("Mother's smell is %s, number of taunts %d" % (fruit, count))

    since Python won't be smart enough to skip the string interpolation.
    instead, write it as

    debug("Mother's smell is %s, number of taunts %d", fruit, count)

    and do the interpolation inside the debug function.
    --
    Kjetil T.
     
    Kjetil Torgrim Homme, Oct 2, 2004
    #4
  5. Dan Perl

    Ian Parker Guest

    idiom for debug code? [1/1]

    In message <>, Dan Perl
    <> writes
    >I was not thinking of actually removing code. The analogy to C++
    >preprocessing was just an example. I am looking at something that can be
    >determined at run-time, not compile-time.
    >
    >And you're right, I'm not really concerned about the overhead of an "if"
    >because I will not use this extensively anyway. This problem occurred to me
    >when I decided to add an import for win32traceutil. I'm looking for a way
    >to have that permanently in the code and enabling/disabling it only at
    >run-time.
    >
    >I have thought of alternatives like an environment variable or a
    >configuration parameter, but I was looking for other ideas. I thought this
    >should be a common issue and that there could be an idiom for it.
    >
    >Dan
    >
    >PS: in the case of importing win32traceutil, I guess that checking the
    >environment would be necessary anyway, or the import should be in a try
    >statement, ignoring the exceptions.
    >
    >"Larry Bates" <> wrote in message
    >news:...
    >> Dan Perl wrote:
    >>> Is there a mechanism or an idiom for adding code for debugging so that it
    >>> can easily be removed in the production code? I am thinking of something
    >>> similar to the C/C++ preprocessor statements with which you can compile
    >>> an application with the debug code or without it (the default).
    >>>
    >>> Dan

    >> Personally I think you should consider NOT removing the debugging code.
    >> It never ceases to amaze me how many times I must have a client run
    >> the application with debug set so that the program logs details about
    >> the running process and intermediate results. The output logfile can
    >> then be easily emailed, faxed, etc. to me so that I can determine what
    >> is REALLY the problem. The overhead of these if _debug: "type"
    >> statements seems incredibly low compared to the ability to get this
    >> information when needed. Just some thoughts based on my experience
    >> of the last 30+ years.
    >>
    >> Larry Bates
    >> Syscon, Inc.

    >
    >


    I' use the following technique. I originally wrote it four years or so
    ago, so I'm sure it could be much improved. In fact looking at it now,
    I ma deeply embarrassed to post it, and know it could be dramatically
    improved both by using more modern features of Python, by better coding
    and by some comprehension of design.

    I have a utility.py module which contains my common functions and I
    import this in every module, plus a wrapper for the debug function:

    import utility

    debug = 0

    def dprint( *args):
    """
    print function toggled on or off by module debug variable
    """
    global debug
    if debug:
    apply(utility._dprint,args)


    This allows me to scatter dprint functions throughout the module instead
    of print statements, and enable or suppress them by setting the global
    'debug', e.g.

    def func(x):
    global total
    total += x
    dprint("total is currently",total)

    to get an output of something like:
    ! module.py:83 in func: total is currently 10


    The relevant code from my utility.py is attached.


    I would welcome rewrites of this code, of course.

    Regards

    Ian
    --
    Ian Parker
     
    Ian Parker, Oct 2, 2004
    #5
  6. Dan Perl

    Steve Holden Guest

    Re: idiom for debug code? [1/1]

    Ian Parker wrote:

    [...]
    > I have a utility.py module which contains my common functions and I
    > import this in every module, plus a wrapper for the debug function:
    >
    > import utility
    >
    > debug = 0
    >
    > def dprint( *args):
    > """
    > print function toggled on or off by module debug variable
    > """
    > global debug
    > if debug:
    > apply(utility._dprint,args)
    >
    >
    > This allows me to scatter dprint functions throughout the module instead
    > of print statements, and enable or suppress them by setting the global
    > 'debug', e.g.
    >
    > def func(x):
    > global total
    > total += x
    > dprint("total is currently",total)
    >
    > to get an output of something like:
    > ! module.py:83 in func: total is currently 10
    >


    Well, I don't do anything as sophisticated as dprint on the output side,
    and might consider lifting it. But in terms of selecting which debug
    code gets executed I often use something like

    if 3 in debug:

    because this makes it easy to selectively turn different bits of debug
    code on or off. Of course the test takes a little longer, but when not
    debugging it doesn't take long to check whether an integer is in an
    empty list.

    regards
    Steve
     
    Steve Holden, Oct 3, 2004
    #6
  7. Dan Perl

    Dan Perl Guest

    Re: idiom for debug code? [1/1]

    Thanks, Ian. It's late Saturday for me now so I will look more at the code
    later, but so far what I get from what you sent is only dprint and that is a
    logging utility. That's not what I was looking for. OTOH, I see that your
    posting was probably only a first installment, so I am looking forward to
    more details, including more parts from your utility module.

    The solution that you are using for the dprint utility involves a global
    variable. How do you change its value? Is that still by making a change in
    the code, even if the change is only one place? I would like a solution
    that does not involve any change in the code and is determined strictly at
    run-time. I was thinking of something like an environment variable or a
    configuration attribute. Of course, your solution can be adapted to set the
    global variable based on an environment variable or based on configuration.

    As for rewriting your code, did you take a look at the newsgroup thread
    about the new logging utility?

    Dan

    "Ian Parker" <> wrote in message
    news:...
    > In message <>, Dan Perl
    > <> writes
    >>I was not thinking of actually removing code. The analogy to C++
    >>preprocessing was just an example. I am looking at something that can be
    >>determined at run-time, not compile-time.
    >>
    >>And you're right, I'm not really concerned about the overhead of an "if"
    >>because I will not use this extensively anyway. This problem occurred to
    >>me
    >>when I decided to add an import for win32traceutil. I'm looking for a way
    >>to have that permanently in the code and enabling/disabling it only at
    >>run-time.
    >>
    >>I have thought of alternatives like an environment variable or a
    >>configuration parameter, but I was looking for other ideas. I thought
    >>this
    >>should be a common issue and that there could be an idiom for it.
    >>
    >>Dan
    >>
    >>PS: in the case of importing win32traceutil, I guess that checking the
    >>environment would be necessary anyway, or the import should be in a try
    >>statement, ignoring the exceptions.
    >>
    >>"Larry Bates" <> wrote in message
    >>news:...
    >>> Dan Perl wrote:
    >>>> Is there a mechanism or an idiom for adding code for debugging so that
    >>>> it
    >>>> can easily be removed in the production code? I am thinking of
    >>>> something
    >>>> similar to the C/C++ preprocessor statements with which you can compile
    >>>> an application with the debug code or without it (the default).
    >>>>
    >>>> Dan
    >>> Personally I think you should consider NOT removing the debugging code.
    >>> It never ceases to amaze me how many times I must have a client run
    >>> the application with debug set so that the program logs details about
    >>> the running process and intermediate results. The output logfile can
    >>> then be easily emailed, faxed, etc. to me so that I can determine what
    >>> is REALLY the problem. The overhead of these if _debug: "type"
    >>> statements seems incredibly low compared to the ability to get this
    >>> information when needed. Just some thoughts based on my experience
    >>> of the last 30+ years.
    >>>
    >>> Larry Bates
    >>> Syscon, Inc.

    >>
    >>

    >
    > I' use the following technique. I originally wrote it four years or so
    > ago, so I'm sure it could be much improved. In fact looking at it now,
    > I ma deeply embarrassed to post it, and know it could be dramatically
    > improved both by using more modern features of Python, by better coding
    > and by some comprehension of design.
    >
    > I have a utility.py module which contains my common functions and I
    > import this in every module, plus a wrapper for the debug function:
    >
    > import utility
    >
    > debug = 0
    >
    > def dprint( *args):
    > """
    > print function toggled on or off by module debug variable
    > """
    > global debug
    > if debug:
    > apply(utility._dprint,args)
    >
    >
    > This allows me to scatter dprint functions throughout the module instead
    > of print statements, and enable or suppress them by setting the global
    > 'debug', e.g.
    >
    > def func(x):
    > global total
    > total += x
    > dprint("total is currently",total)
    >
    > to get an output of something like:
    > ! module.py:83 in func: total is currently 10
    >
    >
    > The relevant code from my utility.py is attached.
    >



    --------------------------------------------------------------------------------


    >
    > I would welcome rewrites of this code, of course.
    >
    > Regards
    >
    > Ian
    > --
    > Ian Parker
    >
     
    Dan Perl, Oct 3, 2004
    #7
  8. Dan Perl

    Dan Perl Guest

    Re: idiom for debug code? [1/1]

    "Dan Perl" <> wrote in message
    news:...
    > Thanks, Ian. It's late Saturday for me now so I will look more at the
    > code later, but so far what I get from what you sent is only dprint and
    > that is a logging utility. That's not what I was looking for. OTOH, I
    > see that your posting was probably only a first installment, so I am
    > looking forward to more details, including more parts from your utility
    > module.


    I just realized that maybe you meant that posting to be the only one and not
    an installment, hence the one of ONE ([1/1]). Too bad. Could you post also
    the rest of your utility script? Or would that break some copyright (do you
    own it?)?

    > The solution that you are using for the dprint utility involves a global
    > variable. How do you change its value? Is that still by making a change
    > in the code, even if the change is only one place? I would like a
    > solution that does not involve any change in the code and is determined
    > strictly at run-time. I was thinking of something like an environment
    > variable or a configuration attribute. Of course, your solution can be
    > adapted to set the global variable based on an environment variable or
    > based on configuration.
    >
    > As for rewriting your code, did you take a look at the newsgroup thread
    > about the new logging utility?
    >
    > Dan
     
    Dan Perl, Oct 3, 2004
    #8
  9. Dan Perl

    Ian Parker Guest

    Re: idiom for debug code? [1/1]

    In message <HzJ7d.288401$4o.8905@fed1read01>, Steve Holden
    <> writes
    >Ian Parker wrote:
    >
    >[...]
    >> I have a utility.py module which contains my common functions and I
    >> import this in every module, plus a wrapper for the debug function:
    >> import utility
    >> debug = 0
    >> def dprint( *args):
    >> """
    >> print function toggled on or off by module debug variable
    >> """
    >> global debug
    >> if debug:
    >> apply(utility._dprint,args)
    >> This allows me to scatter dprint functions throughout the module
    >>instead
    >> of print statements, and enable or suppress them by setting the global
    >> 'debug', e.g.
    >> def func(x):
    >> global total
    >> total += x
    >> dprint("total is currently",total)
    >> to get an output of something like:
    >> ! module.py:83 in func: total is currently 10
    >>

    >
    >Well, I don't do anything as sophisticated as dprint on the output
    >side, and might consider lifting it. But in terms of selecting which
    >debug code gets executed I often use something like
    >
    > if 3 in debug:
    >
    >because this makes it easy to selectively turn different bits of debug
    >code on or off. Of course the test takes a little longer, but when not
    >debugging it doesn't take long to check whether an integer is in an
    >empty list.
    >
    >regards
    > Steve


    I like that - it does give a flexibility that my dprint lacks. I do
    tend to find that, although I may enable my dprint only for a function
    or method, I still comment out some of the dprint calls.

    Regards

    Ian

    --
    Ian Parker
     
    Ian Parker, Oct 3, 2004
    #9
  10. Dan Perl

    Ian Parker Guest

    Re: idiom for debug code? [1/1]

    In message <>, Dan Perl
    <> writes
    >Thanks, Ian. It's late Saturday for me now so I will look more at the code
    >later, but so far what I get from what you sent is only dprint and that is a
    >logging utility. That's not what I was looking for. OTOH, I see that your
    >posting was probably only a first installment, so I am looking forward to
    >more details, including more parts from your utility module.
    >
    >The solution that you are using for the dprint utility involves a global
    >variable. How do you change its value? Is that still by making a change in
    >the code, even if the change is only one place? I would like a solution
    >that does not involve any change in the code and is determined strictly at
    >run-time. I was thinking of something like an environment variable or a
    >configuration attribute. Of course, your solution can be adapted to set the
    >global variable based on an environment variable or based on configuration.
    >
    >As for rewriting your code, did you take a look at the newsgroup thread
    >about the new logging utility?
    >
    >Dan
    >
    >"Ian Parker" <> wrote in message
    >news:...
    >> In message <>, Dan Perl
    >> <> writes
    >>>I was not thinking of actually removing code. The analogy to C++
    >>>preprocessing was just an example. I am looking at something that can be
    >>>determined at run-time, not compile-time.
    >>>
    >>>And you're right, I'm not really concerned about the overhead of an "if"
    >>>because I will not use this extensively anyway. This problem occurred to
    >>>me
    >>>when I decided to add an import for win32traceutil. I'm looking for a way
    >>>to have that permanently in the code and enabling/disabling it only at
    >>>run-time.
    >>>
    >>>I have thought of alternatives like an environment variable or a
    >>>configuration parameter, but I was looking for other ideas. I thought
    >>>this
    >>>should be a common issue and that there could be an idiom for it.
    >>>
    >>>Dan
    >>>
    >>>PS: in the case of importing win32traceutil, I guess that checking the
    >>>environment would be necessary anyway, or the import should be in a try
    >>>statement, ignoring the exceptions.
    >>>
    >>>"Larry Bates" <> wrote in message
    >>>news:...
    >>>> Dan Perl wrote:
    >>>>> Is there a mechanism or an idiom for adding code for debugging so that
    >>>>> it
    >>>>> can easily be removed in the production code? I am thinking of
    >>>>> something
    >>>>> similar to the C/C++ preprocessor statements with which you can compile
    >>>>> an application with the debug code or without it (the default).
    >>>>>
    >>>>> Dan
    >>>> Personally I think you should consider NOT removing the debugging code.
    >>>> It never ceases to amaze me how many times I must have a client run
    >>>> the application with debug set so that the program logs details about
    >>>> the running process and intermediate results. The output logfile can
    >>>> then be easily emailed, faxed, etc. to me so that I can determine what
    >>>> is REALLY the problem. The overhead of these if _debug: "type"
    >>>> statements seems incredibly low compared to the ability to get this
    >>>> information when needed. Just some thoughts based on my experience
    >>>> of the last 30+ years.
    >>>>
    >>>> Larry Bates
    >>>> Syscon, Inc.
    >>>
    >>>

    >>
    >> I' use the following technique. I originally wrote it four years or so
    >> ago, so I'm sure it could be much improved. In fact looking at it now,
    >> I ma deeply embarrassed to post it, and know it could be dramatically
    >> improved both by using more modern features of Python, by better coding
    >> and by some comprehension of design.
    >>
    >> I have a utility.py module which contains my common functions and I
    >> import this in every module, plus a wrapper for the debug function:
    >>
    >> import utility
    >>
    >> debug = 0
    >>
    >> def dprint( *args):
    >> """
    >> print function toggled on or off by module debug variable
    >> """
    >> global debug
    >> if debug:
    >> apply(utility._dprint,args)
    >>
    >>
    >> This allows me to scatter dprint functions throughout the module instead
    >> of print statements, and enable or suppress them by setting the global
    >> 'debug', e.g.
    >>
    >> def func(x):
    >> global total
    >> total += x
    >> dprint("total is currently",total)
    >>
    >> to get an output of something like:
    >> ! module.py:83 in func: total is currently 10
    >>
    >>
    >> The relevant code from my utility.py is attached.
    >>

    >
    >
    >--------------------------------------------------------------------------------
    >
    >
    >>
    >> I would welcome rewrites of this code, of course.
    >>
    >> Regards
    >>
    >> Ian
    >> --
    >> Ian Parker
    >>

    >
    >


    That was indeed the only instalment. Yes, thinking about it now, dprint
    is simply a logging system, so I really should look at the logging
    module -(which didn't exist when I wrote dprint). Feel free to lift the
    code - I certainly lifted chunks of it from other examples I find on
    this newsgroup.

    Yes, 'debug' is a global variable. The reason for the inclusion of the
    dprint wrapper in each module is to make use of the module 'debug' to
    give a little more granularity. Two points:

    In a function, I frequently do what feels like a very clumsy hack:

    def func(x,y):
    global debug ; olddebug = debug
    # debug = 1
    .
    .
    debug = olddebug
    return

    so that any embedded dprints can be controlled more specifically.

    One other feature of dprint: if you call it without parameters it will
    attempt to display the arguments of the function in which it occurs.

    What other functions do I have in my utility.py? The same as everybody
    else - all those things you find yourself writing that you think you
    will use often. No I'm not publishing it - the code is frequently even
    worse than that of dprint.

    Regards

    Ian
    --
    Ian Parker
     
    Ian Parker, Oct 3, 2004
    #10
  11. Dan Perl

    Larry Bates Guest

    Dan Perl wrote:
    > I was not thinking of actually removing code. The analogy to C++
    > preprocessing was just an example. I am looking at something that can be
    > determined at run-time, not compile-time.
    >
    > And you're right, I'm not really concerned about the overhead of an "if"
    > because I will not use this extensively anyway. This problem occurred to me
    > when I decided to add an import for win32traceutil. I'm looking for a way
    > to have that permanently in the code and enabling/disabling it only at
    > run-time.
    >
    > I have thought of alternatives like an environment variable or a
    > configuration parameter, but I was looking for other ideas. I thought this
    > should be a common issue and that there could be an idiom for it.
    >
    > Dan
    >
    > PS: in the case of importing win32traceutil, I guess that checking the
    > environment would be necessary anyway, or the import should be in a try
    > statement, ignoring the exceptions.
    >
    > "Larry Bates" <> wrote in message
    > news:...
    >
    >>Dan Perl wrote:
    >>
    >>>Is there a mechanism or an idiom for adding code for debugging so that it
    >>>can easily be removed in the production code? I am thinking of something
    >>>similar to the C/C++ preprocessor statements with which you can compile
    >>>an application with the debug code or without it (the default).
    >>>
    >>>Dan

    >>
    >>Personally I think you should consider NOT removing the debugging code.
    >>It never ceases to amaze me how many times I must have a client run
    >>the application with debug set so that the program logs details about
    >>the running process and intermediate results. The output logfile can
    >>then be easily emailed, faxed, etc. to me so that I can determine what
    >>is REALLY the problem. The overhead of these if _debug: "type"
    >>statements seems incredibly low compared to the ability to get this
    >>information when needed. Just some thoughts based on my experience
    >>of the last 30+ years.
    >>
    >>Larry Bates
    >>Syscon, Inc.

    >
    >
    >


    Dan,

    You have 3 choices:

    1) Interrogate the environment to see what needs to be done. Like
    figuring out you are on Windows do one thing, on Linux something else.
    sys.platform comes in handy for this choice.

    2) Arguments to the processor (e.g. program -W). Then you can do
    different things depending on the arguments. I use getopt to pick
    these up.

    3) Entries in .INI file. I use this a LOT. Entries to turn on
    tracing, set my debug level, saving default paths, etc. can easily
    be set in configuration files (see ConfigParser). I actually use
    config files AND arguments on the processor call simultaneously.
    Processor call arguments override anything in the .INI file.
    I have code that has thousands of lines of if _debug > 1: dosomething
    and just don't see any performance problems.

    Larry Bates
    Syscon, Inc.
     
    Larry Bates, Oct 5, 2004
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. RonL
    Replies:
    0
    Views:
    776
  2. ringos75
    Replies:
    0
    Views:
    997
    ringos75
    Apr 14, 2005
  3. Robert Brewer

    RE: idiom for debug code?

    Robert Brewer, Sep 30, 2004, in forum: Python
    Replies:
    1
    Views:
    361
    Dan Perl
    Sep 30, 2004
  4. Roman Suzi
    Replies:
    13
    Views:
    627
    Bengt Richter
    Jan 7, 2005
  5. eliben
    Replies:
    30
    Views:
    730
    eliben
    Jun 25, 2008
Loading...

Share This Page