log and figure out what bits are slow and optimize them.

Discussion in 'Python' started by sajuptpm, Feb 10, 2012.

  1. sajuptpm

    sajuptpm Guest

    Hi,

    I want to log time taken to complete database requests inside a method/
    function using decorator . is it possible ????
    I think, i have to inject log code inside the method/fuctions or
    modify it.
    I wrote a decorator to log taken by a method/function to complete it
    execution and its working well.

    My requirement : log everything and figure out what bits are slow and
    optimize them.

    What are your suggestions ??
     
    sajuptpm, Feb 10, 2012
    #1
    1. Advertising

  2. On 10 February 2012 12:30, sajuptpm <> wrote:
    > Hi,
    >
    > I want to log time taken to complete database requests inside a method/
    > function using decorator .  is it possible ????
    > I think, i have to inject log code inside the method/fuctions or
    > modify it.
    > I wrote a decorator to log taken by a method/function to complete it
    > execution and its working well.
    >
    > My requirement : log everything and figure out what bits are slow and
    > optimize them.
    >
    > What are your suggestions ??


    Are you familiar with this?

    http://docs.python.org/library/profile.html

    --
    Arnaud
     
    Arnaud Delobelle, Feb 10, 2012
    #2
    1. Advertising

  3. sajuptpm

    sajuptpm Guest

    Hi,

    Yes i saw profile module,
    I think i have to do function call via

    cProfile.run('foo()')

    I know, we can debug this way.

    But, i need a fixed logging system and want to use it in production.
    I think, we can't permanently include profile's debugging code
    in source code,
    will cause any performance issue ??
     
    sajuptpm, Feb 10, 2012
    #3
  4. Please don't top post.

    On 10/02/2012 12:59, Saju M wrote:
    > Yes i saw profile module,
    > I think, i have to do function call via
    > cProfile.run('foo()')
    > I know, we can debug this way.
    > But, I need a fixed logging system and want to use it in production.
    > I think, we can't permanently include profile's debugging code in
    > source code,
    > will cause any performance issue ??
    >


    How about http://docs.python.org/library/logging.html ?

    --
    Cheers.

    Mark Lawrence.
     
    Mark Lawrence, Feb 10, 2012
    #4
  5. sajuptpm

    Kev Dwyer Guest

    sajuptpm wrote:

    > Hi,
    >
    > Yes i saw profile module,
    > I think i have to do function call via
    >
    > cProfile.run('foo()')
    >
    > I know, we can debug this way.
    >
    > But, i need a fixed logging system and want to use it in production.
    > I think, we can't permanently include profile's debugging code
    > in source code,
    > will cause any performance issue ??


    *Any* instrumentation code is going to affect performance.

    It's a trade-off that you need to analyse and manage in the context of your
    application.
     
    Kev Dwyer, Feb 10, 2012
    #5
  6. sajuptpm

    John Gordon Guest

    In <> Kev Dwyer <> writes:

    > *Any* instrumentation code is going to affect performance.


    Funny story about that...

    I wanted to profile some code of mine, and a colleague recommended the
    'hotshot' module.

    It's pretty easy to use: there are functions to start profiling, stop
    profiling and print results.

    So I added the function calls and ran my code.... and it took a really
    long time. I mean a REALLY long time. In fact I eventually had to kill
    the process.

    I briefly wondered if my coworker was playing a prank on me... then I
    realized that I had neglected to call the function to stop profiling!

    So when I went to print the results, it was still profiling... endlessly.

    (Okay, maybe it wasn't that funny.)

    --
    John Gordon A is for Amy, who fell down the stairs
    B is for Basil, assaulted by bears
    -- Edward Gorey, "The Gashlycrumb Tinies"
     
    John Gordon, Feb 10, 2012
    #6
  7. sajuptpm

    sajuptpm Guest

    I decided to create a decorator like.

    import cProfile
    def debug_time(method):
    def timed(*args, **kw):
    prof = cProfile.Profile()
    prof.enable(subcalls=False, builtins=False)
    result = prof.runcall(method, *args, **kw)
    #prof.print_stats()
    msg = "\n\n\n\n#######################################"
    msg += "\n\nURL : %s" %(tg.request.url)
    msg += "\nMethod: %r" %(method.__name__)
    print "--ddd--------", type(prof.getstats())
    msg += "\n\nStatus : %s" %(prof.print_stats())
    msg += "\n\n#######################################"
    print msg
    LOGGER.info(msg)
    return result
    return timed

    Ref : http://stackoverflow.com/questions/...s-a-method-call-and-logs-the-profiling-result


    I want to log it in existing log file in my project,
    so i tried prof.print_stats() and prof.getstats(). prof.getstats()
    will need extra loop for fetch data.
    prof.print_stats() will log library calls also.

    Please suggest a better way to log profiler output.
     
    sajuptpm, Feb 11, 2012
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. sarmin kho
    Replies:
    2
    Views:
    840
    A. Lloyd Flanagan
    Jun 15, 2004
  2. Leena Jethwa

    Create a log-in & log-out page

    Leena Jethwa, May 1, 2009, in forum: Ruby
    Replies:
    2
    Views:
    107
    Mike Stephens
    May 1, 2009
  3. Matthew
    Replies:
    3
    Views:
    92
    Matthew
    Feb 10, 2004
  4. it_says_BALLS_on_your forehead

    optimize log parsing

    it_says_BALLS_on_your forehead, Oct 4, 2005, in forum: Perl Misc
    Replies:
    21
    Views:
    256
  5. David Mark
    Replies:
    61
    Views:
    485
    David Mark
    Dec 19, 2009
Loading...

Share This Page