Profiling Python Apps on Mac?

[RGK]

I'm writing a python app on a Mac (in Eclipse + PyDev w/ Python2.5 &
wxPython under OSX 10.4)

As I make program architecture decisions, it would be nice to be able to
profile the choices. Should I add that extra thread? Is this big-assed
xml object I just created horribly bloated or kind of ordinary.

Is there anything out there I should look into to if I want to see how
those things are affecting my app? The closest I have is the widget
iStat, but it's a very static low resolution view of what's really going on.

Thx in advance.
Ross.

 

[Philip Semanchuk]

I'm writing a python app on a Mac (in Eclipse + PyDev w/ Python2.5 &
wxPython under OSX 10.4)

As I make program architecture decisions, it would be nice to be
able to profile the choices. Should I add that extra thread? Is
this big-assed xml object I just created horribly bloated or kind of
ordinary.

Is there anything out there I should look into to if I want to see
how those things are affecting my app? The closest I have is the
widget iStat, but it's a very static low resolution view of what's
really going on.

Is there any reason that this wouldn't work?

http://docs.python.org/library/hotshot.html

 

[RGK]

Is there any reason that this wouldn't work?

http://docs.python.org/library/hotshot.html

It suggests that it doesn't work well with threads, but as I didn't know
about any options, it's a step forward. Thanks for the pointer.

R.

 

[Pierre-Alain Dorange]

As I make program architecture decisions, it would be nice to be able to
profile the choices. Should I add that extra thread? Is this big-assed
xml object I just created horribly bloated or kind of ordinary.

python -m profile yourscript.py

see <http://docs.python.org/library/profile.html>

 

[David Cournapeau]

I'm writing a python app on a Mac (in Eclipse + PyDev w/ Python2.5 &
wxPython under OSX 10.4)

As I make program architecture decisions, it would be nice to be able to
profile the choices. Should I add that extra thread? Is this big-assed xml
object I just created horribly bloated or kind of ordinary.

Is there anything out there I should look into to if I want to see how those
things are affecting my app? The closest I have is the widget iStat, but
it's a very static low resolution view of what's really going on.

On Mac OS X, dtrace can sometimes be quite useful. The Apple-provided
python has the necessary 'probes'.

http://blogs.sun.com/binujp/entry/dtrace_provider_for_python

There are some useful scripts in the dtrace toolkit geared toward python:

http://www.brendangregg.com/dtrace.html#DTraceToolkit

If you want dtrace support for a bare python (built from sources), I
made a quick patch, "documented" here:

http://cournape.wordpress.com/?s=dtrace

I don't know if it works for 2.6 (I doubt it would work out of the box
- I don't claim any understanding of dtrace internals, I just applied
the apple patches, removing all the apple specifics I did not care
about).

David

 

[Robert Kern]

I'm writing a python app on a Mac (in Eclipse + PyDev w/ Python2.5 &
wxPython under OSX 10.4)

As I make program architecture decisions, it would be nice to be able to
profile the choices. Should I add that extra thread? Is this big-assed
xml object I just created horribly bloated or kind of ordinary.

Is there anything out there I should look into to if I want to see how
those things are affecting my app? The closest I have is the widget
iStat, but it's a very static low resolution view of what's really going
on.

I have a script kernprof.py which provides a few conveniences over the builtin
cProfile module. One of its modes of operation is to inject a decorator into the
__builtins__. It will enable the profiler on entry to the method and disable it
on exit. This lets you localize your profile results to just the part of your
code that you are interested in. I found this especially useful in GUI apps
which require user interaction to trigger the part of the code you are actually
interesting in profiling. You don't want the interesting parts of your profile
to be obscured by the GUI event loop waiting for your input.

You can get it as part of my line_profiler package (which you may also be
interested in; cProfile profiles function calls, line_profiler profiles
individual lines).

http://pypi.python.org/pypi/line_profiler

You can view the profile results interactively with "python -m pstats
my_script.py.prof", RunSnakeRun, or pyprof2calltree if you manage to install
kcachegrind on your system:

http://www.vrplumber.com/programming/runsnakerun/
http://pypi.python.org/pypi/pyprof2calltree/1.1.0

I don't recommend using hotshot because it is deprecated and slow to postprocess
the data dumps. Also, I don't recommend using the plain profile module because
it slows down your program rather more than cProfile. See the Python
documentation for an overview of these modules:

http://docs.python.org/library/profile

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco