what does "execfile" mean within profiler output and why does it not have a attached line number

[Rahul]

"profile" tells me that most of my runtime was spent in just one part (1.28
sec cumulatively out of 1.29 secs. But what is "execfile"? I don't see this
as a function call with my python code. Also what's the 0 in the snippet:
":0(execfile)"? Isn't there supposed to be a line-number?

Looking up "execfile" in the python manual leads me to "exec": "This
statement supports dynamic execution of Python code."

But that seems pretty generic; how can I now try figuring out which part of
my python file is the bottleneck?

Sorry, I'm a newbiee to profiling.


######################################################################
51651 function calls (37762 primitive calls) in 1.290 CPU seconds
ncalls tottime percall cumtime percall filename:lineno(function)
[snip]
1 0.010 0.010 1.280 1.280 :0(execfile)
[snip]
##########################################################

 

[John Machin]

"profile" tells me that most of my runtime was spent in just one part (1.28
sec cumulatively out of 1.29 secs. But what is "execfile"? I don't see this
as a function call with my python code. Also what's the 0 in the snippet:  
":0(execfile)"? Isn't there supposed to be a line-number?

Looking up "execfile" in the python manual leads me to "exec": "This
statement supports dynamic execution of Python code."

But that seems pretty generic; how can I now try figuring out which part of
my python file is the bottleneck?

Sorry, I'm a newbiee to profiling.

######################################################################
       51651 function calls (37762 primitive calls) in 1.290 CPU seconds
 ncalls  tottime  percall  cumtime  percall filename:lineno(function)
[snip]
    1    0.010    0.010    1.280    1.280 :0(execfile)
[snip]
##########################################################

That means no more than "the profiler executed all of your code
once, it took 0.01 seconds inside the execfile itself, 0.01 seconds
per execution, total time spent by the execfile *and* what it called
was 1.28 seconds ("cum" == "cumulative"), again 1.28 secs per
execution"

So ignore that and look at the figures for the app functions/methods.

 

[Robert Kern]

"profile" tells me that most of my runtime was spent in just one part (1.28
sec cumulatively out of 1.29 secs. But what is "execfile"? I don't see this
as a function call with my python code. Also what's the 0 in the snippet:
":0(execfile)"? Isn't there supposed to be a line-number?

It's a builtin function, so it has no filename or line number.

execfile is a function that reads a Python file and executes its code. This is
almost certainly the execfile call from profile.py itself that is running your
code. Ignore it.

Looking up "execfile" in the python manual leads me to "exec": "This
statement supports dynamic execution of Python code."

But that seems pretty generic; how can I now try figuring out which part of
my python file is the bottleneck?

To quickly find your hotspots, start by sorting by 'time' (that would be
displayed as the 'tottime' column in the human-readable output). That tells you
how much time is spent in each function itself, excluding the time it spends
calling out to other functions. For example, per the docs under "Instant User’s
Manual" (which you might want to spend a little more time with):

p.sort_stats('time').print_stats(10)

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Rahul]

To quickly find your hotspots, start by sorting by 'time' (that would
be displayed as the 'tottime' column in the human-readable output).
That tells you how much time is spent in each function itself,
excluding the time it spends calling out to other functions. For
example, per the docs under "Instant Userƒ Ts Manual" (which you might
want to spend a little more time with):

p.sort_stats('time').print_stats(10)

Thanks Robert. I was executing the profiler on the commandline like so:

python -m profile ~/bin/visualize.py *.nc

Is there any way to pass further options of the form sort etc. via this
invocation. The manual did not specify usage of this form.

 

[Robert Kern]

Thanks Robert. I was executing the profiler on the commandline like so:

python -m profile ~/bin/visualize.py *.nc

Is there any way to pass further options of the form sort etc. via this
invocation. The manual did not specify usage of this form.

$ python -m profile --help
Usage: profile.py [-o output_file_path] [-s sort] scriptfile [arg] ...

Options:
-h, --help show this help message and exit
-o OUTFILE, --outfile=OUTFILE
Save stats to <outfile>
-s SORT, --sort=SORT Sort order when printing to stdout, based on
pstats.Stats class

Additionally, you can run pstats on the outfile for an interactive viewer:

$ python -m pstats visualize.prof

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco