Profiling from *within* a module

[Leo Breebaart]

Hi all,

I'm new to Python, and I have hit upon something that I just cannot seem
to solve or understand using the available documentation.

Given the file 'leotest.py':

---------------------------------------------------------------------------
""" leotest.py - test ways of profiling python functions. """

def foo():
pass

def test():
for i in range(10000): foo()

def myprofile():
import profile
profile.run('test()')

if __name__ == "__main__":
myprofile()
---------------------------------------------------------------------------

If I call this as "python leotest.py", everything works and a profile
is generated:

[cygnus] % python leotest.py
10003 function calls in 0.040 CPU seconds
[...]

Profiling from the Python interpreter is also no problem:
10003 function calls in 0.030 CPU seconds
[...]

But try to have the profile called from 'within' the module, and Python
just doesn't seem to want to know:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "leotest.py", line 11, in myprofile
profile.run('test()')
File "/usr/lib/python2.3/profile.py", line 71, in run
prof = prof.run(statement)
File "/usr/lib/python2.3/profile.py", line 403, in run
return self.runctx(cmd, dict, dict)
File "/usr/lib/python2.3/profile.py", line 409, in runctx
exec cmd in globals, locals
File "<string>", line 1, in ?
NameError: name 'test' is not defined


I think I'm simply misunderstanding how modules and functions and scopes
interact in Python, but I can't discover where exactly I'm going wrong
-- and I can't find a workaround either. Surely it is *somehow* possible
to do what I want, i.e. initiate the profile from within the module
itself? After all, it works in the command-line version!

Would anybody here be willing to explain this behaviour to me?

Many thanks in advance,

 

[David Boddie]

Given the file 'leotest.py':

---------------------------------------------------------------------------
""" leotest.py - test ways of profiling python functions. """

def foo():
pass

def test():
for i in range(10000): foo()

def myprofile():
import profile
profile.run('test()')

if __name__ == "__main__":
myprofile()
---------------------------------------------------------------------------

Fine so far.

[...]

[Using the interpreter interactively...]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "leotest.py", line 11, in myprofile
profile.run('test()')
File "/usr/lib/python2.3/profile.py", line 71, in run
prof = prof.run(statement)

The following two lines of code indicate what's happening:

File "/usr/lib/python2.3/profile.py", line 403, in run
return self.runctx(cmd, dict, dict)
File "/usr/lib/python2.3/profile.py", line 409, in runctx
exec cmd in globals, locals
File "<string>", line 1, in ?
NameError: name 'test' is not defined

Looking at the profile.py file, it seems that the profiler executes the command
you give in the __main__ namespace. To get around this, you need to use the
Profile class directly:

def myprofile():
import profile

# Create a profiler instance.
p = profile.Profile()

# Retrieve a dictionary corresponding to the modules namespace.
dict = globals()

# Profile the test function and print the statistics.
p.runctx('test()', dict, dict)
p.print_stats()

This should work from either the command line or within an interactive session.
There may be a cleaner way to do the above if there's a convenience function
like profile.run which takes global and local attribute dictionaries.

I think I'm simply misunderstanding how modules and functions and scopes
interact in Python, but I can't discover where exactly I'm going wrong
-- and I can't find a workaround either.

I think you do understand how the scopes interact. It's just that the profile.run
function is playing tricks behind your back. ;-)

David