Trace dynamically compiled code?

[Ed Leafe]

Hi,

Thanks to the help of many on this list, I've been able to take code
that is created by the user in my app and add it to an object as an
instance method. The technique used is roughly:

nm = "myMethod"
code = """def myMethod(self):
print "Line 1"
print "My Value is %s" % self.Value
return
"""
compCode = compile(code, "", "exec")
exec compCode
exec "self.%s = %s.__get__(self)" % (nm, nm)

This is working great, but now I'm wondering if there is a way to
enable pdb tracing of the code as it executes? When tracing "normal"
code, pdb will show you the name of the script being executed, the
line number and the source code for the line about to be executed.
But when stepping through code compiled dynamically as above, the
current line's source code is not available to pdb, and thus does not
display.

Does anyone know a way to compile the dynamic code so that pdb can
'see' the source? I suppose I could write it all out to a bunch of
temp files, but that would be terribly messy. Are there any neater
solutions?

-- Ed Leafe
-- http://leafe.com
-- http://dabodev.com

 

[Ziga Seilnacht]

Hi,

Thanks to the help of many on this list, I've been able to take code
that is created by the user in my app and add it to an object as an
instance method. The technique used is roughly:

Just some notes about your code:

nm = "myMethod"
code = """def myMethod(self):
print "Line 1"
print "My Value is %s" % self.Value
return
"""
compCode = compile(code, "", "exec")
exec compCode

Try not using bare exec statements, since they pollute the local scope.
In your example you could use:

compCode = compile(code, "", "exec")
d = {}
exec compCode in d
func = d[nm]

See http://docs.python.org/ref/exec.html for details.

exec "self.%s = %s.__get__(self)" % (nm, nm)

You don't need dynamic execution here; you can simply use
setattr and the new module:

import new
method = new.instancemethod(func, self)
setattr(self, nm, method)

and yes, I remember that I was the one who suggested you
the __get__ hack.

This is working great, but now I'm wondering if there is a way to
enable pdb tracing of the code as it executes? When tracing "normal"
code, pdb will show you the name of the script being executed, the
line number and the source code for the line about to be executed.
But when stepping through code compiled dynamically as above, the
current line's source code is not available to pdb, and thus does not
display.

Does anyone know a way to compile the dynamic code so that pdb can
'see' the source? I suppose I could write it all out to a bunch of
temp files, but that would be terribly messy. Are there any neater
solutions?

You should check py lib: http://codespeak.net/py/current/doc/ ,
specifically the py.code "module". Then you can modify the
function from above:

import inspect
f = inspect.currentframe()
lineno = f.f_lineno - 5 # or some other constant
filename = f.f_code.co_filename

import py
co = py.code.Code(func)
new_code = co.new(co_lineno=lineno, co_filename=filename)
new_func = new.function(new_code, func.func_globals, nm,
func.func_defaults, func.func_closure)

-- Ed Leafe
-- http://leafe.com
-- http://dabodev.com

Ziga Seilnacht

 

[Ed Leafe]

Try not using bare exec statements, since they pollute the local
scope.
In your example you could use:

compCode = compile(code, "", "exec")
d = {}
exec compCode in d
func = d[nm]

OK, noted and changed in my source.

You don't need dynamic execution here; you can simply use
setattr and the new module:

import new
method = new.instancemethod(func, self)
setattr(self, nm, method)

and yes, I remember that I was the one who suggested you
the __get__ hack.

I've made the change you suggested. May I ask why one is better than
the other? Namespace pollution doesn't seem to be an issue here.

You should check py lib: http://codespeak.net/py/current/doc/ ,
specifically the py.code "module". Then you can modify the
function from above:

import inspect
f = inspect.currentframe()
lineno = f.f_lineno - 5 # or some other constant
filename = f.f_code.co_filename

import py
co = py.code.Code(func)
new_code = co.new(co_lineno=lineno, co_filename=filename)
new_func = new.function(new_code, func.func_globals, nm,
func.func_defaults, func.func_closure)

OK, thanks for the lead. I will take a look at this as soon as I
have the time to sit down and attempt to digest it. But before I do,
will I be able to add code like this to the same part of the
framework where the dynamic code is created, and have it work? I ask
this because an even bigger problem is the matter of creating dynamic
classes on the fly from saved XML files. I have a class that converts
the XML to the corresponding Python code for the class, and it's
working great, except for the tracing through the debugger issue.
Would I be able to apply the py lib stuff to this, too?

-- Ed Leafe
-- http://leafe.com
-- http://dabodev.com