Trace dynamically compiled code?

Discussion in 'Python' started by Ed Leafe, Mar 14, 2006.

  1. Ed Leafe

    Ed Leafe Guest

    Hi,

    Thanks to the help of many on this list, I've been able to take code
    that is created by the user in my app and add it to an object as an
    instance method. The technique used is roughly:

    nm = "myMethod"
    code = """def myMethod(self):
    print "Line 1"
    print "My Value is %s" % self.Value
    return
    """
    compCode = compile(code, "", "exec")
    exec compCode
    exec "self.%s = %s.__get__(self)" % (nm, nm)

    This is working great, but now I'm wondering if there is a way to
    enable pdb tracing of the code as it executes? When tracing "normal"
    code, pdb will show you the name of the script being executed, the
    line number and the source code for the line about to be executed.
    But when stepping through code compiled dynamically as above, the
    current line's source code is not available to pdb, and thus does not
    display.

    Does anyone know a way to compile the dynamic code so that pdb can
    'see' the source? I suppose I could write it all out to a bunch of
    temp files, but that would be terribly messy. Are there any neater
    solutions?

    -- Ed Leafe
    -- http://leafe.com
    -- http://dabodev.com
    Ed Leafe, Mar 14, 2006
    #1
    1. Advertising

  2. Ed Leafe wrote:
    > Hi,
    >
    > Thanks to the help of many on this list, I've been able to take code
    > that is created by the user in my app and add it to an object as an
    > instance method. The technique used is roughly:


    Just some notes about your code:

    > nm = "myMethod"
    > code = """def myMethod(self):
    > print "Line 1"
    > print "My Value is %s" % self.Value
    > return
    > """
    > compCode = compile(code, "", "exec")
    > exec compCode


    Try not using bare exec statements, since they pollute the local scope.
    In your example you could use:

    compCode = compile(code, "", "exec")
    d = {}
    exec compCode in d
    func = d[nm]

    See http://docs.python.org/ref/exec.html for details.

    > exec "self.%s = %s.__get__(self)" % (nm, nm)


    You don't need dynamic execution here; you can simply use
    setattr and the new module:

    import new
    method = new.instancemethod(func, self)
    setattr(self, nm, method)

    and yes, I remember that I was the one who suggested you
    the __get__ hack.

    > This is working great, but now I'm wondering if there is a way to
    > enable pdb tracing of the code as it executes? When tracing "normal"
    > code, pdb will show you the name of the script being executed, the
    > line number and the source code for the line about to be executed.
    > But when stepping through code compiled dynamically as above, the
    > current line's source code is not available to pdb, and thus does not
    > display.
    >
    > Does anyone know a way to compile the dynamic code so that pdb can
    > 'see' the source? I suppose I could write it all out to a bunch of
    > temp files, but that would be terribly messy. Are there any neater
    > solutions?


    You should check py lib: http://codespeak.net/py/current/doc/ ,
    specifically the py.code "module". Then you can modify the
    function from above:

    import inspect
    f = inspect.currentframe()
    lineno = f.f_lineno - 5 # or some other constant
    filename = f.f_code.co_filename

    import py
    co = py.code.Code(func)
    new_code = co.new(co_lineno=lineno, co_filename=filename)
    new_func = new.function(new_code, func.func_globals, nm,
    func.func_defaults, func.func_closure)


    > -- Ed Leafe
    > -- http://leafe.com
    > -- http://dabodev.com


    Ziga Seilnacht
    Ziga Seilnacht, Mar 14, 2006
    #2
    1. Advertising

  3. Ed Leafe

    Ed Leafe Guest

    On Mar 14, 2006, at 12:45 PM, Ziga Seilnacht wrote:

    > Try not using bare exec statements, since they pollute the local
    > scope.
    > In your example you could use:
    >
    > compCode = compile(code, "", "exec")
    > d = {}
    > exec compCode in d
    > func = d[nm]


    OK, noted and changed in my source.

    >> exec "self.%s = %s.__get__(self)" % (nm, nm)

    >
    > You don't need dynamic execution here; you can simply use
    > setattr and the new module:
    >
    > import new
    > method = new.instancemethod(func, self)
    > setattr(self, nm, method)
    >
    > and yes, I remember that I was the one who suggested you
    > the __get__ hack.


    I've made the change you suggested. May I ask why one is better than
    the other? Namespace pollution doesn't seem to be an issue here.

    >> Does anyone know a way to compile the dynamic code so that pdb can
    >> 'see' the source? I suppose I could write it all out to a bunch of
    >> temp files, but that would be terribly messy. Are there any neater
    >> solutions?

    >
    > You should check py lib: http://codespeak.net/py/current/doc/ ,
    > specifically the py.code "module". Then you can modify the
    > function from above:
    >
    > import inspect
    > f = inspect.currentframe()
    > lineno = f.f_lineno - 5 # or some other constant
    > filename = f.f_code.co_filename
    >
    > import py
    > co = py.code.Code(func)
    > new_code = co.new(co_lineno=lineno, co_filename=filename)
    > new_func = new.function(new_code, func.func_globals, nm,
    > func.func_defaults, func.func_closure)


    OK, thanks for the lead. I will take a look at this as soon as I
    have the time to sit down and attempt to digest it. But before I do,
    will I be able to add code like this to the same part of the
    framework where the dynamic code is created, and have it work? I ask
    this because an even bigger problem is the matter of creating dynamic
    classes on the fly from saved XML files. I have a class that converts
    the XML to the corresponding Python code for the class, and it's
    working great, except for the tracing through the debugger issue.
    Would I be able to apply the py lib stuff to this, too?

    -- Ed Leafe
    -- http://leafe.com
    -- http://dabodev.com
    Ed Leafe, Mar 15, 2006
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rukmal Fernando
    Replies:
    4
    Views:
    852
    Richard Grimes [MVP]
    Oct 27, 2003
  2. Matthias S.
    Replies:
    1
    Views:
    650
    Daniel Fisher\(lennybacon\)
    Nov 30, 2005
  3. Klaus Schneider
    Replies:
    1
    Views:
    540
    Rolf Magnus
    Dec 2, 2004
  4. Jiho Han
    Replies:
    0
    Views:
    517
    Jiho Han
    Aug 22, 2006
  5. lander
    Replies:
    5
    Views:
    588
    bruce barker
    Mar 5, 2008
Loading...

Share This Page