C Python: Running Python code within function scope

[channel727272]

The Python C API function PyEval_EvalCode let's you execute compiled Pythoncode. I want to execute a block of Python code as if it were executing within the scope of a function, so that it has its own dictionary of local variables which don't affect the global state.

This seems easy enough to do, since PyEval_EvalCode lets you provide a Global and Local dictionary:

PyObject* PyEval_EvalCode(PyCodeObject *co, PyObject *globals, PyObject *locals)

The problem I run into has to do with how Python looks up variable names. Consider the following code, that I execute with PyEval_EvalCode:

myvar = 300
def func():
return myvar

func()

This simple code actually raises an error, because Python is unable to findthe variable myvar from within func. Even though myvar is in the local dictionary in the outer scope, Python doesn't copy it into the local dictionary in the inner scope. The reason for this is as follows:

Whenever Python looks up a variable name, first it checks locals, then it checks globals, and finally it checks builtins. At module scope, locals and globals are the SAME dictionary object. So the statement x = 5 at module scope will place x in the the locals dictionary, which is also the globals dictionary. Now, a function defined at module scope which needs to lookup xwon't find x within the function-scope locals, because Python doesn't copymodule-scope locals into function-scope locals. But this normally isn't a problem, because it can find x in globals.

x = 5
def foo():
print(x) # This works because 'x' in globals() == True

It's only with nested functions, that Python seems to copy outer-scope locals into inner-scope locals. (It also seems to do so lazily, only if they are needed within the inner scope.)

def foo():
x = 5
def bar():
print(x) # Now 'x' in locals() == True
bar()


So the result of all this is that, when executing code at module scope, youHAVE to make sure that your global dictionary and local dictionary are theSAME object, otherwise module-scope functions won't be able to access module-scope variables.

But in my case, I don't WANT the global dictionary and local dictionary to be the same. So I need some way to tell the Python interpreter that I am executing code at function scope. Is there some way to do this? I looked at the PyCompileFlags as well as the additional arguments to PyEval_EvalCodeEx and can't find any way to do this.

 

[Terry Reedy]

The Python C API function PyEval_EvalCode let's you execute compiled
Python code. I want to execute a block of Python code as if it were
executing within the scope of a function, so that it has its own
dictionary of local variables which don't affect the global state.

You cannot really do this. The local namespace of functions is not a dict.

This seems easy enough to do, since PyEval_EvalCode lets you provide
a Global and Local dictionary:

PyObject* PyEval_EvalCode(PyCodeObject *co, PyObject *globals,
PyObject *locals)

When you do this, you are executing code as if in a class statement, not
as if in a function.

The problem I run into has to do with how Python looks up variable
names. Consider the following code, that I execute with
PyEval_EvalCode:

myvar = 300
def func(): return myvar
func()

if you wrapped this in def outer(), myvar would be a nonlocal variable,
neither local nor global. exec does not allow for that.

This simple code actually raises an error, because Python is unable
to find the variable myvar from within func.

This is the same as if you indented the above under class xxx:
....

It's only with nested functions, that Python seems to copy
outer-scope locals into inner-scope locals.

I was surprised to find that nonlocals do appear in locals(), but I
would not exactly say that they are copied into the actual local namespace.

def outer():
x = 1
def inner(z=3):
y = 2
print(x)
print(locals())
inner()
print(inner.__code__.co_varnames, inner.__code__.co_freevars)

outer()
#
1
{'x': 1, 'y': 2, 'z': 3}
('z', 'y') ('x',)

Varnames are the local names, freevars are the nonlocal names.

In any case, this complexity requires the presence of an outer function
when the code is compiled. There is no way to simulate it after the fact
by fiddling with just locals and globals when the compiled code is
exec'ed. Class statement, on the other hand, simply introduce a local
namespace separate from the module namespace.