Problem with exec

[Antoon Pardon]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()


When I execute this I get the following error:

Traceback (most recent call last):
File "mod1.py", line 13, in ?
foo()
File "mod1.py", line 10, in foo
exec "assign()" in dct
File "<string>", line 1, in ?
File "mod1.py", line 5, in assign
setattr(config, 'Start' , [13, 26, 29, 34])
NameError: global name 'config' is not defined

Now I don't understand this. In the documentation I read the following:

If only the first expression after in is specified, it should be a
dictionary, which will be used for both the global and the local
variables.

I provided a dictionary to be used for the global variables and it
contains a 'config' entry, so why doesn't this work?

 

[Larry Bates]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()


When I execute this I get the following error:

Traceback (most recent call last):
File "mod1.py", line 13, in ?
foo()
File "mod1.py", line 10, in foo
exec "assign()" in dct
File "<string>", line 1, in ?
File "mod1.py", line 5, in assign
setattr(config, 'Start' , [13, 26, 29, 34])
NameError: global name 'config' is not defined

Now I don't understand this. In the documentation I read the following:

If only the first expression after in is specified, it should be a
dictionary, which will be used for both the global and the local
variables.

I provided a dictionary to be used for the global variables and it
contains a 'config' entry, so why doesn't this work?

Not entirely sure why you want to do what you have outlined
here but this works:

class Cfg:
pass
#config = Cfg()

def assign(config):
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
assign(config)
print config.Start

foo()

You should probably post what you are trying to do. Maybe we
can make a suggestion about the best approach.

-Larry Bates

 

[Peter Otten]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()


When I execute this I get the following error:

Traceback (most recent call last):
File "mod1.py", line 13, in ?
foo()
File "mod1.py", line 10, in foo
exec "assign()" in dct
File "<string>", line 1, in ?
File "mod1.py", line 5, in assign
setattr(config, 'Start' , [13, 26, 29, 34])
NameError: global name 'config' is not defined

Now I don't understand this. In the documentation I read the following:

If only the first expression after in is specified, it should be a
dictionary, which will be used for both the global and the local
variables.

I provided a dictionary to be used for the global variables and it
contains a 'config' entry, so why doesn't this work?

If you have a module

v
def f(): return v


and call f in another module

print f()

where no global variable v is defined, would you expect that call to fail?
Of course not, but how can f look up the variable v then? The function
object keeps a reference of the global namespace it was defined in.
For your example that means the assign() function will look up the config
object in the module's globals(), not in the dictionary you provide to exec

One way to fix your problem would be to define the assign() function inside
the exec'd string.

Peter

 

[Fredrik Lundh]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()

When I execute this I get the following error:

Traceback (most recent call last):
File "mod1.py", line 13, in ?
foo()
File "mod1.py", line 10, in foo
exec "assign()" in dct
File "<string>", line 1, in ?
File "mod1.py", line 5, in assign
setattr(config, 'Start' , [13, 26, 29, 34])
NameError: global name 'config' is not defined

Now I don't understand this.

I thought you were an expert?

Python's lexically scoped, not dynamically scoped. Using a specific
global context for the statement "assign()" doesn't mean that code
called by that statement will suddenly get the same global context.

</F>

 

[Peter Otten]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()


When I execute this I get the following error:

Traceback (most recent call last):
File "mod1.py", line 13, in ?
foo()
File "mod1.py", line 10, in foo
exec "assign()" in dct
File "<string>", line 1, in ?
File "mod1.py", line 5, in assign
setattr(config, 'Start' , [13, 26, 29, 34])
NameError: global name 'config' is not defined

Now I don't understand this. In the documentation I read the following:

If only the first expression after in is specified, it should be a
dictionary, which will be used for both the global and the local
variables.

I provided a dictionary to be used for the global variables and it
contains a 'config' entry, so why doesn't this work?

If you have a module

v = 42
def f(): return v


and call f in another module

print f()

where no global variable v is defined, would you expect that call to fail?
Of course not, but how can f look up the variable v then? The function
object keeps a reference of the global namespace it was defined in.
For your example that means the assign() function will look up the config
object in the module's globals(), not in the dictionary you provide to exec

One way to fix your problem would be to define the assign() function inside
the exec'd string.

Peter

 

[Antoon Pardon]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()


When I execute this I get the following error:

Traceback (most recent call last):
File "mod1.py", line 13, in ?
foo()
File "mod1.py", line 10, in foo
exec "assign()" in dct
File "<string>", line 1, in ?
File "mod1.py", line 5, in assign
setattr(config, 'Start' , [13, 26, 29, 34])
NameError: global name 'config' is not defined

Now I don't understand this. In the documentation I read the following:

If only the first expression after in is specified, it should be a
dictionary, which will be used for both the global and the local
variables.

I provided a dictionary to be used for the global variables and it
contains a 'config' entry, so why doesn't this work?

If you have a module

v = 42
def f(): return v


and call f in another module

print f()

where no global variable v is defined, would you expect that call to fail?
Of course not, but how can f look up the variable v then?

But I am not just calling. I'm using exec. And from the documentation
from exec I get the impression you can use it so that a function
will have temporarily a different reference to global namespace.

 

[Antoon Pardon]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()

[ ... ]

You should probably post what you are trying to do. Maybe we
can make a suggestion about the best approach.

I'm using PLY. The assign function is a dumbded down version
of a production function that will be called during the parsing
of a config file. Each time a line of the form:

var = val

is encounterd I do setattr(config, 'var', val)

The problem is that doing it this way means config needs to be global.
which I'm trying to avoid, in case some leftovers may cause trouble
when I read in a new configuration or should I ever have different
threads parsing files at the same time.

The other way would be passing the 'config' variable around in the
productions, but this would complicate things.

So what I am trying to do was provide a global namespace to the call
to fool a function using a global name into using a provided local name.

 

[Peter Otten]

And from the documentation
from exec I get the impression you can use it so that a function
will have temporarily a different reference to global namespace.

That impression confuses two things:

(1) A function object carries a global namespace with it. That namespace is
fixed when the function definition is executed, not when the function is
called.

(2) You may provide a global namespace to exec. What's in that may be
altered by rebinding operations (=, def, etc.) in the exec't string.
Functions defined here use that namespace as their global namespace while
functions just executed here don't.

If you could provide a function with a different namespace when it's called,
e. g

f() in namespace

would look up its globals in namespace, that might be an interesting concept
but it's not how Python works.

Peter

 

[Peter Otten]

I have the following little piece of code:

class Cfgass
#config = Cfg()

def assign():
setattr(config, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
dct = {'config':config, 'assign':assign}
exec "assign()" in dct
print config.Start

foo()

[ ... ]

You should probably post what you are trying to do. Maybe we
can make a suggestion about the best approach.

I'm using PLY. The assign function is a dumbded down version
of a production function that will be called during the parsing
of a config file. Each time a line of the form:

var = val

is encounterd I do setattr(config, 'var', val)

The problem is that doing it this way means config needs to be global.
which I'm trying to avoid, in case some leftovers may cause trouble
when I read in a new configuration or should I ever have different
threads parsing files at the same time.

The other way would be passing the 'config' variable around in the
productions, but this would complicate things.

So what I am trying to do was provide a global namespace to the call
to fool a function using a global name into using a provided local name.

Maybe you could use a bound method?

class Cfg:
def assign(self):
setattr(self, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
namespace = dict(assign=config.assign)
exec "assign()" in namespace
print config.Start

Peter

 

[Michael Spencer]

If you could provide a function with a different namespace when it's called,
e. g

f() in namespace

would look up its globals in namespace, that might be an interesting concept
but it's not how Python works.

Peter

It does seem like an interesting concept, so I took it as an opportunity to
waste some time ;-)

The 'thunk' function, defined below, turns a function into code that can be
exec'd in another environment:

First, the example:

@thunk
def simple_assigns():
a = 3
b = 4
def square(p):
return p * p

# test in a clean environment:

>>> d = dict(__builtins__ = None)
>>> exec simple_assigns in d
>>> d

>>> d["square"](4)

16



Another example: we can use a thunk to define classes - who needs a class
statement?

def maketype(func):
"""Treat the body of func as a class suite"""
cls = type(func.func_name, (), dict(__builtins__ = None))
c = thunk(func)
exec c in globals(), dwrapper(cls)
return cls

@maketype
def ClassA():
def __init__(self):
self.init = True


Michael

Anyway, here's the function:
#----------- def_hacks.py

"""Silly operations on bytecode"""

from dis import HAVE_ARGUMENT, opmap
from types import CodeType as code

def gendis(co):
"""Yield atomic operations from bytecode"""
coiter = iter(co)
for char in coiter:
op = ord(char)
if op >= HAVE_ARGUMENT:
yield [op, ord(coiter.next()), ord(coiter.next())]
else:
yield [op]

def thunk(func):
"""Hack bytecode so that it uses a real local dictionary rather than
optimized locals"""

out = []
c= func.func_code
codestring = c.co_code


replace_map = {
opmap["STORE_FAST"]: opmap["STORE_NAME"],
opmap["LOAD_FAST"]: opmap["LOAD_NAME"],
opmap["DELETE_FAST"]: opmap["DELETE_NAME"],
}

names_list = list(c.co_names)

# optimized locals are indexed in co_varnames
# non-locals are indexed in co_names
# so when we switch operations, we have to change the
# index variables too

name_map = dict((ix, names_list.index(name))
for ix, name in enumerate(c.co_varnames)
if name in names_list)

for atom in gendis(codestring):
opcode = atom[0]
if opcode in replace_map:
atom[0] = replace_map[opcode]
varindex = atom[1] + 256 * atom[2]
atom[1:] = reversed(divmod(name_map[varindex], 256))
out.append("".join(chr(byte) for byte in atom))

codestring = "".join(out)
# Make a new code object, using most of the properties of the original
# but with new codestring, no arguments, and with flags adjusted
return code(0, #c.co_argcount
c.co_nlocals, c.co_stacksize, c.co_flags-3,
codestring, c.co_consts, c.co_names, c.co_varnames,
c.co_filename, c.co_name, c.co_firstlineno, c.co_lnotab,
c.co_freevars, c.co_cellvars)

 

[Antoon Pardon]

I'm using PLY. The assign function is a dumbded down version
of a production function that will be called during the parsing
of a config file. Each time a line of the form:

var = val

is encounterd I do setattr(config, 'var', val)

The problem is that doing it this way means config needs to be global.
which I'm trying to avoid, in case some leftovers may cause trouble
when I read in a new configuration or should I ever have different
threads parsing files at the same time.

The other way would be passing the 'config' variable around in the
productions, but this would complicate things.

So what I am trying to do was provide a global namespace to the call
to fool a function using a global name into using a provided local name.

Maybe you could use a bound method?

class Cfg:
def assign(self):
setattr(self, 'Start' , [13, 26, 29, 34])

def foo():
config = Cfg()
namespace = dict(assign=config.assign)
exec "assign()" in namespace
print config.Start

Peter

Maybe bound methods will provide a solution, but it won't be so simple
as your suggestion. PLY uses introspection to get at the names of
productions functions. Something like:

def p_assignment(p):

'''assignment : NAME EQUALS list SEMICOL'''

print p[1], p[3]
setattr(Config , p[1] , p[3])

...
yacc.yacc() # This will somehow look into this module and collect
# all functions starting with p_ and together with
# the productions in the doc strings produce a parser.
...
yacc.parse(s) # This wil parse s and in the parse proces call
# p_assignment each time a "assignment" production
# is encountered.


Now when I put the p_ functions in a class the yacc.yacc call no longer
finds them. There may be a solution for this, but it is not in the
documentation, so I'll have to dive into the source.