Problem with exec

Discussion in 'Python' started by Antoon Pardon, Dec 16, 2005.

  1. I have the following little piece of code:

    class Cfg:pass
    #config = Cfg()

    def assign():
    setattr(config, 'Start' , [13, 26, 29, 34])

    def foo():
    config = Cfg()
    dct = {'config':config, 'assign':assign}
    exec "assign()" in dct
    print config.Start

    foo()


    When I execute this I get the following error:

    Traceback (most recent call last):
    File "mod1.py", line 13, in ?
    foo()
    File "mod1.py", line 10, in foo
    exec "assign()" in dct
    File "<string>", line 1, in ?
    File "mod1.py", line 5, in assign
    setattr(config, 'Start' , [13, 26, 29, 34])
    NameError: global name 'config' is not defined

    Now I don't understand this. In the documentation I read the following:

    If only the first expression after in is specified, it should be a
    dictionary, which will be used for both the global and the local
    variables.

    I provided a dictionary to be used for the global variables and it
    contains a 'config' entry, so why doesn't this work?

    --
    Antoon Pardon
     
    Antoon Pardon, Dec 16, 2005
    #1
    1. Advertising

  2. Antoon Pardon

    Larry Bates Guest

    Antoon Pardon wrote:
    > I have the following little piece of code:
    >
    > class Cfg:pass
    > #config = Cfg()
    >
    > def assign():
    > setattr(config, 'Start' , [13, 26, 29, 34])
    >
    > def foo():
    > config = Cfg()
    > dct = {'config':config, 'assign':assign}
    > exec "assign()" in dct
    > print config.Start
    >
    > foo()
    >
    >
    > When I execute this I get the following error:
    >
    > Traceback (most recent call last):
    > File "mod1.py", line 13, in ?
    > foo()
    > File "mod1.py", line 10, in foo
    > exec "assign()" in dct
    > File "<string>", line 1, in ?
    > File "mod1.py", line 5, in assign
    > setattr(config, 'Start' , [13, 26, 29, 34])
    > NameError: global name 'config' is not defined
    >
    > Now I don't understand this. In the documentation I read the following:
    >
    > If only the first expression after in is specified, it should be a
    > dictionary, which will be used for both the global and the local
    > variables.
    >
    > I provided a dictionary to be used for the global variables and it
    > contains a 'config' entry, so why doesn't this work?
    >


    Not entirely sure why you want to do what you have outlined
    here but this works:

    class Cfg:
    pass
    #config = Cfg()

    def assign(config):
    setattr(config, 'Start' , [13, 26, 29, 34])

    def foo():
    config = Cfg()
    assign(config)
    print config.Start

    foo()

    You should probably post what you are trying to do. Maybe we
    can make a suggestion about the best approach.

    -Larry Bates
     
    Larry Bates, Dec 16, 2005
    #2
    1. Advertising

  3. Antoon Pardon

    Peter Otten Guest

    Antoon Pardon wrote:

    > I have the following little piece of code:
    >
    > class Cfg:pass
    > #config = Cfg()
    >
    > def assign():
    > setattr(config, 'Start' , [13, 26, 29, 34])
    >
    > def foo():
    > config = Cfg()
    > dct = {'config':config, 'assign':assign}
    > exec "assign()" in dct
    > print config.Start
    >
    > foo()
    >
    >
    > When I execute this I get the following error:
    >
    > Traceback (most recent call last):
    > File "mod1.py", line 13, in ?
    > foo()
    > File "mod1.py", line 10, in foo
    > exec "assign()" in dct
    > File "<string>", line 1, in ?
    > File "mod1.py", line 5, in assign
    > setattr(config, 'Start' , [13, 26, 29, 34])
    > NameError: global name 'config' is not defined
    >
    > Now I don't understand this. In the documentation I read the following:
    >
    > If only the first expression after in is specified, it should be a
    > dictionary, which will be used for both the global and the local
    > variables.
    >
    > I provided a dictionary to be used for the global variables and it
    > contains a 'config' entry, so why doesn't this work?



    If you have a module

    v
    def f(): return v


    and call f in another module

    print f()

    where no global variable v is defined, would you expect that call to fail?
    Of course not, but how can f look up the variable v then? The function
    object keeps a reference of the global namespace it was defined in.
    For your example that means the assign() function will look up the config
    object in the module's globals(), not in the dictionary you provide to exec

    One way to fix your problem would be to define the assign() function inside
    the exec'd string.

    Peter
     
    Peter Otten, Dec 16, 2005
    #3
  4. Antoon Pardon wrote:

    > I have the following little piece of code:
    >
    > class Cfg:pass
    > #config = Cfg()
    >
    > def assign():
    > setattr(config, 'Start' , [13, 26, 29, 34])
    >
    > def foo():
    > config = Cfg()
    > dct = {'config':config, 'assign':assign}
    > exec "assign()" in dct
    > print config.Start
    >
    > foo()
    >
    > When I execute this I get the following error:
    >
    > Traceback (most recent call last):
    > File "mod1.py", line 13, in ?
    > foo()
    > File "mod1.py", line 10, in foo
    > exec "assign()" in dct
    > File "<string>", line 1, in ?
    > File "mod1.py", line 5, in assign
    > setattr(config, 'Start' , [13, 26, 29, 34])
    > NameError: global name 'config' is not defined
    >
    > Now I don't understand this.


    I thought you were an expert?

    Python's lexically scoped, not dynamically scoped. Using a specific
    global context for the statement "assign()" doesn't mean that code
    called by that statement will suddenly get the same global context.

    </F>
     
    Fredrik Lundh, Dec 16, 2005
    #4
  5. Antoon Pardon

    Peter Otten Guest

    Antoon Pardon wrote:

    > I have the following little piece of code:
    >
    > class Cfg:pass
    > #config = Cfg()
    >
    > def assign():
    > setattr(config, 'Start' , [13, 26, 29, 34])
    >
    > def foo():
    > config = Cfg()
    > dct = {'config':config, 'assign':assign}
    > exec "assign()" in dct
    > print config.Start
    >
    > foo()
    >
    >
    > When I execute this I get the following error:
    >
    > Traceback (most recent call last):
    > File "mod1.py", line 13, in ?
    > foo()
    > File "mod1.py", line 10, in foo
    > exec "assign()" in dct
    > File "<string>", line 1, in ?
    > File "mod1.py", line 5, in assign
    > setattr(config, 'Start' , [13, 26, 29, 34])
    > NameError: global name 'config' is not defined
    >
    > Now I don't understand this. In the documentation I read the following:
    >
    > If only the first expression after in is specified, it should be a
    > dictionary, which will be used for both the global and the local
    > variables.
    >
    > I provided a dictionary to be used for the global variables and it
    > contains a 'config' entry, so why doesn't this work?



    If you have a module

    v = 42
    def f(): return v


    and call f in another module

    print f()

    where no global variable v is defined, would you expect that call to fail?
    Of course not, but how can f look up the variable v then? The function
    object keeps a reference of the global namespace it was defined in.
    For your example that means the assign() function will look up the config
    object in the module's globals(), not in the dictionary you provide to exec

    One way to fix your problem would be to define the assign() function inside
    the exec'd string.

    Peter
     
    Peter Otten, Dec 16, 2005
    #5
  6. Op 2005-12-16, Peter Otten schreef <>:
    > Antoon Pardon wrote:
    >
    >> I have the following little piece of code:
    >>
    >> class Cfg:pass
    >> #config = Cfg()
    >>
    >> def assign():
    >> setattr(config, 'Start' , [13, 26, 29, 34])
    >>
    >> def foo():
    >> config = Cfg()
    >> dct = {'config':config, 'assign':assign}
    >> exec "assign()" in dct
    >> print config.Start
    >>
    >> foo()
    >>
    >>
    >> When I execute this I get the following error:
    >>
    >> Traceback (most recent call last):
    >> File "mod1.py", line 13, in ?
    >> foo()
    >> File "mod1.py", line 10, in foo
    >> exec "assign()" in dct
    >> File "<string>", line 1, in ?
    >> File "mod1.py", line 5, in assign
    >> setattr(config, 'Start' , [13, 26, 29, 34])
    >> NameError: global name 'config' is not defined
    >>
    >> Now I don't understand this. In the documentation I read the following:
    >>
    >> If only the first expression after in is specified, it should be a
    >> dictionary, which will be used for both the global and the local
    >> variables.
    >>
    >> I provided a dictionary to be used for the global variables and it
    >> contains a 'config' entry, so why doesn't this work?

    >
    >
    > If you have a module
    >
    > v = 42
    > def f(): return v
    >
    >
    > and call f in another module
    >
    > print f()
    >
    > where no global variable v is defined, would you expect that call to fail?
    > Of course not, but how can f look up the variable v then?


    But I am not just calling. I'm using exec. And from the documentation
    from exec I get the impression you can use it so that a function
    will have temporarily a different reference to global namespace.

    --
    Antoon Pardon
     
    Antoon Pardon, Dec 16, 2005
    #6
  7. Op 2005-12-16, Larry Bates schreef <>:
    > Antoon Pardon wrote:
    >> I have the following little piece of code:
    >>
    >> class Cfg:pass
    >> #config = Cfg()
    >>
    >> def assign():
    >> setattr(config, 'Start' , [13, 26, 29, 34])
    >>
    >> def foo():
    >> config = Cfg()
    >> dct = {'config':config, 'assign':assign}
    >> exec "assign()" in dct
    >> print config.Start
    >>
    >> foo()


    > [ ... ]
    >
    > You should probably post what you are trying to do. Maybe we
    > can make a suggestion about the best approach.


    I'm using PLY. The assign function is a dumbded down version
    of a production function that will be called during the parsing
    of a config file. Each time a line of the form:

    var = val

    is encounterd I do setattr(config, 'var', val)

    The problem is that doing it this way means config needs to be global.
    which I'm trying to avoid, in case some leftovers may cause trouble
    when I read in a new configuration or should I ever have different
    threads parsing files at the same time.

    The other way would be passing the 'config' variable around in the
    productions, but this would complicate things.

    So what I am trying to do was provide a global namespace to the call
    to fool a function using a global name into using a provided local name.

    --
    Antoon Pardon
     
    Antoon Pardon, Dec 16, 2005
    #7
  8. Antoon Pardon

    Peter Otten Guest

    Antoon Pardon wrote:

    > And from the documentation
    > from exec I get the impression you can use it so that a function
    > will have temporarily a different reference to global namespace.


    That impression confuses two things:

    (1) A function object carries a global namespace with it. That namespace is
    fixed when the function definition is executed, not when the function is
    called.

    (2) You may provide a global namespace to exec. What's in that may be
    altered by rebinding operations (=, def, etc.) in the exec't string.
    Functions defined here use that namespace as their global namespace while
    functions just executed here don't.

    If you could provide a function with a different namespace when it's called,
    e. g

    f() in namespace

    would look up its globals in namespace, that might be an interesting concept
    but it's not how Python works.

    Peter
     
    Peter Otten, Dec 16, 2005
    #8
  9. Antoon Pardon

    Peter Otten Guest

    Antoon Pardon wrote:

    > Op 2005-12-16, Larry Bates schreef <>:
    >> Antoon Pardon wrote:
    >>> I have the following little piece of code:
    >>>
    >>> class Cfg:pass
    >>> #config = Cfg()
    >>>
    >>> def assign():
    >>> setattr(config, 'Start' , [13, 26, 29, 34])
    >>>
    >>> def foo():
    >>> config = Cfg()
    >>> dct = {'config':config, 'assign':assign}
    >>> exec "assign()" in dct
    >>> print config.Start
    >>>
    >>> foo()

    >
    >> [ ... ]
    >>
    >> You should probably post what you are trying to do. Maybe we
    >> can make a suggestion about the best approach.

    >
    > I'm using PLY. The assign function is a dumbded down version
    > of a production function that will be called during the parsing
    > of a config file. Each time a line of the form:
    >
    > var = val
    >
    > is encounterd I do setattr(config, 'var', val)
    >
    > The problem is that doing it this way means config needs to be global.
    > which I'm trying to avoid, in case some leftovers may cause trouble
    > when I read in a new configuration or should I ever have different
    > threads parsing files at the same time.
    >
    > The other way would be passing the 'config' variable around in the
    > productions, but this would complicate things.
    >
    > So what I am trying to do was provide a global namespace to the call
    > to fool a function using a global name into using a provided local name.



    Maybe you could use a bound method?

    class Cfg:
    def assign(self):
    setattr(self, 'Start' , [13, 26, 29, 34])

    def foo():
    config = Cfg()
    namespace = dict(assign=config.assign)
    exec "assign()" in namespace
    print config.Start

    Peter
     
    Peter Otten, Dec 16, 2005
    #9
  10. Peter Otten wrote:

    >
    > If you could provide a function with a different namespace when it's called,
    > e. g
    >
    > f() in namespace
    >
    > would look up its globals in namespace, that might be an interesting concept
    > but it's not how Python works.
    >
    > Peter
    >

    It does seem like an interesting concept, so I took it as an opportunity to
    waste some time ;-)

    The 'thunk' function, defined below, turns a function into code that can be
    exec'd in another environment:

    First, the example:

    @thunk
    def simple_assigns():
    a = 3
    b = 4
    def square(p):
    return p * p

    # test in a clean environment:
    >>> d = dict(__builtins__ = None)
    >>> exec simple_assigns in d
    >>> d

    {'__builtins__': None, 'a': 3, 'b': 4, 'square': <function square at 0x017A09B0>}
    >>> d["square"](4)

    16



    Another example: we can use a thunk to define classes - who needs a class
    statement?

    def maketype(func):
    """Treat the body of func as a class suite"""
    cls = type(func.func_name, (), dict(__builtins__ = None))
    c = thunk(func)
    exec c in globals(), dwrapper(cls)
    return cls

    @maketype
    def ClassA():
    def __init__(self):
    self.init = True

    >>> a = ClassA()
    >>> a

    <def_hacks.ClassA object at 0x019AE810>
    >>> a.init

    True
    >>>



    Michael

    Anyway, here's the function:
    #----------- def_hacks.py

    """Silly operations on bytecode"""

    from dis import HAVE_ARGUMENT, opmap
    from types import CodeType as code

    def gendis(co):
    """Yield atomic operations from bytecode"""
    coiter = iter(co)
    for char in coiter:
    op = ord(char)
    if op >= HAVE_ARGUMENT:
    yield [op, ord(coiter.next()), ord(coiter.next())]
    else:
    yield [op]

    def thunk(func):
    """Hack bytecode so that it uses a real local dictionary rather than
    optimized locals"""

    out = []
    c= func.func_code
    codestring = c.co_code


    replace_map = {
    opmap["STORE_FAST"]: opmap["STORE_NAME"],
    opmap["LOAD_FAST"]: opmap["LOAD_NAME"],
    opmap["DELETE_FAST"]: opmap["DELETE_NAME"],
    }

    names_list = list(c.co_names)

    # optimized locals are indexed in co_varnames
    # non-locals are indexed in co_names
    # so when we switch operations, we have to change the
    # index variables too

    name_map = dict((ix, names_list.index(name))
    for ix, name in enumerate(c.co_varnames)
    if name in names_list)

    for atom in gendis(codestring):
    opcode = atom[0]
    if opcode in replace_map:
    atom[0] = replace_map[opcode]
    varindex = atom[1] + 256 * atom[2]
    atom[1:] = reversed(divmod(name_map[varindex], 256))
    out.append("".join(chr(byte) for byte in atom))

    codestring = "".join(out)
    # Make a new code object, using most of the properties of the original
    # but with new codestring, no arguments, and with flags adjusted
    return code(0, #c.co_argcount
    c.co_nlocals, c.co_stacksize, c.co_flags-3,
    codestring, c.co_consts, c.co_names, c.co_varnames,
    c.co_filename, c.co_name, c.co_firstlineno, c.co_lnotab,
    c.co_freevars, c.co_cellvars)
     
    Michael Spencer, Dec 16, 2005
    #10
  11. Op 2005-12-16, Peter Otten schreef <>:
    > Antoon Pardon wrote:
    >
    >> I'm using PLY. The assign function is a dumbded down version
    >> of a production function that will be called during the parsing
    >> of a config file. Each time a line of the form:
    >>
    >> var = val
    >>
    >> is encounterd I do setattr(config, 'var', val)
    >>
    >> The problem is that doing it this way means config needs to be global.
    >> which I'm trying to avoid, in case some leftovers may cause trouble
    >> when I read in a new configuration or should I ever have different
    >> threads parsing files at the same time.
    >>
    >> The other way would be passing the 'config' variable around in the
    >> productions, but this would complicate things.
    >>
    >> So what I am trying to do was provide a global namespace to the call
    >> to fool a function using a global name into using a provided local name.

    >
    >
    > Maybe you could use a bound method?
    >
    > class Cfg:
    > def assign(self):
    > setattr(self, 'Start' , [13, 26, 29, 34])
    >
    > def foo():
    > config = Cfg()
    > namespace = dict(assign=config.assign)
    > exec "assign()" in namespace
    > print config.Start
    >
    > Peter


    Maybe bound methods will provide a solution, but it won't be so simple
    as your suggestion. PLY uses introspection to get at the names of
    productions functions. Something like:

    def p_assignment(p):

    '''assignment : NAME EQUALS list SEMICOL'''

    print p[1], p[3]
    setattr(Config , p[1] , p[3])

    ...
    yacc.yacc() # This will somehow look into this module and collect
    # all functions starting with p_ and together with
    # the productions in the doc strings produce a parser.
    ...
    yacc.parse(s) # This wil parse s and in the parse proces call
    # p_assignment each time a "assignment" production
    # is encountered.


    Now when I put the p_ functions in a class the yacc.yacc call no longer
    finds them. There may be a solution for this, but it is not in the
    documentation, so I'll have to dive into the source.

    --
    Antoon Pardon
     
    Antoon Pardon, Dec 19, 2005
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hal Vaughan
    Replies:
    11
    Views:
    1,155
    Gordon Beaton
    May 22, 2006
  2. tedsuzman
    Replies:
    2
    Views:
    7,124
    Michel Claveau, résurectionné d'outre-bombe inform
    Jul 21, 2004
  3. Ted
    Replies:
    1
    Views:
    478
    Duncan Booth
    Jul 22, 2004
  4. Guillermo Riojas
    Replies:
    0
    Views:
    184
    Guillermo Riojas
    Nov 26, 2010
  5. Random Task
    Replies:
    12
    Views:
    674
    Joe Smith
    Dec 4, 2005
Loading...

Share This Page