replacement of rexec?

[Huaiyu Zhu]

What is the prefered way to eval a string like "('a', 1)"? These strings
are representations of simple objects of type int, str, or dict, tuple or
list made of them. I do not want to use naked eval in case I run the
script on a wrong file that happens to contain Python code.

Previously I used the following

from rexec import RExec
r_eval = RExec().r_eval

After upgrading to 2.3 it no longer works:

File "/usr/local/lib/python2.3/rexec.py", line 184, in __init__
raise RuntimeError, "This code is not secure in Python 2.2 and 2.3"
RuntimeError: This code is not secure in Python 2.2 and 2.3

I understand that rexec is not secure against malicious attacks. However,
it is safer than eval for preventing accidents. What should I use in
place? The ideal thing I'm looking for is a function that can parse only
"non-active" python objects and raise exception on anything else.

Huaiyu

 

[Paul Clinch]

You could use the new compiler module to parse and inspect the result like this:

import compiler as cplr
mod = cplr.parseFile("filetoload")
for node in mod.node.nodes:
if 'Function' in str(node):
break

etc..
There are probably more accurate ways of determining that the file is what
you expect.

Otherwise you are back to manually parsing the file as text.

Hope this helps, Paul Clinch

 

[Huaiyu Zhu]

You could use the new compiler module to parse and inspect the result like this:

import compiler as cplr
mod = cplr.parseFile("filetoload")
for node in mod.node.nodes:
if 'Function' in str(node):
break

Thanks for the suggestion. Is there existing code to assemble the parsed
tree back to the intended Python object? It's probably not too hard to
write one, but I wonder if this is already done?

Huaiyu

 

[Skip Montanaro]

Huaiyu> Thanks for the suggestion. Is there existing code to assemble
Huaiyu> the parsed tree back to the intended Python object? It's
Huaiyu> probably not too hard to write one, but I wonder if this is
Huaiyu> already done?

I'm not aware of anything. Assuming your inspection process shows the file
is okay, why not just call execfile()?

Skip

 

[Evan Simpson]

Huaiyu> Thanks for the suggestion. Is there existing code to assemble
Huaiyu> the parsed tree back to the intended Python object?

I'm not aware of anything. Assuming your inspection process shows the file
is okay, why not just call execfile()?

compiler.pycodegen and new.function or exec can be used to make a code
object and execute it. As Skip says, though, this is pointless unless
you are mutating the AST, as Zope's RestrictedPython package does.

Cheers,

Evan @ 4-am

 

[Jeremy Fincher]

I'm not aware of anything. Assuming your inspection process shows the file
is okay, why not just call execfile()?

Probably because that won't get him a useful value He'd have to
use eval (but you probably knew that and just typo'ed execfile; I'm
mostly just saying it to try and prevent confusion on his part).

Jeremy

 

[Ian McMeans]

Are there plans to replace rexec? It seems like a useful library to
have (Zope is a good example of where it would be useful)

 

[Huaiyu Zhu]

Probably because that won't get him a useful value He'd have to
use eval (but you probably knew that and just typo'ed execfile; I'm
mostly just saying it to try and prevent confusion on his part).

That's the point - the file contains multiple strings to be converted to
python objects. I've written the following code over the weekend. It is
more complicated than I'd like. Hopefully someone can replace part or all
of it by some builtin functionality.

#-------------------------------------------------------
import compiler
def safe_eval(s):
"""
Evaluate strings that only contains the following structures:
const, tuple, list, dict
"""
stmts = compiler.parse(s).node.nodes
#print stmts
assert len(stmts) == 1
node = compiler.parse(s).node.nodes[0]
assert node.__class__ == compiler.ast.Discard
nodes = node.getChildNodes()
assert len(nodes) == 1
return safe_assemble(nodes[0])

seq_types = {
compiler.ast.Tuple: tuple,
compiler.ast.List: list,
}
map_types = {
compiler.ast.Dict: dict,
}

def safe_assemble(node):
""" Recursively assemble parsed ast node """
cls = node.__class__
if cls == compiler.ast.Const:
return node.value
elif cls in seq_types:
nodes = node.nodes
args = map(safe_assemble, nodes)
return seq_types[cls](args)
elif cls in map_types:
keys, values = zip(*node.items)
keys = map(safe_assemble, keys)
values = map(safe_assemble, values)
return map_types[cls](zip(keys, values))
else:
raise "Type not allowed", cls


if __name__ == '__main__':
tests = [
"3j",
"1, 2.5",
"['abc', 0xF7]",
"1, ['abc', [2,3]], {(4,5):[6.5, 8]}",
]
for s in tests:
print safe_eval(s)
#-------------------------------------------------------


Huaiyu

 

[John Roth]

This looks rather interesting. I frankly think we need it
as a built-in.

John Roth

Skip Montanaro <[email protected]> wrote in message


Probably because that won't get him a useful value He'd have to
use eval (but you probably knew that and just typo'ed execfile; I'm
mostly just saying it to try and prevent confusion on his part).

That's the point - the file contains multiple strings to be converted to
python objects. I've written the following code over the weekend. It is
more complicated than I'd like. Hopefully someone can replace part or all
of it by some builtin functionality.

#-------------------------------------------------------
import compiler
def safe_eval(s):
"""
Evaluate strings that only contains the following structures:
const, tuple, list, dict
"""
stmts = compiler.parse(s).node.nodes
#print stmts
assert len(stmts) == 1
node = compiler.parse(s).node.nodes[0]
assert node.__class__ == compiler.ast.Discard
nodes = node.getChildNodes()
assert len(nodes) == 1
return safe_assemble(nodes[0])

seq_types = {
compiler.ast.Tuple: tuple,
compiler.ast.List: list,
}
map_types = {
compiler.ast.Dict: dict,
}

def safe_assemble(node):
""" Recursively assemble parsed ast node """
cls = node.__class__
if cls == compiler.ast.Const:
return node.value
elif cls in seq_types:
nodes = node.nodes
args = map(safe_assemble, nodes)
return seq_types[cls](args)
elif cls in map_types:
keys, values = zip(*node.items)
keys = map(safe_assemble, keys)
values = map(safe_assemble, values)
return map_types[cls](zip(keys, values))
else:
raise "Type not allowed", cls


if __name__ == '__main__':
tests = [
"3j",
"1, 2.5",
"['abc', 0xF7]",
"1, ['abc', [2,3]], {(4,5):[6.5, 8]}",
]
for s in tests:
print safe_eval(s)
#-------------------------------------------------------


Huaiyu

 

[John J. Lee]

[Haiyu Zhu wrote:]

def safe_eval(s):
"""
Evaluate strings that only contains the following structures:
const, tuple, list, dict
"""

This looks rather interesting. I frankly think we need it
as a built-in.

[...]

Not a builtin, necessarily, but in the standard library would be nice.

I've written something like this before, only more ad-hoc, and wished
there was something like this available.


John