Yet another attempt at a safe eval() call

[Chris Angelico]

Someone has already created a module that does this called numexpr. Is
there some reason why you don't want to use that?

array(22L)

Is that from PyPI? It's not in my Python 3.3 installation. Obvious
reason not to use it: Unaware of it.

ChrisA

 

[Oscar Benjamin]

Is that from PyPI? It's not in my Python 3.3 installation. Obvious
reason not to use it: Unaware of it.

My apologies. I should have at least provided a link:
http://code.google.com/p/numexpr/

I installed it from the ubuntu repo under the name python-numexpr. It
is also on PyPI:
http://pypi.python.org/pypi/numexpr

numexpr is a well established project intended primarily for memory
and cache efficient computations over large arrays of data. Possibly
as a side effect, it can also be used to evaluate simple algebraic
expressions involving ordinary scalar variables.


Oscar

 

[matt.newville]

My apologies. I should have at least provided a link:

http://code.google.com/p/numexpr/



I installed it from the ubuntu repo under the name python-numexpr. It

is also on PyPI:

http://pypi.python.org/pypi/numexpr



numexpr is a well established project intended primarily for memory

and cache efficient computations over large arrays of data. Possibly

as a side effect, it can also be used to evaluate simple algebraic

expressions involving ordinary scalar variables.





Oscar

The asteval module http://pypi.python.org/pypi/asteval/0.9 and
http://newville.github.com/asteval/ might be another alternative. It's not as fast as numexpr, but a bit more general. It uses the ast module to "compile" an expression into the AST, then walks through that, intercepting Name nodes and using a flat namespace of variables. It disallows imports anddoes not support all python constructs, but it is a fairly complete in supporting python syntax.

It makes no claim at actually being safe from malicious attack, but should be safer than a straight eval(), and prevent accidental problems when evaluating user-input as code. If anyone can find exploits within it, I'd be happy to try to fix them.

--Matt

 

[matt.newville]

My apologies. I should have at least provided a link:

http://code.google.com/p/numexpr/



I installed it from the ubuntu repo under the name python-numexpr. It

is also on PyPI:

http://pypi.python.org/pypi/numexpr



numexpr is a well established project intended primarily for memory

and cache efficient computations over large arrays of data. Possibly

as a side effect, it can also be used to evaluate simple algebraic

expressions involving ordinary scalar variables.





Oscar

The asteval module http://pypi.python.org/pypi/asteval/0.9 and
http://newville.github.com/asteval/ might be another alternative. It's not as fast as numexpr, but a bit more general. It uses the ast module to "compile" an expression into the AST, then walks through that, intercepting Name nodes and using a flat namespace of variables. It disallows imports anddoes not support all python constructs, but it is a fairly complete in supporting python syntax.

It makes no claim at actually being safe from malicious attack, but should be safer than a straight eval(), and prevent accidental problems when evaluating user-input as code. If anyone can find exploits within it, I'd be happy to try to fix them.

--Matt

 

[Grant Edwards]

Someone has already created a module that does this called numexpr. Is
there some reason why you don't want to use that?

1) I didn't know about it, and my Googling didn't find it.

2) It's not part of the standard library, and my program needs to be
distributed as a single source file.

 

[Oscar Benjamin]

1) I didn't know about it, and my Googling didn't find it.

2) It's not part of the standard library, and my program needs to be
distributed as a single source file.

That's an unfortunate restriction. It also won't be possible to reuse
the code from numexpr (for technical rather than legal reasons).
Perhaps asteval will be more helpful in that sense.

Otherwise presumably the shunting-yard algorithm comes out a little
nicer in Python than in C (it would be useful if something like this
were available on PyPI as a pure Python module):
http://en.wikipedia.org/wiki/Shunting_yard_algorithm#C_example


Oscar

 

[Ken Seehart]

I've written a small assembler in Python 2.[67], and it needs to
evaluate integer-valued arithmetic expressions in the context of a
symbol table that defines integer values for a set of names. The
"right" thing is probably an expression parser/evaluator using ast, but
it looked like that would take more code that the rest of the assembler
combined, and I've got other higher-priority tasks to get back to.

Will ast.literal_eval do what you want?

No. Grant needs to support variables, not just literal constants, hence
the symbol table.

Apologies for the delayed response...

Seems like it would be a bit safer and easier to approach this problem
by stretching the capability of ast.literal_eval() rather than
attempting to sandbox eval().

How about ast.literal_eval after performing lexical substitution using
the symbol table?

Assignment into the symbol table, and error handling, are exercises left
to the reader.

Something vaguely like this:

/pseudocode:/

def safe_eval(s, symbols={}):
while search(s, r'\w+'):
replace match with '('+repr(symbols[match])+')' in s
return ast.literal_eval(s)

- Ken