idea for testing tools

[Jens Theisen]

Hello,

I find it annoying that one has to write

self.assertEqual(x, y)

rather than just

assert x == y

when writing tests. This is a nuisance in all the programming
languages I know of (which are not too many). In Python however, there
appears to be a better alternative. The piece of code below gives the
benefit of printing the violating values in case of a testing failure
as well as the concise syntax:

The snippet

def test_test():
def foo(x):
return x + 3
x = 1
y = 2
assert foo(x) < y + x


try:
test_test()
except AssertionError:
analyse()

would give:

Traceback (most recent call last):
File "./ast-post.py", line 138, in ?
test_test()
File "./ast-post.py", line 134, in test_test
assert foo(x) < y + x
AssertionError

failure analysis:

foo: <function foo at 0xb7c9148c>
x: 1
( x ): 1
foo ( x ): 4
y: 2
x: 1
y + x: 3
foo ( x ) < y + x: False


The code that makes this possible relies only on code present in the
standard library (being traceback, inspect and the parsing stuff)
while being as short as:

#!/usr/bin/python

import sys, types
import traceback, inspect
import parser, symbol, token
import StringIO

def get_inner_frame(tb):
while tb.tb_next:
tb = tb.tb_next
return tb.tb_frame

def visit_ast(visitor, ast):
sym = ast[0]
vals = ast[1:]

assert len(vals) > 0
is_simple = len(vals) == 1
is_leaf = is_simple and type(vals[0]) != types.TupleType

if not is_leaf:
visitor.enter()
for val in vals:
visit_ast(visitor, val)
visitor.leave()

if is_leaf:
visitor.leaf(sym, vals[0])
elif is_simple:
visitor.simple(sym, vals[0])
else:
visitor.compound(sym, vals)


class ast_visitor:
def enter(self):
pass

def leave(self):
pass

def leaf(self, sym, val):
pass

def simple(self, sym, val):
pass

def compound(self, sym, vals):
pass


class simple_printer(ast_visitor):
def __init__(self, stream):
self.stream = stream

def leaf(self, sym, val):
print >>self.stream, val,

def str_from_ast(ast):
s = StringIO.StringIO()
visit_ast(simple_printer(s), ast)
return s.getvalue()

class assertion_collector(ast_visitor):
def __init__(self, statements):
self.statements = statements

def compound(self, sym, vals):
if sym == symbol.assert_stmt:
# two nodes: the "assert" name and the expression
self.statements.append(vals[1])

class pretty_evaluate(ast_visitor):
def __init__(self, globals_, locals_):
self.globals = globals_
self.locals = locals_

def _expr(self, expression):
code = compile(expression, '<internal>', 'eval')

try:
result = eval(code, self.globals, self.locals)
except Exception, e:
result = e

print '%50s: %s' % (expression, str(result))

def compound(self, sym, vals):
ast = [ sym ]
ast.extend(vals)

expression = str_from_ast(ast)

self._expr(expression)

def leaf(self, sym, val):
if sym == token.NAME:
self._expr(val)

def analyse():
type_, exc, tb = sys.exc_info()

frame = get_inner_frame(tb)

try:
filename, line, fun, context, index = (
inspect.getframeinfo(frame, 1)
)

ast = parser.suite(context[0].lstrip()).totuple()

assert_statements = [ ]
visit_ast(assertion_collector(assert_statements), ast)

traceback.print_exc()

print "\nfailure analysis:\n"

for statement in assert_statements:
visit_ast(
pretty_evaluate(frame.f_globals, frame.f_locals), statement)

finally:
del frame

 

[Bruno Desthuilliers]

Jens Theisen a écrit :

Hello,

I find it annoying that one has to write

self.assertEqual(x, y)

rather than just

assert x == y

when writing tests. This is a nuisance in all the programming
languages I know of (which are not too many).

http://codespeak.net/py/current/doc/test.html#assert-with-the-assert-statement

 

[Jens Theisen]

http://codespeak.net/py/current/doc/test.html#assert-with-the-assert-statement

Ok, I didn't come across this before.

I didn't work for me though, even the simple case

#!/usr/bin/python

a = 1
b = 2

def test_some():
assert a == b

didn't reveal the values for a and b, though some more complex cases
showed something.

 

[Eduardo \EdCrypt\ O. Padoan]

#!/usr/bin/python

a = 1
b = 2

def test_some():
assert a == b

didn't reveal the values for a and b, though some more complex cases
showed something.

def test_some():
print 'a:', a, 'b:', b
assert a == b

http://codespeak.net/py/current/doc/test.html#debug-with-the-print-statement

--
EduardoOPadoan (eopadoan->altavix::com)
Bookmarks: http://del.icio.us/edcrypt
Blog: http://edcrypt.blogspot.com
Jabber: edcrypt at jabber dot org
ICQ: 161480283
GTalk: eduardo dot padoan at gmail dot com
MSN: eopadoan at altavix dot com

 

[Paul Rubin]

def test_some():
assert a == b

didn't reveal the values for a and b, though some more complex cases
showed something.

I usually use

assert a == b, (a,b)

 

[Eduardo \EdCrypt\ O. Padoan]

That's hardly desirable. If one is writing a test library that goes as

far as reparsing the assert statements, I can't see the point of
requiring the user to clutter his test suite with such spurious print
statements. After all, that's one of the main points of test suites in
the first place (that's why there is assertEqual).

It will be only be printed when the test fails, along with the rest of
the info. The tests will not be "cluttered" by this litle print.