doctest environment question

[thomas.guest]

I'm not making progress with the following and would appreciate any
help.

Here's an interpreted Python session.

import sys
def f(): pass ....
this_module = sys.modules[__name__]
delattr(this_module, 'f')
f()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

Suppose I want to doctest this behaviour. I cut and paste the above
into a file "test.txt" then run:

python -c "import doctest; doctest.testfile('test.txt')"

This gives me unexpected test failures:

python -c "import doctest; doctest.testfile('test.txt')"
**********************************************************************
File "test.txt", line 5, in test.txt
Failed example:
delattr(this_module, 'f')
Exception raised:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/doctest.py", line 1212, in __run
compileflags, 1) in test.globs
File "<doctest test.txt[3]>", line 1, in <module>
delattr(this_module, 'f')
AttributeError: f
**********************************************************************
File "test.txt", line 6, in test.txt
Failed example:
f()
Expected:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined
Got nothing
**********************************************************************
1 items had failures:
2 of 5 in test.txt
***Test Failed*** 2 failures.

 

[Peter Otten]

I'm not making progress with the following and would appreciate any
help.

Here's an interpreted Python session.

import sys
def f(): pass ...
this_module = sys.modules[__name__]
delattr(this_module, 'f')
f()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

Suppose I want to doctest this behaviour. I cut and paste the above
into a file "test.txt" then run:

python -c "import doctest; doctest.testfile('test.txt')"

This gives me unexpected test failures:

python -c "import doctest; doctest.testfile('test.txt')"
**********************************************************************
File "test.txt", line 5, in test.txt
Failed example:
delattr(this_module, 'f')
Exception raised:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/doctest.py", line 1212, in __run
compileflags, 1) in test.globs
File "<doctest test.txt[3]>", line 1, in <module>
delattr(this_module, 'f')
AttributeError: f
**********************************************************************
File "test.txt", line 6, in test.txt
Failed example:
f()
Expected:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined
Got nothing
**********************************************************************
1 items had failures:
2 of 5 in test.txt
***Test Failed*** 2 failures.

The doctest code is executed in a module without a __name__, it seems.
Unfortunately (in this case) the builtin module serves as a fallback
helping out with its own name:
'__builtin__'


What to do about it? doctest could be changed to execute code snippets in a
module with a name and a sys.modules entry though I don't see much benefit
here.

Peter

 

[tag]

The doctest code is executed in a module without a __name__, it seems.
Unfortunately (in this case) the builtin module serves as a fallback
helping out with its own name:


'__builtin__'

What to do about it? doctest could be changed to execute code snippets in a
module with a name and a sys.modules entry though I don't see much benefit
here.

Peter, thanks for the quick response, but I don't quite understand
what you're saying. I don't want to change doctest -- I want to find a
way to make my example pass using doctest.

doctest.testfile comes with lots of parameters, and I suspect if I
knew how to do it I could pass in the correct parameters to make this
example work. It's not what I actually want to document/test, it's a
minimal example which demonstrates the problem.

Thanks again.

 

[Peter Otten]

Peter, thanks for the quick response, but I don't quite understand
what you're saying. I don't want to change doctest -- I want to find a
way to make my example pass using doctest.

The environment that doctest provides is similar to the interactive
interpreter but not identical. In particular there is no meaningful
__name__.

doctest.testfile comes with lots of parameters, and I suspect if I
knew how to do it I could pass in the correct parameters to make this
example work. It's not what I actually want to document/test, it's a
minimal example which demonstrates the problem.

Here are two alternatives:

(1)Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

(2)

def f(): pass ....
del globals()["f"]
f()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'f' is not defined

If these don't work you'll have to give a bit more context.

Peter

 

[tag]

If these don't work you'll have to give a bit more context.

Peter

Thanks again Peter. Here's something much closer to what I really want
to do. You should be able to cut and paste this post into a file
"post.txt". Running the command `python -c "import doctest;
doctest.testfile('post.txt')"` gives a test failure even though
everything works fine in an interpreted Python session. I'd like to
find a way to make the doctest pass.
.... " Return a function which announces calls to the input
function. "
.... def wrapper(*v, **k):
.... print "Calling %s" % f.__name__
.... return f(*v, **k)
.... return wrapper

We can rebind a function to announce calls to it:
Calling g

Or we can use decorator syntax:
.... def f(): pass
....Calling f

Here's a function which rebinds a function at the top level of a
module (it won't work for nested functions).
.... " Rebind f within a module so that calls to f are announced. "
.... import inspect
.... setattr(inspect.getmodule(f), f.__name__, announce(f))

Let's give it a try. This next works fine in an interactive Python
session but fails when doctested.
Calling h


Here's the doctest failure:

python -c "import doctest; doctest.testfile('post.txt')"
**********************************************************************
File "post.txt", line 37, in post.txt
Failed example:
announce_function(h)
Exception raised:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/doctest.py", line 1212, in __run
compileflags, 1) in test.globs
File "<doctest post.txt[8]>", line 1, in <module>
announce_function(h)
File "<doctest post.txt[6]>", line 4, in announce_function
setattr(inspect.getmodule(f), f.__name__, announce(f))
AttributeError: 'NoneType' object has no attribute 'h'
**********************************************************************
File "post.txt", line 38, in post.txt
Failed example:
h()
Expected:
Calling h
Got nothing
**********************************************************************
1 items had failures:
2 of 10 in post.txt
***Test Failed*** 2 failures.

 

[Peter Otten]

Thanks again Peter. Here's something much closer to what I really want
to do. You should be able to cut and paste this post into a file
"post.txt". Running the command `python -c "import doctest;
doctest.testfile('post.txt')"` gives a test failure even though
everything works fine in an interpreted Python session. I'd like to
find a way to make the doctest pass.

... " Return a function which announces calls to the input
function. "
... def wrapper(*v, **k):
... print "Calling %s" % f.__name__
... return f(*v, **k)
... return wrapper

We can rebind a function to announce calls to it:

Calling g

Or we can use decorator syntax:

... def f(): pass
...
Calling f

Here's a function which rebinds a function at the top level of a
module (it won't work for nested functions).

... " Rebind f within a module so that calls to f are announced. "
... import inspect
... setattr(inspect.getmodule(f), f.__name__, announce(f))

inspect.getmodule(f) returns None because f() is not defined in a module.
You can either move f() to a helper module and then

from helper_module import f

or modify anouncement_function() to not rely on that non-existent module
.... " Rebind f within a module so that calls to f are announced. "
.... f.func_globals[f.__name__] = announce(f)
....

Let's give it a try. This next works fine in an interactive Python
session but fails when doctested.

Calling h

Even when it works, implicitly modifying global variables is bad style.

Peter

 

[tag]

[snip]

inspect.getmodule(f) returns None because f() is not defined in a module.

OK. But there was a module when I ran interactively?

You can either move f() to a helper module and then

from helper_module import f
Yes.

or modify anouncement_function() to not rely on that non-existent module

... " Rebind f within a module so that calls to f are announced. "
... f.func_globals[f.__name__] = announce(f)
...

I think this is what I should be doing. Very nice! You're modifying
f's own gloabl environment.

Even when it works, implicitly modifying global variables is bad style.

I have to admit it didn't feel right, but I didn't come up with the
idea you present. Thanks again for your help.

 

[Peter Otten]

OK. But there was a module when I ran interactively?

Yes. Looking into the doctest source, there is a -- deprecated -- class
called Tester that provides a module. I don't know why this approach was
dropped.

Peter

 

[Gabriel Genellina]

Here's a function which rebinds a function at the top level of a
module (it won't work for nested functions).

... " Rebind f within a module so that calls to f are announced. "
... import inspect
... setattr(inspect.getmodule(f), f.__name__, announce(f))

Let's give it a try. This next works fine in an interactive Python
session but fails when doctested.

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

 

[tag]

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

import announce
import spam
announce.announce_module(spam)

.... code which calls into spam module

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

 

[tag]

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

 

[tag]

The version given by Peter Otten may do what you want, but I'd consider if
you really need an announce_function in the first place, given all the
other ways you already have to do the same thing.
Implicitely rebinding globals does not look good.

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

import announce
import spam
announce.announce_module(spam)

.... code which calls into spam module

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

 

[Gabriel Genellina]

Thanks for the advice Gabriel. The use case I have is that I'd like to
be able to decorate classes and even modules in this way:

import announce
import spam
announce.announce_module(spam)

... code which calls into spam module

Here, "announce.announce_module" has a look in "spam", finds all the
functions and instancemethods, and decorates them (rebinds them) by
announcing calls to these functions and instancemethods.

It's something I've found useful, though clearly the behaviour of
"spam" has been drastically changed. I'd appreciate advice on better
ways to achieve this kind of thing, or why it doesn't look good.

Ah, then you *have* a reference to the containing module. announce_module
might be something like this:

def announce_module(module):
for fname in find_all_functions_by_name_in_module(module):
function = getattr(module, fname)
setattr(module, fname, announce_function(function))

And no need to guess anything. It works even if the original function name
is not the same as the name used in the module (this may happen when you
assign a function with another name).
Remember "Explicit is better than implicit" and "In the face of ambiguity,
refuse the temptation to guess."