Replacing module with a stub for unit testing

[pigmartian]

Hi,

I'm working on a unit test framework for a module. The module I'm
testing indirectly calls another module which is expensive to access
--- CDLLs whose functions access a database.

test_MyModule --->MyModule--->IntermediateModule---

ExpensiveModule

I want to create a stub of ExpensiveModule and have that be accessed
by IntermediateModule instead of the real version

test_MyModule --->MyModule--->IntermediateModule---

ExpensiveModuleStub

I tried the following in my unittest:

import ExpensiveModuleStub
sys.modules['ExpensiveModule'] = ExpensiveModuleStub # Doesn't
work

But, import statements in the IntermediateModule still access the real
ExpensiveModule, not the stub.

The examples I can find of creating and using Mock or Stub objects
seem to all follow a pattern where the fake objects are passed in as
arguments to the code being tested. For example, see the "Example
Usage" section here: http://python-mock.sourceforge.net. But that
doesn't work in my case as the module I'm testing doesn't directly use
the module that I want to replace.

Can anybody suggest something?

Thanks,

Scott

 

[Steven D'Aprano]

Hi,

I'm working on a unit test framework for a module. The module I'm
testing indirectly calls another module which is expensive to access ---
CDLLs whose functions access a database. ....
The examples I can find of creating and using Mock or Stub objects seem
to all follow a pattern where the fake objects are passed in as
arguments to the code being tested. For example, see the "Example
Usage" section here: http://python-mock.sourceforge.net. But that
doesn't work in my case as the module I'm testing doesn't directly use
the module that I want to replace.

Can anybody suggest something?

Sounds like a job for monkey-patching!

Currently, you have this:

# inside test_MyModule:
import MyModule
test_code()


# inside MyModule:
import IntermediateModule


# inside IntermediateModule:
import ExpensiveModule


You want to leave MyModule and IntermediateModule as they are, but
replace ExpensiveModule with MockExpensiveModule. Try this:

# inside test_MyModule:
import MyModule
import MockExpensiveModule
MyModule.IntermediateModule.ExpensiveModule = MockExpensiveModule
test_code()



That should work, unless IntermediateModule uses "from ExpensiveModule
import functions" instead of "import ExpensiveModule". In that case, you
will have to monkey-patch each individual object rather than the entire
module:

MyModule.IntermediateModule.afunc = MockExpensiveModule.afunc
MyModule.IntermediateModule.bfunc = MockExpensiveModule.bfunc
MyModule.IntermediateModule.cfunc = MockExpensiveModule.cfunc
# etc...

 

[A. Cavallo]

how about the old and simple:

import ExpensiveModuleStub as ExpensiveModule

On a different league you could make use of decorator and creating caching
objects but that depends entirely on the requirements (how strict your test
must be, test data sizes involved and more, much more details).

Regards,
Antonio

Hi,

I'm working on a unit test framework for a module. The module I'm
testing indirectly calls another module which is expensive to access
--- CDLLs whose functions access a database.

test_MyModule --->MyModule--->IntermediateModule---

ExpensiveModule

I want to create a stub of ExpensiveModule and have that be accessed
by IntermediateModule instead of the real version

test_MyModule --->MyModule--->IntermediateModule---

ExpensiveModuleStub

I tried the following in my unittest:

import ExpensiveModuleStub
sys.modules['ExpensiveModule'] = ExpensiveModuleStub # Doesn't
work

But, import statements in the IntermediateModule still access the real
ExpensiveModule, not the stub.

The examples I can find of creating and using Mock or Stub objects
seem to all follow a pattern where the fake objects are passed in as
arguments to the code being tested. For example, see the "Example
Usage" section here: http://python-mock.sourceforge.net. But that
doesn't work in my case as the module I'm testing doesn't directly use
the module that I want to replace.

Can anybody suggest something?

Thanks,

Scott

 

[Steven D'Aprano]

how about the old and simple:

import ExpensiveModuleStub as ExpensiveModule

No, that won't do, because for it to have the desired effort, it needs to
be inside the IntermediateModule, not the Test_Module. That means that
IntermediateModule needs to know if it is running in "test mode" or "real
mode", and that's (1) possibly impractical, and (2) not great design.

 

[s4g]

Try

import sys
import ExpensiveModuleStub

sys.modules['ExpensiveModule'] = ExpensiveModuleStub
sys.modules['ExpensiveModule'].__name__ = 'ExpensiveModule'

Should do the trick

 

[Fuzzyman]

Hi,

I'm working on a unit test framework for a module.  The module I'm
testing indirectly calls another module which is expensive to access
--- CDLLs whose functions access a database.

    test_MyModule --->MyModule--->IntermediateModule---

ExpensiveModule

I want to create a stub of ExpensiveModule and have that be accessed
by IntermediateModule instead of the real version

    test_MyModule --->MyModule--->IntermediateModule---

ExpensiveModuleStub

I tried the following in my unittest:

    import ExpensiveModuleStub
    sys.modules['ExpensiveModule'] = ExpensiveModuleStub # Doesn't
work

But, import statements in the IntermediateModule still access the real
ExpensiveModule, not the stub.

The examples I can find of creating and using Mock or Stub objects
seem to all follow a pattern where the fake objects are passed in as
arguments to the code being tested.  For example, see the "Example
Usage" section here:http://python-mock.sourceforge.net.  But that
doesn't work in my case as the module I'm testing doesn't directly use
the module that I want to replace.

Can anybody suggest something?

My Mock module, and in particular the patch decorator is designed
explicitly to do this.

You need to be careful because modules and module level globals
(including things your module imports) are global state. If you change
them for the purpose of a test you must *guarantee* to restore them
after the test.

http://www.voidspace.org.uk/python/mock/
http://www.voidspace.org.uk/python/mock/patch.html

In your case if you are testing a module which imports ExpensiveModule
then by ExpensiveModule lives in the *namespace of the module under
test*. You could patch it like this:

from mock import patch
import module

@patch('module.ExpensiveModule)
def testModule(self, mockExpensiveModule):
....

Whilst the test is running 'module' has'ExpensiveModule' mocked out
(replaced) with a Mock instance. This is passed into your test so that
you can setup behaviour and make assertions about how it is used.
After the test is completed the patching is undone.

All the best,


Michael Foord