Re: [Pyunit-interest] Avoiding side-effects in differnet tests

[Steve P. <ste...@ya...>]

Hello Jason,

I'm afraid that, as you have discovered, your assumption is not valid.

The setUp and tearDown methods are explicitly responsible for resetting 
global state such that running a particular test will have no effect upon 
the following tests.

There are any number of ways in which a test which fails to honour this 
protocol can interfere with other tests.

The Tkinter GUI for PyUnit goes to some trouble to reload modules each time 
a suite of tests is run so that code changes are picked up and reflected in 
test suite results. However, even this reloading will not absolve the test 
author of the responsibility for cleaning up after test execution.

Only re-starting the interpreter for each test case would guarantee that 
thread-locking issues like yours will not affect test runs, hence the 
requirement that tests programmatically restore the global environment.

Unit tests typically test individual classes, a new instance of which is 
created in setUp() or elsewhere for each test that is run. In your 
situation, your tested unit is a module rather than a class, and the 
equivalent of creating a new instance for each test would be reload()-ing 
the method in each setUp() method. This is somewhat messier, and subject to 
the vagaries of name bindings and import mechanisms.

The most easily testable modules are therefore structured as collections of 
easily testable classes, with a scattering of module-level functions that 
construct and return them; many of the core Python modules are structured 
in this manner.

Best wishes,

-Steve

-- 
Steve Purcell
http://www.pythonconsulting.com/


On Thursday 01 July 2004 09:58, Jason Smith wrote:
> Hi.  I started using pyunit, and I ran into a worrisome problem.  A
> side-effect had occured from a previous test (the module function
> acquired a threading.Lock, then raised an exception without releasing).
>   That is causing a problem in a later test.  Until now, I had assumed
> that each unit test was operating in a clean environment.  Maybe some
> code will help:
>
> class ATest(unittest.TestCase):
> 	def testA(self):
> 		"""This test will cause a side-effect"""
> 		import myBuggyModule
> 		assert myBuggyModule.functionWithSideEffect()
> 	def testB(self):
> 		"""This test will fail becaus of testA"""
> 		import myBuggyModule
> 		assert myBuggyModule.someOtherFunction()
>
> What I had assumed is that by putting the "import myBuggyModule"
> statement in each test, it would get re-evaluated.  But now it looks
> like that is not the case, and Python uses a previously-evaluated module
>   and just imports it into the test method's namespace.
>
> This feels wrong to me, because each test case should run in a pristine
> environment.  That is especially so since the purpose of test cases is
> to cause problems and detect exceptions.  I think if certain module
> globals need to be set, they should be set in the test case, or in the
> setUp() method.
>
> Any comments or advice are appreciated.  Thanks much.