pyunit-interest

[Pyunit-interest] Feature Request: Cleanup Registry

[Jim F. <ji...@zo...>]

Problem

   the Zope 3 project is making extensive use of PyUnit.  Zope 3 unit
   tests often have to register components with various component
   registries. These registrations need to be torn down after each
   test. This became very tedious. For this reason, we've implemented a
   cleanup registry. Any stateful global objects, like component
   registries register functions with this cleanup registry. Test
   classes provide tearDown methods that call a method on the cleanup
   registry, which calls all the registered functions.  This is an
   extremely useful pattern that might be beneficial to other
   projects. Further, it would be nice if test classes didn't have to
   provide a tear-down method that just called the cleanup registry (or
   mix-in a class that provided such a tearDown.

Proposal

   Add a cleanup registry to the unittest module.  The unittest module
   would provide an additional function, addCleanUp, which takes a
   callable object to be called without arguments after running each
   unit test. After each test, and after running the test class's
   tearDown method, each registered cleanup callable will be called.

   Suppose there was an application module with a global dictionary
   that was manipulated by application routines:

      TheData = {}

   The module would register the clear method of this dictionary with
   unittest.addCleanUp to make sure the dictionary was reinitialized
   after each test:

      import unittest
      unittest.addCleanUp(TheData.clear)

I'd be happy to provide a patch.

Jim

-- 
Jim Fulton           mailto:ji...@zo...       Python Powered!
CTO                  (888) 344-4332            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

Re: [Pyunit-interest] Feature Request: Cleanup Registry

[Ype K. <yk...@xs...>]

Jim,

I made a similar proposal some time ago. Below is the reply
of Steve Purcell. He convinced me.

Jim wrote:
>Problem
>
>  the Zope 3 project is making extensive use of PyUnit.  Zope 3 unit
>  tests often have to register components with various component
>  registries. These registrations need to be torn down after each
>  test. This became very tedious. For this reason, we've implemented a
>  cleanup registry. Any stateful global objects, like component
>  registries register functions with this cleanup registry. Test
>  classes provide tearDown methods that call a method on the cleanup
>  registry, which calls all the registered functions.  This is an
>  extremely useful pattern that might be beneficial to other
>  projects. Further, it would be nice if test classes didn't have to
>  provide a tear-down method that just called the cleanup registry (or
>  mix-in a class that provided such a tearDown.
>
>Proposal
>
>  Add a cleanup registry to the unittest module.  The unittest module
>  would provide an additional function, addCleanUp, which takes a
>  callable object to be called without arguments after running each
>  unit test. After each test, and after running the test class's
>  tearDown method, each registered cleanup callable will be called.
>
>  Suppose there was an application module with a global dictionary
>  that was manipulated by application routines:
>
>     TheData = {}
>
>  The module would register the clear method of this dictionary with
>  unittest.addCleanUp to make sure the dictionary was reinitialized
>  after each test:
>
>     import unittest
>     unittest.addCleanUp(TheData.clear)
>
>I'd be happy to provide a patch.


This is Steve's reply to my post:

Date: Sun, 23 Sep 2001 13:38:09 +0200
From: Steve Purcell <ste...@ya...>
To: Ype Kingma <yk...@xs...>
Cc: pyu...@li...

Subject: Re: [Pyunit-interest] setup/teardown for a TestSuite


Hi Ype,

Previous discussion on this list of this topic may be hidden behind obscure
subject lines; you're not the first to suggest a setUp and tearDown for
TestSuite.

However, this defeats one of the central ideas of the framework, which is
that each test should be able to run either in isolation or in arbitrary
combination with other tests. That is, each test case should provide for the
setting up and tidying up of the environment in which it runs.

Test cases need not even run as part of a suite:

   class MyTestCase(unittest.TestCase):
       def testSomething(self): pass

   unittest.TextTestRunner().run(MyTestCase())

If the system must be placed into some state before a number of tests are
run, and if that state is certain to be unaffected by the tests, there are
a couple of strategies.

Firstly, each test case could be given a 'setUp' method that lazily sets up
that state, possibly using a helper module.

Secondly, the environment could be set up before the TestRunner is invoked
(or 'unittest.main()' called).

It's worth going to some trouble to make each TestCase outwardly independent
of other code.

Best wishes,

-Steve




Ype Kingma wrote:
> Hello pyunit'ers,
>
> I recently started using unittest.py, and it serves me well.
> At some point I needed to setUp and tearDown a TestSuite.
> The pattern is:
> - bring a system into an interesting state,
> - perform some testcases on the system,
> - bring the system back into it's original state.
> This is useful when setup and/or teardown
> are non trivial and there are more than a few testcases.
>
> I checked the archives a bit but there did not seem to be
> any discussion on this.
>
> So I subclassed SetUpTestSuite from TestSuite. I added methods
> setUpSuite() and tearDownSuite(), called these from the
> __call__() method and allowed for exceptions during setup/teardown
> being reported as test errors.
>
> Having written the code I realised I could have done it by
> implementing a TestCase to execute a TestSuite.
>
> Another look at the code made me realise that it might be added
> into TestSuite without introducing too many backward incompatibilities,
> after all I guess that TestSuite isn't subclassed very often.
>
> The working code is below, comments invited, please use it
> as you see fit.
>
> Ype


--
Steve Purcell, Pythangelist
Get testing at http://pyunit.sourceforge.net/
GPG fingerprint: EB79 2AB1 D494 16F9 8C3B  DAC6 5341 30CE BCCA EBEA
-- 

Re: [Pyunit-interest] Feature Request: Cleanup Registry

[Jim F. <ji...@zo...>]

My request and yours are not all that similar. I'd still like a response
to *my* request.

Jim

-- 
Jim Fulton           mailto:ji...@zo...       Python Powered!
CTO                  (888) 344-4332            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

Re: [Pyunit-interest] Feature Request: Cleanup Registry

[Ype K. <yk...@xs...>]

Jim,

you wrote:

>My request and yours are not all that similar. I'd still like a response
>to *my* request.
>
>Jim
>


you wrote earlier:
>Problem
>
>  the Zope 3 project is making extensive use of PyUnit.  Zope 3 unit
>  tests often have to register components with various component
>  registries. These registrations need to be torn down after each
>  test. This became very tedious. For this reason, we've implemented a
>  cleanup registry. Any stateful global objects, like component
>  registries register functions with this cleanup registry. Test
>  classes provide tearDown methods that call a method on the cleanup
>  registry, which calls all the registered functions.  This is an
>  extremely useful pattern that might be beneficial to other
>  projects. Further, it would be nice if test classes didn't have to
>  provide a tear-down method that just called the cleanup registry (or
>  mix-in a class that provided such a tearDown.

You're right, it's in the tearDown() of a test and not in the tearDown()
of a test suite.

In case cleaning the registry is your only tearDown() functionality
with a mix-in you end up defining:


class RegistryCleaner:
    def tearDown(self): theRegistry.cleanup()

class someZope3TestCase( RegistryCleaner, unittest.testcase):
    ....


In case mixing in is too tedious you can use straight inheritance:


class RegistryCleaningTestCase( unittest.testcase):
    def tearDown(self): theRegistry.cleanup()
    # No test...() methods here.
    # Evt. move out of module scope to prevent searching for test...() methods.

class someZope3TestCase( RegistryCleaningTestCase):
    # test... methods here


When you need more tearDown() functionality depending on the actual test case
class, you can call back:


class RegistryCleaner:
    def tearDown(self):
        theRegistry.cleanup()
        self.tearDown2() # if hasattr(self, 'tearDown2')


and define tearDown2() instead of tearDown() in someZope3TestCase.

Here it would be nice to be able to call all tearDown() methods in
a multiple inheritance hierarchy, but I don't know how to do that in python.

I don't think any of this is tedious, and I guess even more compact
ways are possible.

Have fun,
Ype

-- 

Re: [Pyunit-interest] Feature Request: Cleanup Registry

[Steve P. <ste...@ya...>]

Hi Jim,

Sorry for the slow response.

Interesting suggestion. I wonder about a couple of things.

Firstly, not every test would require every cleanup function to be
called, so calling every function after every test seems a high
price to pay for convenience. Imagine running 10,000 tests,
where each block of 100 requires a single (but different) clean-up
function: that's 1,000,000 cleanup function invocations instead
of 10,000.

Secondly, I'm nervous about the notion of adding clean-up functions
to the clean-up registry from 'outside' testing code, as in the
example you provided. What if the test module is imported multiple
times, e.g. by the unittest GUI? -- then, clean-up functions might
be registered multiple times, presumably in global variables in
the unittest module.

Cleaning-up of component registries is to be done after every
Zope test, as I understand things. In that case, I'd think
a common base class for your Zope test cases is in order, with a
standardised 'tearDown()' that all subclasses should call if they
also implement a 'tearDown()' method. This is a safer solution, in
my opinion.

Or am I missing something?

Very best wishes,

-Steve


Jim Fulton wrote:
> Problem
> 
>   the Zope 3 project is making extensive use of PyUnit.  Zope 3 unit
>   tests often have to register components with various component
>   registries. These registrations need to be torn down after each
>   test. This became very tedious. For this reason, we've implemented a
>   cleanup registry. Any stateful global objects, like component
>   registries register functions with this cleanup registry. Test
>   classes provide tearDown methods that call a method on the cleanup
>   registry, which calls all the registered functions.  This is an
>   extremely useful pattern that might be beneficial to other
>   projects. Further, it would be nice if test classes didn't have to
>   provide a tear-down method that just called the cleanup registry (or
>   mix-in a class that provided such a tearDown.
> 
> Proposal
> 
>   Add a cleanup registry to the unittest module.  The unittest module
>   would provide an additional function, addCleanUp, which takes a
>   callable object to be called without arguments after running each
>   unit test. After each test, and after running the test class's
>   tearDown method, each registered cleanup callable will be called.
> 
>   Suppose there was an application module with a global dictionary
>   that was manipulated by application routines:
> 
>      TheData = {}
> 
>   The module would register the clear method of this dictionary with
>   unittest.addCleanUp to make sure the dictionary was reinitialized
>   after each test:
> 
>      import unittest
>      unittest.addCleanUp(TheData.clear)
> 
> I'd be happy to provide a patch.
> 
> Jim

Re: [Pyunit-interest] Feature Request: Cleanup Registry

[Jim F. <ji...@zo...>]

Steve Purcell wrote:
> Hi Jim,
> 
> Sorry for the slow response.
> 
> Interesting suggestion. I wonder about a couple of things.
> 
> Firstly, not every test would require every cleanup function to be
> called, so calling every function after every test seems a high
> price to pay for convenience. Imagine running 10,000 tests,
> where each block of 100 requires a single (but different) clean-up
> function: that's 1,000,000 cleanup function invocations instead
> of 10,000.

Good point.

> Secondly, I'm nervous about the notion of adding clean-up functions
> to the clean-up registry from 'outside' testing code, as in the
> example you provided. What if the test module is imported multiple
> times, e.g. by the unittest GUI?

Typically, the registrations would be done by application modules,
not test modules. Would the unittest GUI reload application modules?
That sounds like a recipe for disaster.

...


> Cleaning-up of component registries is to be done after every
> Zope test, as I understand things. In that case, I'd think
> a common base class for your Zope test cases is in order, with a
> standardised 'tearDown()' that all subclasses should call if they
> also implement a 'tearDown()' method. This is a safer solution, in
> my opinion.

That is the solution I'm using now. You are right, in that tests that are
*known* not to produce side-effects can avoid the extra clean up work.

> Or am I missing something?

Nope.

I like your scalability argument.

Thanks.

BTW, neither of your responses seem to have shown up in
the source forge mail archive.

Jim

-- 
Jim Fulton           mailto:ji...@zo...       Python Powered!
CTO                  (888) 344-4332            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org