pyunit-interest

RE: [Pyunit-interest] Run all tests in the current directory

[Patrick K. O'B. <po...@or...>]

> On Thu, 2002-03-14 at 15:23, Patrick K. O'Brien wrote:
> > I have a tests directory containing a number of unittest test
> modules and
> > wanted an easy way to run every single test without having to
> create suites
> > in each module. Since I couldn't find anything like this in the docs, in
> > anyone else's code, or on the Python Cookbook.
> <snip/>
>
> FWIW, in the pybeep project (http://pybeep.sourceforge.net) we use a
> similar script (CVSROOT/test/testall.py), liberally filched from Mark
> Pilgrims most excellent Dive into Python
> (http://diveintopython.org/regression_divein.html)
>
> Sean Allen

I missed Mark's version when I was looking for something like this. Thanks
for the link. And Mark's version puts mine to shame - his uses regular
expressions, filter, lambda and map (three times, no less) to do the exact
same thing. <0.5 wink>

---
Patrick K. O'Brien
Orbtech

RE: [Pyunit-interest] Run all tests in the current directory

[Patrick K. O'B. <po...@or...>]

A bit heavier than what I was after. But very nice. Lots of good ideas in
there. I'm sure I'll have more of a need for these features as my set of
tests grows. Thanks for the tip.

---
Patrick K. O'Brien
Orbtech

> -----Original Message-----
> From: pyu...@li...
> [mailto:pyu...@li...]On Behalf Of Jeremy
> Hylton
> Sent: Thursday, March 14, 2002 6:16 PM
> To: po...@or...; Sean Allen; PyUnit
> Subject: Re: [Pyunit-interest] Run all tests in the current directory
>
>
> There's a fairly elaborate test driver that we use for Zope
> and StandaloneZODB.  It's basic function is to find all the
> unittest suites in a large collection of Python packages,
> based on a few simple naming conventions.  Test suites are
> contained in subpackages named tests, in module with names
> that start with 'test'.  Each module has a function
> test_suite() that returns the unittest suite.
>
> It's got a bunch of other handy features:
>
> - connected to distutils setup/build commands
> - use regexes to specify particular modules or tests methods
> - print tracebacks immediately, rather than at the end (very
> handy when the full test suite takes 5-10 minutes)
> - run the tests in a loop to look for memory leaks
>
> You can find it at cvs.zope.org.  Look for Zope3/test.py on
> the Zope-3x-branch or StandaloneZODB/test.py on the trunk.
>
> Jeremy
>
> _______________________________________________
> Pyunit-interest mailing list
> Pyu...@li...
> https://lists.sourceforge.net/lists/listinfo/pyunit-interest

Re: [Pyunit-interest] Run all tests in the current directory

[Jeremy H. <je...@zo...>]

On Thu, 14 Mar 2002 18:42:00 -0600
 "Patrick K. O'Brien" <po...@or...> wrote:
> A bit heavier than what I was after. But very nice. Lots
> of good ideas in
> there. I'm sure I'll have more of a need for these
> features as my set of
> tests grows. Thanks for the tip.

It is certainly a complicated script.  And you're right that
the features may be useful as your test suite grows.  We
ended up needing these features on projects with large
and/or slow test suites.  Zope has 1200 unit tests.  An
internal project had functional tests driven by unittest
that took several minutes to run.  

Using regexs to select test suites and cases was probably
the most useful feature.  We had a few isolated tests that
failed intermittently.  We could use the basic test harness
to select those tests, run them in a loop, and print
tracebacks as soon as they happend.  It was the only sane
alternative :-).

Jeremy

Re: [Pyunit-interest] Run all tests in the current directory

[Ken C. <ke...@in...>]

I'm having some trouble with this script, it only seems to run the tests
from the last module loaded.  Here's my version with some debugging
added:

ken@temp:~/temp$ cat pytest.py
#!/usr/bin/env python2.2

import unittest
import glob
import os
import sys

def suite():
   sys.path.append('.')
   for filename in glob.glob('test_*.py'):
        print "Importing %s..." % (filename)
        module = __import__(os.path.splitext(filename)[0])
        print "Directory: %s" % (str(dir(module)))

   print "Final Directory: %s" % (str(dir(module)))
   suite = unittest.TestSuite()
   suite.addTest(unittest.defaultTestLoader.loadTestsFromModule(module))
   print "Loaded a total of %d test cases." % (suite.countTestCases())
   return suite
    

if __name__ == '__main__':
    unittest.main(defaultTest='suite')

I'm using the testcase example included in the python2.2-unit debian
package:

ken@temp:~/temp$ ls -l test_*.py
-rw-r--r--    1 ken      ken          2510 Mar 19 14:47 test_lists.py
-rw-r--r--    1 ken      ken           538 Mar 19 14:45 test_many.py
-rwxr-xr-x    1 ken      ken          1664 Mar 19 14:46 test_widget.py

Renamed of course to work with this script.

Here's what I get:

ken@temp:~/temp$ ./pytest.py
Importing test_lists.py...
Directory: ['ListTestCase', 'UserList', 'UserListTestCase',
'__builtins__', '__doc__', '__file__', '__name__', 'unittest']
Importing test_many.py...
Directory: ['ProlificTestCase', '__builtins__', '__doc__', '__file__',
'__name__', 'unittest']
Importing test_widget.py...
Directory: ['Widget', 'WidgetTestCase', '__builtins__', '__doc__',
'__file__', '__name__', 'unittest']
Final Directory: ['Widget', 'WidgetTestCase', '__builtins__', '__doc__',
'__file__', '__name__', 'unittest']
Loaded a total of 2 test cases.
.F
======================================================================
FAIL: Resizing of widgets
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_widget.py", line 28, in testResize
    assert self.widget.size() == (100,150), \
AssertionError: wrong size after resize

----------------------------------------------------------------------
Ran 2 tests in 0.003s

FAILED (failures=1)

As you can see the namespace seems to be getting replaced each time
__import__ is called.  Is this normal?  Change with 2.2 maybe?

Ken

[Pyunit-interest] Re: Run all tests in the current directory

[Ken C. <ke...@in...>]

OK, looks like my problem was a formatting issue, the addTest message
needs to be called after each __import__ therefore it should be in the
for loop.  Simple enough to fix, now I think I really am in business.

Thanks all.

Ken

RE: [Pyunit-interest] Re: Run all tests in the current directory

[Patrick K. O'B. <po...@or...>]

Email line wrapping messed up the formatting of the original bit of code
that I sent to the list. It sounds you've got things figured out now. If you
run into a snag, holler.

---
Patrick K. O'Brien
Orbtech

> -----Original Message-----
> From: pyu...@li...
> [mailto:pyu...@li...]On Behalf Of Ken
> Causey
> Sent: Tuesday, March 19, 2002 4:12 PM
> To: pyu...@li...
> Subject: [Pyunit-interest] Re: Run all tests in the current directory
>
>
> OK, looks like my problem was a formatting issue, the addTest message
> needs to be called after each __import__ therefore it should be in the
> for loop.  Simple enough to fix, now I think I really am in business.
>
> Thanks all.
>
> Ken
>
>
>
> _______________________________________________
> Pyunit-interest mailing list
> Pyu...@li...
> https://lists.sourceforge.net/lists/listinfo/pyunit-interest

Re: [Pyunit-interest] Run all tests in the current directory

[Jeremy H. <je...@zo...>]

There's a fairly elaborate test driver that we use for Zope
and StandaloneZODB.  It's basic function is to find all the
unittest suites in a large collection of Python packages,
based on a few simple naming conventions.  Test suites are
contained in subpackages named tests, in module with names
that start with 'test'.  Each module has a function
test_suite() that returns the unittest suite.

It's got a bunch of other handy features:

- connected to distutils setup/build commands
- use regexes to specify particular modules or tests methods
- print tracebacks immediately, rather than at the end (very
handy when the full test suite takes 5-10 minutes)
- run the tests in a loop to look for memory leaks

You can find it at cvs.zope.org.  Look for Zope3/test.py on
the Zope-3x-branch or StandaloneZODB/test.py on the trunk.

Jeremy