Re: [Pyunit-interest] comments on PyUnit

[Steve P. <ste...@ya...>]

Hi Fred, Guido and all,

Fred L. Drake, Jr. wrote:
>     I'd like to see one set or the other become recommended usage, and
>     offer names for all the functions.  Using the two sets of names is
>     confusing, especially since one set is the logical negation of the
>     other.  I'd also like to see only one of assert*Equal() and
>     assert*Equals() be documented; having two names so similar seems
>     unnecessary.
> 
>     Guido has indicated a preference for the fail*() names; I have no
>     preference either way.


I agree that recommending one consistent set of names is a good idea. The
'fail' methods have been around longer, but I think that the 'assert*' methods
may be clearer for non-native English speakers, who must be used to the
assert keyword already.

On the whole, I would vote for the 'assert' methods, because I get the
impression that they are more widely used at this point in time.

>     The names provided in the latest version are:
> 
>     fail*()      | assert*()
>     -------------+------------------------------------
>     fail()       | 
>     failIf()     | 
>     failUnless() | assert_()
>                  | assertRaises()
>                  | assertEqual(), assertEquals()
>                  | assertNotEqual() assertNotEquals()
> 
>     Note that while there is a (small!) overlap, neither group
>     completely covers the available functionality.

Right: 'failIf()' is redundant, and there is no 'assertionFailed()'.

I suggest the following standard set:

    fail()
    assert_()
    assertRaises()
    assertEqual()
    assertNotEqual()

There's no great harm in documenting the synonym 'failUnless()', since
'assert_()' is slightly ugly.


>   - Guido suggests using a unique exception for test failures, similar
>     to the use of TestFailed in the Python regression test.  This
>     allows the test errors to be distinguished from the traditional
>     use of "assert" in Python modules.  For backward compatibility it
>     might be reasonable to subclass from AssertionError.

That doesn't help backward compatibility, because the framework would still
have to catch 'AssertionError' in order to detect failures.

I have yet to come across a situation where making this distinction between
deeply buried AssertionErrors and deliberately provoked ones is important.
The worst that can currently happen is that some buried 'assert' statements
can cause a test to look like it's failing rather than erroring.

(I also like the idea that test writers can still use the 'assert' statement
for their tests if it suits them.)


>     Having all the fail*()/assert*() methods call fail() to actually
>     raise the appropriate expception would be an easy way to isolate
>     this, and allow TestCase subclasses to do things differently
>     should the need arise.

As a slight simplification to the code, it makes sense. However, it would
never make sense for TestCase subclasses to change that (fail()) aspect of
their behaviour, since the framework strictly defines which exception is
considered a failure.


>   - There is no facility to skip a test without failing.  This can be
>     reasonable in cases where a platform does not support the full
>     capability of the tested module.  An example application of this
>     is in test.test_largefile in the Python regression test -- if a
>     platform does not support large files, it is not considered an
>     error for the tests not to be completed.

I thought about this recently, and I came to the conclusion that tests should
be skipped by explicitly omitting them from a test suite, along the
following lines:

    def suite():
       try:
          import some_tests
          return unittest.defaultTestLoader.loadTestsFromModule(some_tests)
       except ImportError:
          return TestSuite() # empty suite

I don't really like the alternative, which is adding a 'skipped' result.
The Python regression test suite is probably unusual in requiring
the skipping of some tests. Most test suites are more straightforward.

On the whole, at this late stage in the Python 2.1 release, I wouldn't be
inclined to change anything much.

Am I making any sense? (Hint: if so, it's never happened before...)

-Steve


-- 
Steve Purcell, Pythangelist
Get testing at http://pyunit.sourceforge.net/
Any opinions expressed herein are my own and not necessarily those of Yahoo