Re: [Httpunit-develop] Disabled tests - known bugs or not yet implemented features?

[Russell G. <ru...@go...>]

I have now fixed those broken tests that I could, and moved the others  
to pending-tests. We can harvest them from there according to our  
priorities. I have reverted nekohtml to 0.9.5 in order to maintain the  
previous functionality. The one case that requires 1.9.6 will work  
automatically if the user selects that version. We can upgrade when  
the new problem 1.9.6 introduces is fixed.

On May 16, 2008, at 8:48 AM, Russell Gold wrote:

>
> OK, I see the problem. A basic rule of HttpUnit through its  
> development has been to follow the agile development approach, which  
> among other things says: unit tests must always run at 100%. Unit  
> tests verify that the code does what the developers believe it does.  
> Test should never be committed to the baseline unless they pass, and  
> regressions in the unit tests are not permitted. Under certain very  
> extraordinary circumstances, we may disable a test - usually this is  
> only when that test is found to be intermittent or platform-specific.
>
> There is a second kind of tests for which the rules you are  
> describing would make sense - acceptance tests. It makes perfect  
> sense for applications to run acceptance tests at less than 100% and  
> track increasing success rates as a measure of completeness. As a  
> library, HttpUnit has never used those. Instead, the approach has  
> been to use the bug database as an indicator of problems not yet  
> solved. That means that we do not close bugs simply by creating  
> tests and committed them to the repository, commented out. Unless  
> the feature actually works, the bug report should remain open.
>
> It would make sense, I think, to adopt an approach of having a  
> second suite of new tests that correspond to open bugs. These would  
> not be part of the 'ant test' target, but could be added as those  
> bugs are fixed. The feature priorities you mention could be tied in  
> nicely with this second suite.
>