Re: [Junit-devel] Feature request: @Assumes

[David S. <da...@sa...>]

I'll respond at ju...@ya......

On Wed, Sep 14, 2011 at 5:18 AM, Stephen Connolly
<ste...@gm...> wrote:
> Consider the case where you are testing a List class...
>
> we have
>
> public class ListTest {
>
>  @Test
>  public void newListIsEmpty() {
>    assertThat(new List().isEmpty(), is(true);
>  }
>
>  @Test
>  public void newListHasSizeZero() {
>    assertThat(new List().size(), is(0));
>  }
>
>  @Test
>  public void addPutsAnElementIntoAnEmptyList() {
>    List l = new List();
>    l.add(new Object());
>    assertThat(l.isEmpty(), is(false));
>  }
>
>  @Test
>  public void addIncreasesSizeOfPopulatedListByOne() {
>    List l = new List();
>    l.add(new Object());
>    int s = l.size();
>    l.add(new Object());
>    assertThat(l.size(), is(s + 1));
>  }
>
> }
>
> We now want to add some tests of the delete functionality... but the
> reality is that until/unless some of the preceding tests are passing,
> the tests for delete are meaningless. We could have a perfectly
> functional List.delete() method but until such time as the above tests
> are passing, there is no way to tell that the method does not work.
>
> Now I could code my tests like such
>
>  @Test
>  public void deleteIsANoOpOnEmptyList() {
>    List l = new List();
>    assumeThat(l.isEmpty(), is(true));
>    l.delete(new Object());
>  }
>
> But all that I am doing is repeating code from the preceding tests,
> having changed all those tests' assertThat(...)s into assumeThat(...)s
>
> That does not seem agile to me, copy & paste & search & replace... ban
> code smell there
>
> I would much rather be able to annotate the tests with an @Assumes
> annotation that indicates that the test assumes that the specified
> tests are passing, e.g.
>
>  @Test
>  @Assumes("newListIsEmpty")
>  public void deleteIsANoOpOnEmptyList() {
>    List l = new List();
>    l.delete(new Object());
>  }
>
>  @Test
>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>  public void deleteRemovesAnElement() {
>    List l = new List();
>    Object o = new Object();
>    l.add(o);
>    l.delete(o);
>    assertThat(l.isEmpty(), is(true));
>  }
>
> In fact in my initial example of tests, there are some additional
> assumptions that I didn't make explicit
>
>
>  @Test
>  @Assumes("newListIsEmpty")
>  public void addPutsAnElementIntoAnEmptyList() {
>    ...
>  }
>
> and
>
>  @Test
>  @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList")
>  public void addIncreasesSizeOfPopulatedListByOne() {
>    ...
>  }
>
> Now you could get some of this functionality via a TestRule...
>
> You could watch tests to see if they pass, and skip tests annotated
> with the annotation if assumed functionality is failing, but that
> would result in sporadic failures of, e.g. deleteRemovesAnElement
> because of the failing newListIsEmpty being executed _after_
> deleteRemovesAnElement rather than before.
>
> The simple point is that the test result of deleteRemovesAnElement is
> meaningless until its assumptions are true, and while I could code the
> assumptions with assumeThat(..)s C&P&S&R is even worse than C&P.
>
> Another alternative to @Assumes would be to invoke the assumed
> method(s) at the start of the test, e.g.
>
>  @Test
>  public void deleteRemovesAnElement() {
>    newListIsEmpty(); // verify assumed functionality
>    addPutsAnElementIntoAnEmptyList();  // verify assumed functionality
>    ...
>  }
>
> That gets rid of the C&P&S&R, but there are two issues with that:
>
>  1. We have to manually invoke any setup/tearDown methods, including
> all those of the rules that the test class has... very messy
>
>  2. The test fails when the assumed test fails. In actuality we can
> say nothing at all about whether deleteRemovesAnElement if a
> newListIsEmpty is not passing... yes we could code the test
> differently, but that is just moving our assumptions somewhere else.
>
> I am sure that there are others out there who feel there is a point 3...
>
>  3. We already ran those tests why waste time running them again?
>
> Well the answer to 3 is that these are UNIT tests which should be very
> fast, so what is the harm...
>
> So, in my view, best practice unit testing needs the ability to mark
> tests as assuming that other tests are passing, so that those tests
> can be skipped when the assumptions are known to be failing or
> skipped. [This is a deliberately loaded criteria... if the
> org.junit.runner.Request does not include the assumed test, then that
> test is neither known failing or known skipped, so we can run the test
> and output a warning that the failure may be because of assumed
> functionality... the use case of executing one and only one test
> repeatedly until you get that test passing]
>
> The annotation would have implications on test sorting, as any assumed
> tests would have to always happen before the assuming tests (as long
> as the assumed tests are in the org.junit.runner.Request)
>
> Also might have to be two annotations, e.g.
>
> @Assumes(methodNames)
> @AssumesClasses(classes)
>
> though in my view the @AssumesClasses is less critical, as these are
> UNIT tests and each test class should be independent to a large
> extent. However I am willing to consider that some people may have
> many test classes for one class under test, one test class containing
> all the tests of the constructors, another testing the Add methods,
> etc. in which case an @AssumesClasses annotation makes sense.
>
> Where tests contain a circular dependency, fail/error both tests
>
> Ok, let the critique begin!
>
> -Stephen
>
> P.S.
>
> I pinged Kent with an earlier version of this idea... but I think that
> he missed the point about eliminating C&P&S&R that this feature would
> provide because I didn't frame the idea correctly...
>
> ---------- Forwarded message ----------
> From: "Kent Beck"
> Date: 13 Sep 2011 17:11
> Subject: Re: JUnit and test dependencies
> To: "Stephen Connolly"
>
> Stephen,
>
> Thank you for articulating your idea so clearly. The short answer is that
> no, we don't plan to support dependencies. If I have tests that are slow
> enough that I care about dependencies, my most productive option is
> generally to work on the design of the software until the tests are fast
> enough that I no longer care. That said, my voice is only one of many. The
> longer answer is that I encourage you to post your idea on the JUnit mailing
> list for community discussion.
>
> Regards,
>
> Kent
>
> On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote:
>
>> Kent,
>>
>> Are there any plans for JUnit to support some test dependencies, such as:
>>
>> public class OnlyRunTestsThatMakeSenseTest {
>>
>>  @Test
>>  public void basicFunctionalityWorks() {
>>    ...
>>  }
>>
>>  @Test
>>  @AssumesPasses("basicFunctionalityWorks")
>>  public void advancedFunctionalityWorks() {
>>    ...
>>  }
>>
>>  @Test
>>  @AssumesPasses("basicFunctionalityWorks")
>>  public void basicFunctionalityWorksWithBevel() {
>>    ...
>>  }
>>
>>  @Test
>>
>  @AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"})
>>  public void advancedFunctionalityWorksWithBevel() {
>>    ...
>>  }
>>
>> }
>>
>> In the above example, no matter what sorting is applied,
>> basicFunctionalityWorks will always be run first, and the other three
>> tests will only be run if basicFunctionalityWorks passed.
>>
>> I see the above being completely in the spirit of unit testing, the
>> point with the above is that the @Before and @After's will be run
>> around each method, you are just saying that there is no point even
>> trying to test the advanced functionality when the basic functionality
>> is broken, skip those tests which we know cannot pass. That allows the
>> person writing advancedFunctionalityWorks to power through the setup
>> that depends on the basic functionality and not have to litter their
>> advanced test with asserts that are redundant because of the basic
>> functionality. Those people who are relying on side-effects should
>> really, for unit tests at least, be invoking the method who's
>> side-effects they depend on directly within their test method, rather
>> than relying on accidental ordering.
>>
>> Having said that, a second feature that I think would be good is
>> something like a @RunAfter and/or @RunBefore which would ensure that
>> the test method is run in sequence even if the before or after tests
>> fail/are skipped. with @RunAfter and @RunBefore I still think the
>> @Before and @After methods should be invoked in-between, this would be
>> moving towards more of a general purpose testing framework as opposed
>> to being unit-testing focused, but JUnit is just too good ;-)
>>
>> Thoughts?
>>
>> -Stephen
>
> ------------------------------------------------------------------------------
> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
> Learn about the latest advances in developing for the
> BlackBerry&reg; mobile platform with sessions, labs & more.
> See new tools and technologies. Register for BlackBerry&reg; DevCon today!
> http://p.sf.net/sfu/rim-devcon-copy1
> _______________________________________________
> Junit-devel mailing list
> Jun...@li...
> https://lists.sourceforge.net/lists/listinfo/junit-devel
>