Screenshot instructions:
Windows
Mac
Red Hat Linux
Ubuntu
Click URL instructions:
Right-click on ad, choose "Copy Link", then paste here →
(This may not be possible with some types of ads)
From: David Saff <david@sa...> - 2011-09-14 17:01:02
|
I'll respond at junit@... On Wed, Sep 14, 2011 at 5:18 AM, Stephen Connolly <stephen.alan.connolly@...> wrote: > Consider the case where you are testing a List class... > > we have > > public class ListTest { > > @Test > public void newListIsEmpty() { > assertThat(new List().isEmpty(), is(true); > } > > @Test > public void newListHasSizeZero() { > assertThat(new List().size(), is(0)); > } > > @Test > public void addPutsAnElementIntoAnEmptyList() { > List l = new List(); > l.add(new Object()); > assertThat(l.isEmpty(), is(false)); > } > > @Test > public void addIncreasesSizeOfPopulatedListByOne() { > List l = new List(); > l.add(new Object()); > int s = l.size(); > l.add(new Object()); > assertThat(l.size(), is(s + 1)); > } > > } > > We now want to add some tests of the delete functionality... but the > reality is that until/unless some of the preceding tests are passing, > the tests for delete are meaningless. We could have a perfectly > functional List.delete() method but until such time as the above tests > are passing, there is no way to tell that the method does not work. > > Now I could code my tests like such > > @Test > public void deleteIsANoOpOnEmptyList() { > List l = new List(); > assumeThat(l.isEmpty(), is(true)); > l.delete(new Object()); > } > > But all that I am doing is repeating code from the preceding tests, > having changed all those tests' assertThat(...)s into assumeThat(...)s > > That does not seem agile to me, copy & paste & search & replace... ban > code smell there > > I would much rather be able to annotate the tests with an @Assumes > annotation that indicates that the test assumes that the specified > tests are passing, e.g. > > @Test > @Assumes("newListIsEmpty") > public void deleteIsANoOpOnEmptyList() { > List l = new List(); > l.delete(new Object()); > } > > @Test > @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList") > public void deleteRemovesAnElement() { > List l = new List(); > Object o = new Object(); > l.add(o); > l.delete(o); > assertThat(l.isEmpty(), is(true)); > } > > In fact in my initial example of tests, there are some additional > assumptions that I didn't make explicit > > > @Test > @Assumes("newListIsEmpty") > public void addPutsAnElementIntoAnEmptyList() { > ... > } > > and > > @Test > @Assumes({"newListIsEmpty","addPutsAnElementIntoAnEmptyList") > public void addIncreasesSizeOfPopulatedListByOne() { > ... > } > > Now you could get some of this functionality via a TestRule... > > You could watch tests to see if they pass, and skip tests annotated > with the annotation if assumed functionality is failing, but that > would result in sporadic failures of, e.g. deleteRemovesAnElement > because of the failing newListIsEmpty being executed _after_ > deleteRemovesAnElement rather than before. > > The simple point is that the test result of deleteRemovesAnElement is > meaningless until its assumptions are true, and while I could code the > assumptions with assumeThat(..)s C&P&S&R is even worse than C&P. > > Another alternative to @Assumes would be to invoke the assumed > method(s) at the start of the test, e.g. > > @Test > public void deleteRemovesAnElement() { > newListIsEmpty(); // verify assumed functionality > addPutsAnElementIntoAnEmptyList(); // verify assumed functionality > ... > } > > That gets rid of the C&P&S&R, but there are two issues with that: > > 1. We have to manually invoke any setup/tearDown methods, including > all those of the rules that the test class has... very messy > > 2. The test fails when the assumed test fails. In actuality we can > say nothing at all about whether deleteRemovesAnElement if a > newListIsEmpty is not passing... yes we could code the test > differently, but that is just moving our assumptions somewhere else. > > I am sure that there are others out there who feel there is a point 3... > > 3. We already ran those tests why waste time running them again? > > Well the answer to 3 is that these are UNIT tests which should be very > fast, so what is the harm... > > So, in my view, best practice unit testing needs the ability to mark > tests as assuming that other tests are passing, so that those tests > can be skipped when the assumptions are known to be failing or > skipped. [This is a deliberately loaded criteria... if the > org.junit.runner.Request does not include the assumed test, then that > test is neither known failing or known skipped, so we can run the test > and output a warning that the failure may be because of assumed > functionality... the use case of executing one and only one test > repeatedly until you get that test passing] > > The annotation would have implications on test sorting, as any assumed > tests would have to always happen before the assuming tests (as long > as the assumed tests are in the org.junit.runner.Request) > > Also might have to be two annotations, e.g. > > @Assumes(methodNames) > @AssumesClasses(classes) > > though in my view the @AssumesClasses is less critical, as these are > UNIT tests and each test class should be independent to a large > extent. However I am willing to consider that some people may have > many test classes for one class under test, one test class containing > all the tests of the constructors, another testing the Add methods, > etc. in which case an @AssumesClasses annotation makes sense. > > Where tests contain a circular dependency, fail/error both tests > > Ok, let the critique begin! > > -Stephen > > P.S. > > I pinged Kent with an earlier version of this idea... but I think that > he missed the point about eliminating C&P&S&R that this feature would > provide because I didn't frame the idea correctly... > > ---------- Forwarded message ---------- > From: "Kent Beck" > Date: 13 Sep 2011 17:11 > Subject: Re: JUnit and test dependencies > To: "Stephen Connolly" > > Stephen, > > Thank you for articulating your idea so clearly. The short answer is that > no, we don't plan to support dependencies. If I have tests that are slow > enough that I care about dependencies, my most productive option is > generally to work on the design of the software until the tests are fast > enough that I no longer care. That said, my voice is only one of many. The > longer answer is that I encourage you to post your idea on the JUnit mailing > list for community discussion. > > Regards, > > Kent > > On Sep 13, 2011, at 8:32 AM, Stephen Connolly wrote: > >> Kent, >> >> Are there any plans for JUnit to support some test dependencies, such as: >> >> public class OnlyRunTestsThatMakeSenseTest { >> >> @Test >> public void basicFunctionalityWorks() { >> ... >> } >> >> @Test >> @AssumesPasses("basicFunctionalityWorks") >> public void advancedFunctionalityWorks() { >> ... >> } >> >> @Test >> @AssumesPasses("basicFunctionalityWorks") >> public void basicFunctionalityWorksWithBevel() { >> ... >> } >> >> @Test >> > @AssumesPasses({"basicFunctionalityWorksWithBevel","advancedFunctionalityWorks"}) >> public void advancedFunctionalityWorksWithBevel() { >> ... >> } >> >> } >> >> In the above example, no matter what sorting is applied, >> basicFunctionalityWorks will always be run first, and the other three >> tests will only be run if basicFunctionalityWorks passed. >> >> I see the above being completely in the spirit of unit testing, the >> point with the above is that the @Before and @After's will be run >> around each method, you are just saying that there is no point even >> trying to test the advanced functionality when the basic functionality >> is broken, skip those tests which we know cannot pass. That allows the >> person writing advancedFunctionalityWorks to power through the setup >> that depends on the basic functionality and not have to litter their >> advanced test with asserts that are redundant because of the basic >> functionality. Those people who are relying on side-effects should >> really, for unit tests at least, be invoking the method who's >> side-effects they depend on directly within their test method, rather >> than relying on accidental ordering. >> >> Having said that, a second feature that I think would be good is >> something like a @RunAfter and/or @RunBefore which would ensure that >> the test method is run in sequence even if the before or after tests >> fail/are skipped. with @RunAfter and @RunBefore I still think the >> @Before and @After methods should be invoked in-between, this would be >> moving towards more of a general purpose testing framework as opposed >> to being unit-testing focused, but JUnit is just too good ;-) >> >> Thoughts? >> >> -Stephen > > ------------------------------------------------------------------------------ > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA > Learn about the latest advances in developing for the > BlackBerry® mobile platform with sessions, labs & more. > See new tools and technologies. Register for BlackBerry® DevCon today! > http://p.sf.net/sfu/rim-devcon-copy1 > _______________________________________________ > Junit-devel mailing list > Junit-devel@... > https://lists.sourceforge.net/lists/listinfo/junit-devel > |