From: SourceForge.net <no...@so...> - 2007-05-24 14:45:44
|
Bugs item #1703148, was opened at 2007-04-18 14:14 Message generated for change (Comment added) made by dsaff You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=115278&aid=1703148&group_id=15278 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: framework Group: None >Status: Pending Resolution: None Priority: 5 Private: No Submitted By: Adrian Kuhn (adriankuhn) Assigned to: Nobody/Anonymous (nobody) Summary: Tests methods with return value should be allowed Initial Comment: There is no apparent reason why a test method must not have a return value. In particular, in the context of test as examples, it is annoying that JUnit does not permit test methods with return values. ---------------------------------------------------------------------- >Comment By: David Saff (dsaff) Date: 2007-05-24 10:45 Message: Logged In: YES user_id=325156 Originator: NO Adrian, I find this idea interesting, but there are other interesting ideas already in the pipeline that are taking up most of my time right now, and I think that I would probably implement it in a different way than you subscribe. Would you have time to prototype a solution in Java using a JUnit 4 custom runner, and try it out on a small project? With a little more experience behind the idea, we could have a better idea of how to move it forward. Thanks, David Saff ---------------------------------------------------------------------- Comment By: Adrian Kuhn (adriankuhn) Date: 2007-05-24 03:01 Message: Logged In: YES user_id=1769328 Originator: YES David, sorry for the delay, I am very busy these days. For example the code would look like this public class Example { @Test public String S() { // eg creates some string, tests it, and returns it } @Test(dependsOn="Test.S") public List X(String string) { // eg uses the passed string to create some list and tests it and returns it } @Test(dependsOn="Test.S") public String Y(String string) { // eg changes the string, tests it and returns it } } Then the testing framework build the complete graph of tests based on the annotations, and starts calling test methods starting from the roots of the graph and caches the results to pass them on to test further down in the graph (or stops following a branch if there is a failure). Special care has to be take for side effects, ie when caching the result of S neither X nor Y may change it. I write some time ago prototype of such a testing framework in smalltalk, there I just assumed that no client test modifies its input objects. But for full-fledged framework you might want to be able to specify policies to mark tests that alter their input, to take care of such cases. cheers, AA ---------------------------------------------------------------------- Comment By: SourceForge Robot (sf-robot) Date: 2007-05-22 22:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: David Saff (dsaff) Date: 2007-05-08 10:51 Message: Logged In: YES user_id=325156 Originator: NO Adrian, In your example, how does the framework know that X and Y are depending on the output of S, in order to filter the results? Can you give me an example of how X, Y, and S would look in Java? Thanks, David Saff ---------------------------------------------------------------------- Comment By: Adrian Kuhn (adriankuhn) Date: 2007-04-19 15:34 Message: Logged In: YES user_id=1769328 Originator: YES David, hmm, I see your concern that allowing return values could maybe muddy the intention of JUnit. My usual reply to such concerns is ``design for the smart, not the dummy'', but with a framework as popular as JUnit, maybe that advice might not be the best :) Anyhow--the Eg paper is mainly about the work of one of Gälli's master students, you should find more in-depth information in examples-oriented testing in Gälli's PhD. Which is available using the following URL http://www.iam.unibe.ch/~scg/Archive/PhD/gaelli-phd.pdf I say should find, because I never read it myself, I learned all about his ideas sitting over a countless beers in the evening. In short: he analysed the code coverage of tests and found that they do not each cover different parts of the code, instead he discovered that the coverages form a partial order. Thus he concluded that tests could be refactored into a hierarchy of tests, where many tests use the return value of one other test as input. He calls the tests in such a hierarchy examples, ie the examples /are/ the tests. The avantage of such a testing framework would be, that it is aware of dependencies between tests and could thus, in case of an error, much more precisely point to the source. Talking about examples in abstract words is hard, thus let me give an example. Let A be a test class with setup S and test methods X and Y. Assume there is an error in S, then both X and Y are red. But we do not know if (i) there are two errors, one in Y and one in X, or if (ii) there is one error in the setup S. However if all S, X and Y would be tests (or examples, as Gälli would call them), then S would return the fixture as return value and X and Y take it as input parameter. And if S fails, the framework can precisely point to S as source, without the above doubt. I have chosen an example with a setup, because actually, JUnit's @Before setup is a kind of poor man's example-orientation, where one "test" is the input of many others. I hope, that this somehow illustrates the idea and possible advantages of example-oriented testing. Of source, example-orientation goes far beyond just mere return values, but it is in this context that my dislike of tests without return values has developed. cheers, AA ---------------------------------------------------------------------- Comment By: David Saff (dsaff) Date: 2007-04-19 13:11 Message: Logged In: YES user_id=325156 Originator: NO Adrian, Thanks for your point. I agree that, all things being equal, a framework should not be over-restrictive in the code it will accept. However, I think that there philosophical reasons to be cautious here. A JUnit test method is designed to work by side-effect, throwing an exception if the software works incorrectly. A return value seems to me to muddy this intent to a future reader--is the test method for asserting, or for constructing? If I'm scanning some tests to quickly understand what they do, I might appreciate that the framework the tests were written in warned the test writer if she tried to use the same name to mean two things. I'm a fan of re-using objects constructed for tests, but I suggest that a utility method for constructing the object tested on is much better at conveying intent to future readers, since the name of the test method should already be taken up in describing the behavior it asserts. That said, I'm always happy to learn new things. I found a paper by Markus Gälli that suggested a framework (Eg) for constructing Example objects to be used both in tests and other places, but didn't see the suggestion that getting these objects directly from the tests themselves was preferable. Do you have another pointer? Thanks, David Saff ---------------------------------------------------------------------- Comment By: Adrian Kuhn (adriankuhn) Date: 2007-04-19 11:19 Message: Logged In: YES user_id=1769328 Originator: YES There is work out there that suggests using tests results to create examples (eg a PhD by Markus Gälli, Univ of Bern). One of the main ideas is that you may reuse the object built in a test in other contexts, and thus gain reusability when tests are allowed to have return values. You broaden ppls possibilities, whereas JUnit works exactly the in both cases. If you allow for return values, people do and will find useful ways to use that. For example me, I often like to inspect the result of a test using Eclipse's scratchbook. Other ppl will find other usages. Hence, the question is not why allow it, but why deny it? There is no reason why test must not return a value. There are technical reasons why test methods must be public and why they cant take no parameters, but for return values ... there is no way how a return value could break the testing framework. Hence, if you may please allow for it as it does no harm, would be great. ---------------------------------------------------------------------- Comment By: David Saff (dsaff) Date: 2007-04-19 10:02 Message: Logged In: YES user_id=325156 Originator: NO Adrian, I'm slow to understand. Can you provide me with a test method that needs a return value to be a good example of usage? Many thanks, David Saff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=115278&aid=1703148&group_id=15278 |