From: Stephen K. <Ste...@or...> - 2005-08-16 05:02:50
|
This has been partially covered before, but I think the logic was inadequate. In summary, Dean Hiller had a problem where his setUp() was failing: > I notice the unit test in > junit.tests.framework.TestCaseTest.testTearDownSetupFails() > specifically is written to show that an Exception in setUp() does not > call tearDown(). > > Is there a reason for this? Kent Beck replied with: > Thank you for submitting your patch. The current behavior is deliberate. > My reasoning for the current behavior is that if setUp() fail, then > tearDown() is useless. SetUp() should never fail. If it does, the > whole premise of the test is false. Do you have concrete examples > where tearDown() would be useful even if setUp() fail? Dean Hiller then explained about his database problem to ensure consistency. We have the same problem, and I think that Kent's position is a bit short-sighted. To back up my claim, I searched for what tearDown() is meant for. It surprised me that it was not used in any of the "test infected" or "JUnit Cooks tour", or even the "Cookbook" - it is obviously intended for advanced usage. In the Cookbook however, it states: "Override tearDown <http://junit.sourceforge.net/javadoc/junit/framework/TestCase.html#tearDown%28%29>() to release any permanent resources you allocated in setUp." The problem here is that if you are allocating permanent resources in setUp, you generally have absolutely no guarantee that they will be created correctly or without exception. Therefore, you cannot assume that setUp() will not fail. Furthermore, since setUp() is almost always enough for a [unit] test, tearDown() should be considered advanced usage, and there will be no ill side-effects of having it run in 99% of test cases. The tests that the code change will affect will probably want it to be the case anyhow. Essentially, for our database tests, our fixture must ensure that there are certain records that our tested (persistence) objects are able to access correctly. Please fix this - we can hack around it for the mo, but it's pretty dumb. Cheers Stephen |
From: Dean H. <de...@xs...> - 2005-08-16 05:07:11
|
I believe this fix is in the head of CVS. It is just not released. You can build your own version of JUnit from CVS which has the fix. dean Stephen Kestle wrote: > This has been partially covered before, but I think the logic was > inadequate. In summary, Dean Hiller had a problem where his setUp() > was failing: > >> I notice the unit test in >> junit.tests.framework.TestCaseTest.testTearDownSetupFails() >> specifically is written to show that an Exception in setUp() does not >> call tearDown(). >> >> Is there a reason for this? > > > Kent Beck replied with: > >> Thank you for submitting your patch. The current behavior is deliberate. >> My reasoning for the current behavior is that if setUp() fail, then >> tearDown() is useless. SetUp() should never fail. If it does, the >> whole premise of the test is false. Do you have concrete examples >> where tearDown() would be useful even if setUp() fail? > > > Dean Hiller then explained about his database problem to ensure > consistency. We have the same problem, and I think that Kent's > position is a bit short-sighted. > > To back up my claim, I searched for what tearDown() is meant for. It > surprised me that it was not used in any of the "test infected" or > "JUnit Cooks tour", or even the "Cookbook" - it is obviously intended > for advanced usage. In the Cookbook however, it states: > > "Override tearDown > <http://junit.sourceforge.net/javadoc/junit/framework/TestCase.html#tearDown%28%29>() > to release any permanent resources you allocated in setUp." > > The problem here is that if you are allocating permanent resources in > setUp, you generally have absolutely no guarantee that they will be > created correctly or without exception. Therefore, you cannot assume > that setUp() will not fail. > > Furthermore, since setUp() is almost always enough for a [unit] test, > tearDown() should be considered advanced usage, and there will be no > ill side-effects of having it run in 99% of test cases. The tests > that the code change will affect will probably want it to be the case > anyhow. > > Essentially, for our database tests, our fixture must ensure that > there are certain records that our tested (persistence) objects are > able to access correctly. > > Please fix this - we can hack around it for the mo, but it's pretty dumb. > > Cheers > > Stephen > > > > > ------------------------------------------------------- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing > & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > _______________________________________________ > Junit-devel mailing list > Jun...@li... > https://lists.sourceforge.net/lists/listinfo/junit-devel |
From: Stephen K. <Ste...@or...> - 2005-08-16 05:36:45
|
Does not look like it - I've updated to version 1.18.2.2 (kbeck on 2 June 05): public void runBare() throws *Throwable *{ Throwable exception= null; * setUp();* try { runTest(); } catch (Throwable running) { exception= running; } *finally *{ try { *tearDown*(); } catch (Throwable tearingDown) { if (exception == null) exception= tearingDown; } } if (exception != null) throw exception; } Cheers Stephen Dean Hiller wrote: > I believe this fix is in the head of CVS. It is just not released. > You can build your own version of JUnit from CVS which has the fix. > dean > > Stephen Kestle wrote: > >> This has been partially covered before, but I think the logic was >> inadequate. In summary, Dean Hiller had a problem where his setUp() >> was failing: >> >>> I notice the unit test in >>> junit.tests.framework.TestCaseTest.testTearDownSetupFails() >>> specifically is written to show that an Exception in setUp() does >>> not call tearDown(). >>> >>> Is there a reason for this? >> >> >> >> Kent Beck replied with: >> >>> Thank you for submitting your patch. The current behavior is >>> deliberate. >>> My reasoning for the current behavior is that if setUp() fail, then >>> tearDown() is useless. SetUp() should never fail. If it does, the >>> whole premise of the test is false. Do you have concrete examples >>> where tearDown() would be useful even if setUp() fail? >> >> >> >> Dean Hiller then explained about his database problem to ensure >> consistency. We have the same problem, and I think that Kent's >> position is a bit short-sighted. >> >> To back up my claim, I searched for what tearDown() is meant for. It >> surprised me that it was not used in any of the "test infected" or >> "JUnit Cooks tour", or even the "Cookbook" - it is obviously intended >> for advanced usage. In the Cookbook however, it states: >> >> "Override tearDown >> <http://junit.sourceforge.net/javadoc/junit/framework/TestCase.html#tearDown%28%29>() >> to release any permanent resources you allocated in setUp." >> >> The problem here is that if you are allocating permanent resources in >> setUp, you generally have absolutely no guarantee that they will be >> created correctly or without exception. Therefore, you cannot assume >> that setUp() will not fail. >> >> Furthermore, since setUp() is almost always enough for a [unit] test, >> tearDown() should be considered advanced usage, and there will be no >> ill side-effects of having it run in 99% of test cases. The tests >> that the code change will affect will probably want it to be the case >> anyhow. >> >> Essentially, for our database tests, our fixture must ensure that >> there are certain records that our tested (persistence) objects are >> able to access correctly. >> >> Please fix this - we can hack around it for the mo, but it's pretty >> dumb. >> >> Cheers >> >> Stephen >> >> >> >> >> ------------------------------------------------------- >> SF.Net email is Sponsored by the Better Software Conference & EXPO >> September 19-22, 2005 * San Francisco, CA * Development Lifecycle >> Practices >> Agile & Plan-Driven Development * Managing Projects & Teams * Testing >> & QA >> Security * Process Improvement & Measurement * >> http://www.sqe.com/bsce5sf >> _______________________________________________ >> Junit-devel mailing list >> Jun...@li... >> https://lists.sourceforge.net/lists/listinfo/junit-devel > > > |
From: Dean H. <de...@xs...> - 2005-08-16 05:58:07
|
oh, whoops, I got that issue confused with another one....yes you are right. I personally am not sure.....I could go either way on that one....At least if setUp fails and tearDown is going to fail, the setup exceptoin would be reported first if the change was made, allowing a try at teardown which in most cases will probably fail IMO. or at least, I have many tests that would, but that's okay, because the setUp exception would be propagated and not the teardown one anyways. In fact, it may be a false sense of teardown in many cases where you think it ran through all of teardown when it did not.....that would be a hell of a bug to go after. I am on fence. dean Stephen Kestle wrote: > Does not look like it - I've updated to version 1.18.2.2 (kbeck on 2 > June 05): > > public void runBare() throws *Throwable *{ > Throwable exception= null; > * setUp();* > try { > runTest(); > } catch (Throwable running) { > exception= running; > } > *finally *{ > try { > *tearDown*(); > } catch (Throwable tearingDown) { > if (exception == null) exception= tearingDown; > } > } > if (exception != null) throw exception; > } > > Cheers > > Stephen > > Dean Hiller wrote: > >> I believe this fix is in the head of CVS. It is just not released. >> You can build your own version of JUnit from CVS which has the fix. >> dean >> >> Stephen Kestle wrote: >> >>> This has been partially covered before, but I think the logic was >>> inadequate. In summary, Dean Hiller had a problem where his setUp() >>> was failing: >>> >>>> I notice the unit test in >>>> junit.tests.framework.TestCaseTest.testTearDownSetupFails() >>>> specifically is written to show that an Exception in setUp() does >>>> not call tearDown(). >>>> >>>> Is there a reason for this? >>> >>> >>> >>> Kent Beck replied with: >>> >>>> Thank you for submitting your patch. The current behavior is >>>> deliberate. >>>> My reasoning for the current behavior is that if setUp() fail, then >>>> tearDown() is useless. SetUp() should never fail. If it does, the >>>> whole premise of the test is false. Do you have concrete examples >>>> where tearDown() would be useful even if setUp() fail? >>> >>> >>> >>> Dean Hiller then explained about his database problem to ensure >>> consistency. We have the same problem, and I think that Kent's >>> position is a bit short-sighted. >>> >>> To back up my claim, I searched for what tearDown() is meant for. >>> It surprised me that it was not used in any of the "test infected" >>> or "JUnit Cooks tour", or even the "Cookbook" - it is obviously >>> intended for advanced usage. In the Cookbook however, it states: >>> >>> "Override tearDown >>> <http://junit.sourceforge.net/javadoc/junit/framework/TestCase.html#tearDown%28%29>() >>> to release any permanent resources you allocated in setUp." >>> >>> The problem here is that if you are allocating permanent resources >>> in setUp, you generally have absolutely no guarantee that they will >>> be created correctly or without exception. Therefore, you cannot >>> assume that setUp() will not fail. >>> >>> Furthermore, since setUp() is almost always enough for a [unit] >>> test, tearDown() should be considered advanced usage, and there will >>> be no ill side-effects of having it run in 99% of test cases. The >>> tests that the code change will affect will probably want it to be >>> the case anyhow. >>> >>> Essentially, for our database tests, our fixture must ensure that >>> there are certain records that our tested (persistence) objects are >>> able to access correctly. >>> >>> Please fix this - we can hack around it for the mo, but it's pretty >>> dumb. >>> >>> Cheers >>> >>> Stephen >>> >>> >>> >>> >>> ------------------------------------------------------- >>> SF.Net email is Sponsored by the Better Software Conference & EXPO >>> September 19-22, 2005 * San Francisco, CA * Development Lifecycle >>> Practices >>> Agile & Plan-Driven Development * Managing Projects & Teams * >>> Testing & QA >>> Security * Process Improvement & Measurement * >>> http://www.sqe.com/bsce5sf >>> _______________________________________________ >>> Junit-devel mailing list >>> Jun...@li... >>> https://lists.sourceforge.net/lists/listinfo/junit-devel >> >> >> |
From: Stephen K. <Ste...@or...> - 2005-08-16 06:28:52
|
Ok, then part of the fix/change would also be a two-part exception to output setUp and tearDown exceptions in one bundle. I reckon tearDown() is the big finally() block that you'd usually write. You have to figure out what has successfully opened. Other data rollback issues ( that I presume you're talking about), are not really covered in the scope of tearDown() in the docs: "Override tearDown <http://junit.sourceforge.net/javadoc/junit/framework/TestCase.html#tearDown%28%29>() to release any permanent resources you allocated in setUp." In any case, going after that "hell of a bug" would be very much akin to going after the one that currently exists. Both could be non-intuitive, but I think we should at least attempt a tearDown [of course]. Cheers Stephen Dean Hiller wrote: > oh, whoops, I got that issue confused with another one....yes you are > right. > I personally am not sure.....I could go either way on that one....At > least if setUp fails and tearDown is going to fail, the setup > exceptoin would be reported first if the change was made, allowing a > try at teardown which in most cases will probably fail IMO. or at > least, I have many tests that would, but that's okay, because the > setUp exception would be propagated and not the teardown one anyways. > In fact, it may be a false sense of teardown in many cases where you > think it ran through all of teardown when it did not.....that would be > a hell of a bug to go after. I am on fence. > dean > > Stephen Kestle wrote: > >> Does not look like it - I've updated to version 1.18.2.2 (kbeck on 2 >> June 05): >> >> public void runBare() throws *Throwable *{ >> Throwable exception= null; >> * setUp();* >> try { >> runTest(); >> } catch (Throwable running) { >> exception= running; >> } >> *finally *{ >> try { >> *tearDown*(); >> } catch (Throwable tearingDown) { >> if (exception == null) exception= tearingDown; >> } >> } >> if (exception != null) throw exception; >> } >> >> Cheers >> >> Stephen >> >> Dean Hiller wrote: >> >>> I believe this fix is in the head of CVS. It is just not released. >>> You can build your own version of JUnit from CVS which has the fix. >>> dean >>> >>> Stephen Kestle wrote: >>> >>>> This has been partially covered before, but I think the logic was >>>> inadequate. In summary, Dean Hiller had a problem where his >>>> setUp() was failing: >>>> >>>>> I notice the unit test in >>>>> junit.tests.framework.TestCaseTest.testTearDownSetupFails() >>>>> specifically is written to show that an Exception in setUp() does >>>>> not call tearDown(). >>>>> >>>>> Is there a reason for this? >>>> >>>> >>>> >>>> >>>> Kent Beck replied with: >>>> >>>>> Thank you for submitting your patch. The current behavior is >>>>> deliberate. >>>>> My reasoning for the current behavior is that if setUp() fail, >>>>> then tearDown() is useless. SetUp() should never fail. If it does, >>>>> the whole premise of the test is false. Do you have concrete >>>>> examples where tearDown() would be useful even if setUp() fail? >>>> >>>> >>>> >>>> >>>> Dean Hiller then explained about his database problem to ensure >>>> consistency. We have the same problem, and I think that Kent's >>>> position is a bit short-sighted. >>>> >>>> To back up my claim, I searched for what tearDown() is meant for. >>>> It surprised me that it was not used in any of the "test infected" >>>> or "JUnit Cooks tour", or even the "Cookbook" - it is obviously >>>> intended for advanced usage. In the Cookbook however, it states: >>>> >>>> "Override tearDown >>>> <http://junit.sourceforge.net/javadoc/junit/framework/TestCase.html#tearDown%28%29>() >>>> to release any permanent resources you allocated in setUp." >>>> >>>> The problem here is that if you are allocating permanent resources >>>> in setUp, you generally have absolutely no guarantee that they will >>>> be created correctly or without exception. Therefore, you cannot >>>> assume that setUp() will not fail. >>>> >>>> Furthermore, since setUp() is almost always enough for a [unit] >>>> test, tearDown() should be considered advanced usage, and there >>>> will be no ill side-effects of having it run in 99% of test cases. >>>> The tests that the code change will affect will probably want it to >>>> be the case anyhow. >>>> >>>> Essentially, for our database tests, our fixture must ensure that >>>> there are certain records that our tested (persistence) objects are >>>> able to access correctly. >>>> >>>> Please fix this - we can hack around it for the mo, but it's pretty >>>> dumb. >>>> >>>> Cheers >>>> >>>> Stephen >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------- >>>> SF.Net email is Sponsored by the Better Software Conference & EXPO >>>> September 19-22, 2005 * San Francisco, CA * Development Lifecycle >>>> Practices >>>> Agile & Plan-Driven Development * Managing Projects & Teams * >>>> Testing & QA >>>> Security * Process Improvement & Measurement * >>>> http://www.sqe.com/bsce5sf >>>> _______________________________________________ >>>> Junit-devel mailing list >>>> Jun...@li... >>>> https://lists.sourceforge.net/lists/listinfo/junit-devel >>> >>> >>> >>> > |