From: SourceForge.net <no...@so...> - 2007-07-13 02:00:44
|
Bugs item #1739095, was opened at 2007-06-18 10:18 Message generated for change (Comment added) made by javabard You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=115278&aid=1739095&group_id=15278 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: test runner Group: None >Status: Open Resolution: Wont Fix Priority: 5 Private: No Submitted By: Michael Schechter (javabard) Assigned to: Nobody/Anonymous (nobody) Summary: Filtering does not appear to work in 4.3.1 Initial Comment: NOTE: This bug is being entered at the request of David Saff, as per message #19459 in the JUnit Yahoo group. Please let me know if you have any questions. I have been successfully using a filter implementation that I provided with one or more method names. These methods were the only ones executed in the test class. This worked seamlessly with 4.1. The implementation is designed to allow the test names to be passed to the main() method, emulating the JUnit 3.8.1 functionality that allowed the specification of test names for execution. This is an exceedingly useful tool when trying to nail down a bug that only occurs in one test scenario. I just recently updated to JUnit 4.3.1, and the filter implementation no longer seems to be applied. There is nothing I can find in the release notes that implies that this would have changed. The frustrating thing is that setting breakpoints in my filter has no effect - it is as if it is never being called. I am including the filter implementation class and the invocation logic. ------------------------------------------------ import java.util.ArrayList; import java.util.List; import org.junit.runner.Description; import org.junit.runner.manipulation.Filter; /** * This class provides a mechanism to filter tests by name, allowing a subset of the tests in a test * class to be defined by a developer. * * @since 2.0 */ public class MatchingFilter extends Filter { /** * The references to the tests that should be executed. */ private List<Description> tests = new ArrayList<Description>(); /** * Initializes an instance of <code>MatchingFilter</code> with the data provided. * * @param aTestClass * the class for which tests will be executed * @param someTestNames * the method names */ public MatchingFilter(Class aTestClass, String[] someTestNames) { for (String testName : someTestNames) { this.tests.add(Description.createTestDescription(aTestClass, testName)); } } /** * Implemented abstract method to provide detail filtering mechanism. Only test methods that match * the provided names will be executed. * * @see org.junit.runner.manipulation.Filter#shouldRun(org.junit.runner.Description) */ @Override public boolean shouldRun(Description aDescription) { return this.tests.contains(aDescription); } /** * Implemented abstract method. * * @see org.junit.runner.manipulation.Filter#describe() */ @Override public String describe() { return "This filter excludes any test methods that do not exactly match the provided names"; } } ******************** Invocation logic ******************** ------------------------------------------------ JUnitCore core = new JUnitCore(); Request testRequest = Request.aClass(aTestClass); if (0 < someArguments.length) { Filter matchFilter = new MatchingFilter(aTestClass, someArguments); testRequest = testRequest.filterWith(matchFilter); } core.run(testRequest); ---------------------------------------------------------------------- >Comment By: Michael Schechter (javabard) Date: 2007-07-12 22:00 Message: Logged In: YES user_id=756949 Originator: YES David, At this point in our cycle, this is pretty much at most a minor annoyance - we expect to be moving to Ant 1.7 and Eclipse Europa (3.3) in the near future. I would like to note the following: 1) Our suite() methods used a JUnit4TestAdapter under the hood, to allow compatibility with the existing tools (Ant 1.6.5 and Eclipse 3.1). This worked well, and allowed our small development team to take advantage of the new JUnit 4 features while still allowing focused testing/debugging by only running a specific test. 2) We never used RunWith as an annotation - since things worked, we never investigated it. 3) The following is the only notice in the release notes about the change that caught us: "Originally, developers who wanted to use a static suite() method from JUnit 3.x with a JUnit 4.x runner had to annotate the class with @RunWith(AllTests.class). In the common case, this requirement has been removed. However, when such a class is wrapped with a JUnit4TestAdapter (which we believe is rare), the results may not be as expected." The problem that confused me is that we only experienced problems when *not* using the suite() method, using the main() method that allows us to pick and choose the executed tests (this is the one below). The suite() method is *never* used in the case where things (from my uneducated perspective) misbehave. Just defining the suite() method was enough to change the Runner behavior, which really threw me for a loop. 4) I really do appreciate your time in walking me through this. I will work with the RunWith annotation to see if that resolves things in the short term, and it will be a useful tool if we run into problems with Ant 1.7. Thanks for your time! ---------------------------------------------------------------------- Comment By: David Saff (dsaff) Date: 2007-07-12 12:00 Message: Logged In: YES user_id=325156 Originator: NO The change in the behavior for auto-guessing the correct runner for classes with a suite() method was intentional. Given a class that has some JUnit 3 characteristics and some JUnit 4 characteristics, we've been trying to fine-tune the best guess that JUnit makes about which runner to use. We've finally settled for simplicity: 1) If there is a @RunWith annotation, obey it. 2) If JUnit 3 can run the class, use JUnit 3 3) Use JUnit 4 You can always override this guess, by annotating with @RunWith(TestClassRunner.class) // for JUnit 4 behavior or @RunWith(OldTestClassRunner.class) // for JUnit 3 behavior ---------------------------------------------------------------------- Comment By: Michael Schechter (javabard) Date: 2007-07-10 16:32 Message: Logged In: YES user_id=756949 Originator: YES I have additionally tried Eclipse Europa, as that has built-in JUnit 4 functionality. This removes the need for the suite() method, which we only are maintaining for backward compatibility with the Eclipse 3.1 JUnit runner. If I add the suite() method and try to run in Eclipse Europa, I get an error that no tests are run, and the diagnostics I added indicate that the method checked in the filter is initializationError0 Hope this helps! ---------------------------------------------------------------------- Comment By: Michael Schechter (javabard) Date: 2007-07-10 13:37 Message: Logged In: YES user_id=756949 Originator: YES The problem only occurs if there is a public static suite() method. Adding this forces JUnit into using the OldTestClassRunner, and no filtering is available. This is a well-defined change from the old behavior, and appears to remove the ability to retrofit the new JUnit 4 capabilities into a JUnit 3.8 execution environment. ---------------------------------------------------------------------- Comment By: David Saff (dsaff) Date: 2007-07-02 15:54 Message: Logged In: YES user_id=325156 Originator: NO Michael, Thanks for the report. We tried looking into this, but we can't seem to replicate the problem. For example, this test passes on our latest checkout: public class FilterTest { public static class ShouldFilter { @Test public void testA() {} @Test public void testB() {} } @Test public void filtersWork() { Request request = Request.aClass(ShouldFilter.class).filterWith( new MatchingFilter(ShouldFilter.class, new String[] { "testA" })); Result result = new JUnitCore().run(request); assertEquals(1, result.getRunCount()); } } Does it fail for you? Can you post a different test that fails? Thanks! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=115278&aid=1739095&group_id=15278 |