[Junit-devel] Redesign of Filter / Filterable interface in JUnit 4

[Jochen H. <jo....@go...>]

Hi all,

Note: I moved the discussion from feature request "1447312"
(https://sourceforge.net/tracker/?func=detail&aid=1447312&group_id=15278&ati
d=365278) to dev mailing list, as agreed by David Saff.

Background:
===========

Test cases can be dependent on prerequisites, e.g. be online in Internet,
have a database available, have database filled with specific test data,
etc. In these cases it can be very annoying to change always the "@Ignore"
behaviour of test cases, because it may be different for each run or
development time. A "@Prerequisite" annotation may support a dynamic
evaluation of specified prerequisites.

Each test case marked with @Prerequisite will be checked, whether the
condition is fulfilled. If not, the test case will be marked as ignored.

For example:
  @Prerequisite (requires="isDBAvailable")
  @Test public void doDBTest() {
    ...
  }

The method isDBAvailable() must be implemented with following signature:
  - must be callable from TestRunner, so must be public.
  - must be callable either from tested object, or from a static helper
class.
  - must return a boolean value.

For example:
  public boolean isDBAvailable() {
    boolean available = ...;
    return available;
  }

This method can also be provided by a static helper class, when specifiying
a callee attribute to annotation.
For example:
  @Prerequisite (requires="isDBAvailable", callee=DBHelper.class)
  @Test public void doDBTest() {
    ...
  }
  public class DBHelper {
    public static boolean isDBAvailable() {
      boolean available = ...;
      return available;
    }
  }

Design questions:
=================

1. The Filter class allows to implement shouldRun(Description desc), where
Description contains metadata about the current test (isSuite, isTest,
hasChildren, and a display name). But, there is no access to the Java method
for any reflection call. This is required, if the implementation has to know
about the existence of an own annotation for the current test method.

A very ugly solution is, to parse the current test class and test method
based on the display name :(


2. When calling the shouldRun(Description desc) method, there is no access
available to current test object.
So, if we want to call the instance method of the prerequisite (e.g.
isDBavailable()), we do NOT have access in filter.

A workaround has been, to implement an AnnotationRunner, which checks for
known annotations, and call specific callbacks (e.g. using the current test
object). Additionally we have to specify @RunWith(AnnotationsRunner.class)
to use this specific runner for the test class.


I think, the current implementation of Filter/Filterable does not support
Annotations very well. Can someone help with other implementation
approaches, or do we have to redesign the current implementation ?

Comments welcome.

Bye, Jochen


My current implementation is available at http://www.junitext.org.


See also:
- First feature request for @Prerequisite. Moved discussion now to
dev-mailing list
  https://sourceforge.net/tracker/?func=detail&aid=1447312&group_id=15278&at
id=365278

- feature request: "Description does not allow access to metadata via
reflection"
  https://sourceforge.net/tracker/index.php?func=detail&aid=1569263&group_id
=15278&atid=365278

- feature request: "Filter / Sorter do NOT have access to current test
object"
  https://sourceforge.net/tracker/index.php?func=detail&aid=1569265&group_id
=15278&atid=365278