Re: [Javapathfinder-user] JPF+RMI

[Michal K. <mic...@gm...>]

Hi Peter,

I'm not sure whether we are talking about same tests and test methods.
Suppose we have the following test (case) with two test methods:

public class MyTestCase {
  @Test
  public void addition() {
    assertEquals(12, simpleMath.add(7, 5));
  }

  @Test
  public void subtraction() {
    assertEquals(9, simpleMath.substract(12, 3));
  }
}

When JUnit runs MyTestCase it runs both methods and reports whether
they finished successfully or not. When the first method fails JUnit
will continue to run the second one. What happens when JUnit is run
under JPF? By default JUnit prints test results on stdout which is not
very useful if I want to collect these results. It is possible to run
JUnit with a different listener that will throw exceptions instead of
printing them on stdout. Such an exception will come to JPF's
propertyViolated - that's ok, but now execution stops and no other
test methods will be run (because JUnit exited with an exception on
some of the previous test method. How to tackle it?

One idea was to add an option into JUnit that would allow to run only
a single test method, which has a disadvantage that @BeforeClass and
@AfterClass initialization methods must be run for each test method.

Do you have any other idea how to be able to run all test methods
under JUnit+JPF so that results can be collected just by a JPF
listener as you suggested earlier.

> The test method itself you can get either from
> Config.getTargetArgParameters(), JVM.getArgs(), or by checking for
> Method.invoke() calls from instructionExecuted() notifications. If you want
> details about failed tests, those you would get from propertyViolated().

Don't you mean test case? In Javadoc there's written
Config.getTargetArgParameters() returns command line arguments passed
to the program executed under JPF. Is it right? In case of JUnit it
means names of test cases (classes) that are to be run (not test
methods).

Thanks.

Michal

On Wed, Aug 27, 2008 at 6:37 PM, Peter C. Mehlitz
<Pet...@na...> wrote:
>>
> Hi Michal,
>
>> Of course, I would rather use an easier solution but I'm not sure
>> whether it could
>> be achieved just by JPF listener. When I execute JUnit under JPF how can I
>> be
>> notified when a single test method has finished? And when an error occurs
>> in one test throwing an exception the whole JPF execution will be stopped
>> after
>> propertyViolated() is called. Is it right? But I would to like to run
>> all test methods
>> regardless they finish with an error or success.
>
> that's simple. You can use the same listener instance for multiple JPF runs
> (== each JUnit test), so you can collect statistics about failed and
> succeeded tests in this listener (which is allocated by your JPF driver).
> The test method itself you can get either from
> Config.getTargetArgParameters(), JVM.getArgs(), or by checking for
> Method.invoke() calls from instructionExecuted() notifications. If you want
> details about failed tests, those you would get from propertyViolated().
>
> -- Peter
>