Menu

#7 Erroneous state coverage with AllRoundTester

None
closed
None
5
2016-01-25
2015-05-30
No

When doing something like the following:

Tester tester = new AllRoundTester(model);
tester.buildGraph();
tester.addCoverageMetric(new StateCoverage());
tester.generate(20);
tester.printCoverage();

the reported state coverage is wrong: it correspond to the states seen since the last reset, not the total number.

This workaround gives me the right number:

tester.addCoverageMetric(new StateCoverage() {
    @Override
    public String getName() {
        return "Total state coverage";
    }
});

Discussion

  • Mark Utting

    Mark Utting - 2016-01-25

    Interesting bug! Turns out that AllRoundTester uses a StateCoverage metric internally, so later StateCoverage metrics added by users were ignored.

    Fix: AllRoundTester now renames the StateCoverage metric that it uses internally, so that users are free to add their own StateCoverage metric if desired.

     
  • Mark Utting

    Mark Utting - 2016-01-25
    • status: open --> closed
    • assigned_to: Mark Utting
    • Group: -->
     
  • Mark Utting

    Mark Utting - 2016-01-25

    Thanks Grégoire for such a clear and useful bug report.

     

Log in to post a comment.

MongoDB Logo MongoDB
Gen AI apps are built with MongoDB Atlas
Atlas offers built-in vector search and global availability across 125+ regions. Start building AI apps faster, all in one place.