#36 Problem with Instrumenter using custom problems

[Kurt Hinkle]

I am attempting to use an Instrumenter to collect run time data for an optimization. Running Example3.java, I have no issues; it appears that the sample problems run fine.

However, once I add a custom problem to the Instrumenter using the "withProblemClass(Class<>, arg1, arg2, etc)" method, I get the following exception:

Exception in thread "main" java.lang.IllegalArgumentException: no reference set available
at org.moeaframework.ProblemBuilder.getReferenceSet(ProblemBuilder.java:248)
at org.moeaframework.Instrumenter.getReferenceSet(Instrumenter.java:528)
at org.moeaframework.Instrumenter.instrument(Instrumenter.java:663)
at org.moeaframework.Executor.runSingleSeed(Executor.java:690)
at org.moeaframework.Executor.run(Executor.java:629)
at com.package.MainApplication.main(MainApplication.java:108)
Java Result: 1

If I don't add the instrumenter to my Executor, the problem executes correctly, but (of course) there is no data in the Instrumenter.

Will you enlighten me on how to solve this issue?

[Kurt Hinkle]

For clarity:

Instrumenter instrumenter = new Instrumenter().withProblemClass(Objective.class, slave, parameters.length).attachAll();

Executor executor = new Executor().withSameProblemAs(instrumenter) .withAlgorithm("NSGAII").withMaxEvaluations(100) .withInstrumenter(instrumenter);

NondominatedPopulation result = executor.run();
Accumulator accumulator = instrumenter.getLastAccumulator();

[Anonymous]

Hi, the instrumenter needs a reference set in order to calculate some of the performance metrics (e.g., hypervolume, generational distance, inverse generational distance). If your custom problem has a known reference set, it can be loaded using Instrumenter#withReferenceSet. Otherwise, you will need to generate a reference set for this problem. One way to do this is to solve the problem many times and merge the results into the reference set:

// evaluate the problem with many seeds
Executor executor = new Executor()
        .withProblemClass(Objective.class, slave, parameters.length)
        .withAlgorithm("NSGAII")
        .withMaxEvaluations(10000);

List<NondominatedPopulation> results = executor.runSeeds(50);

// combine the results into a single reference set
ReferenceSetMerger referenceSet = new ReferenceSetMerger();

for (int i = 0; i < results.size(); i++) {
    referenceSet.add("Seed" + i, results.get(i));
}

// save reference set to a file
PopulationIO.writeObjectives(
        new File("MyRefSet.pf"),
        referenceSet.getCombinedPopulation());

// setup the instrumenter to use your reference set
Instrumenter instrumenter = new Instrumenter()
        .withProblemClass(Objective.class, slave, parameters.length)
        .withReferenceSet(new File("MyRefSet.pf"))
        .attachAll();