#42 using instrumenter on Knapsack problem

[Anonymous]

In the Knapsack example, when i try to use the Instrumenter to collect the GenerationalDistance after every 100 evaluations, i get this error
"no reference set available".
This works fine with predefined problems but running a customized problem such as Knapsack have this error.

[Anonymous]

Some of the performance indicators require a reference set (the best Pareto solutions for a problem). Since the Knapsack problems does not have a reference set, you need to specify one with instrumenter.withReferenceSet(File).

If the reference set for a problem is not known, you can create an approximation by combining the Pareto sets produced from many runs. See for example the ReferenceSetMerger class at http://www.moeaframework.org/javadoc/org/moeaframework/util/ReferenceSetMerger.html.

[Anonymous]

Thanks for answering but i am still not able to catch properly as i am quite new to the framework and just started, it would be really helpful if there is any kind of example available for such implementation

[Anonymous]

Here's an example. I'll be using the 100 item, 2 knapsack instance from http://www.tik.ee.ethz.ch/sop/download/supplementary/testProblemSuite/. You'll want to download both the test data knapsack.100.2 (ftp.tik.ee.ethz.ch) and Pareto-optimal front knapsack.100.2.pareto (ftp.tik.ee.ethz.ch) for this problem and save them to the MOEAFramework-2.4/ folder.

Then, modify the KnapsackExample.java file in the examples/org/moeaframework/examples/ga/knapsack folder to contain:

package org.moeaframework.examples.ga.knapsack;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;

import org.moeaframework.Executor;
import org.moeaframework.Instrumenter;
import org.moeaframework.analysis.collector.Accumulator;
import org.moeaframework.core.NondominatedPopulation;
import org.moeaframework.core.Solution;
import org.moeaframework.util.Vector;

/**

 * Example of binary optimization using the {@link Knapsack} problem on the
 * {@code knapsack.100.2} instance.
 */
public class KnapsackExample {

    /**

     * Starts the example running the knapsack problem.
     * 
     * @param args the command line arguments
     * @throws IOException if an I/O error occurred
     */
    public static void main(String[] args) throws IOException {
        Instrumenter instrumenter = new Instrumenter()
                .withProblemClass(Knapsack.class, new File("knapsack.100.2"))
                .withReferenceSet(new File("knapsack.100.2.pareto"))
                .attachElapsedTimeCollector()
                .attachGenerationalDistanceCollector();

        // solve using NSGA-II
        NondominatedPopulation result = new Executor()
                .withSameProblemAs(instrumenter)
                .withInstrumenter(instrumenter)
                .withAlgorithm("NSGAII")
                .withMaxEvaluations(50000)
                .distributeOnAllCores()
                .run();

        // print the results
        Accumulator accumulator = instrumenter.getLastAccumulator();
        System.out.format("  NFE    Time      Generational Distance%n");

        for (int i=0; i<accumulator.size("NFE"); i++) {
            System.out.format("%5d    %-8.4f  %-8.4f%n",
                    accumulator.get("NFE", i),
                    accumulator.get("Elapsed Time", i),
                    accumulator.get("GenerationalDistance", i));
        }
    }

}

The two key lines are withProblemClass(Knapsack.class, new File("knapsack.100.2")) and withReferenceSet(new File("knapsack.100.2.pareto")). The former sets the file that describes the knapsack problem we are solving. The latter sets the reference set that is required for computing the generational distance.

The output will look similar to the following:

  NFE    Time      Generational Distance
  100    0.1388    5.0155  
  200    0.1972    5.0556  
  300    0.2200    4.1606  
  400    0.2372    3.2332  
  500    0.2508    3.6330  
  600    0.2637    5.1729  
  700    0.2755    7.4643  
  800    0.2877    4.7049  
  900    0.2988    4.3096  
 1000    0.3113    7.5983  
 1100    0.3417    6.1986  
 1200    0.3558    6.2142  
 1300    0.3655    5.3630  
 1400    0.3758    7.6952  
 1500    0.3852    7.7753  
 1600    0.3939    7.7913  
 1700    0.4030    6.4281  
 1800    0.4126    6.4532  
 1900    0.4207    6.4532  
 2000    0.4545    6.4532  
 2100    0.4679    7.9572  
 2200    0.4806    7.9572  
 2300    0.4928    7.9572  
 2400    0.5060    5.5715  
 2500    0.5186    5.5928  
 2600    0.5277    5.5999  
 2700    0.5379    4.2318  
 2800    0.5488    3.9631  
 2900    0.5603    3.9681  
 3000    0.5718    3.9742  
 3100    0.5817    5.0708  
 3200    0.5901    4.6096  
 3300    0.5988    5.0743  
 3400    0.6066    5.0799  
 3500    0.6147    4.2887  
 3600    0.6258    3.7870  
 3700    0.6375    3.5887  
 3800    0.6491    3.4212  
 3900    0.6596    3.4224  
 4000    0.6687    3.2774  
 4100    0.6765    4.0217  
 4200    0.6844    4.2988  
 4300    0.6942    4.2988  
 4400    0.7041    4.6483  
 4500    0.7148    4.3053  
 4600    0.7227    4.0158  
 4700    0.7311    4.2951  
 4800    0.7404    4.0339  
 4900    0.7495    5.7167  
 5000    0.7589    5.1158  
 5100    0.7676    5.1181  
 5200    0.7760    4.6660  
 5300    0.7844    4.3159  
 5400    0.7931    3.8081  
 5500    0.8008    3.8094  
 5600    0.8090    3.2981  
 5700    0.8183    3.1749  
 5800    0.8260    3.1750  
 5900    0.8342    2.9563  
 6000    0.8419    3.0628  
 6100    0.8493    2.8676  
 6200    0.8572    2.8678  
 6300    0.8646    3.0682  
 6400    0.8721    3.1875  
 6500    0.8816    3.0718  
 6600    0.8907    3.3251  
 6700    0.8989    3.3251  
 6800    0.9067    3.3251  
 6900    0.9158    3.1935  
 7000    0.9236    3.1941  

[Anonymous]

Thanks a lot sir, its was really helpful.

[D. Hadka]

Glad this helped. Closing the ticket. Please open a new ticket if you have additional questions.