Lua Evaluator

Writing a compliant evaluator with Lua

Lua is a small, dynamic, garbage collectedi, JIT compiled and embeddable language that has been integrated into MicroGP. It can be used to quickly write simple evaluators and to benefit from automatic management of paralellism by MicroGP when calling external evaluators.

How to use Lua

First of all you must check that your version of MicroGP has been compiled with Lua support. Run ugp3 --version and check the line that starts with Compiled on:

$ ugp3 --version
ugp3 (MicroGP++) v3.3.0 "PalmTree"
Yet another multi-purpose extensible self-adaptive evolutionary algorithm
(c) 2002-2013 by Giovanni Squillero <giovanni.squillero@polito.it>
This is free software, and you are welcome to redistribute it under certain conditions (use option "--license" for details)

Compiled on 10-Oct-2014 17:00:34 with GNU g++ 4.9.1, with Lua support (Lua 5.1 with LuaJIT 2.0.2).

Let's assume your evaluation script is called eval.lua. To use this script with MicroGP, the first step is to set it as the evaluator in the population settings. Open the settings file of your project, and change the parameter evaluatorPathName to the name of your evaluator. You can also set the value of concurrentEvaluations to the number of cores of your processor.

  <!-- evaluator parameters -->
  <evaluation>
    <evaluatorPathName value="eval.lua" />
    <concurrentEvaluations value="4" />
    <removeTempFiles value="true" />
    <evaluatorInputPathName value="individual_%s.in" />
    <evaluatorOutputPathName value="fitness.out" />
  </evaluation>

Anatomy of the evaluation script

The evaluate function

Here is an example of Lua evaluator (from the OneMax sample):

function evaluate(individuals)
  local _, count = string.gsub(individuals[1], "1", "1");
  return count
end

The only required part of the script is an evaluate function, which will receive a table containing individuals to evaluate. In the case of simple genetic evolution, the individuals table will always contain exactly one individual (at index 1 - tables are indexed from 1 in Lua). The function has to return the fitness of that individual.

In case of group evolution, the individuals table will contain from 1 to n individuals, and the function must return the fitness of this group of individuals as a group.

When the fitness has multiple components, all of them must be returned simultaneously using Lua's multiple return syntax. If you want tot include a description of the fitness, add a last return value of type string. This last parameter is optional. Example:

function evaluate(individuals)
  -- A fitness with 3 components
  return 3.45, 100, 0.00001, "My comment"
end

The rest of the script can contain other code: definition of helper functions, tests, reading configuration... MicroGP will evaluate the whole script one time when it starts, and then will make repeated calls to the evaluate function. For example, in the lamps sample, we read the parameters of the problem (size of the room and of the lamp) at the begining of the script.

-- We read the parameters of the problem.
local file, msg = io.open("parameters.txt", "r")
if not file then
  error("Error while opening parameters.txt: " .. msg)
end
local contents = file:read("*all")
file:close()
local radius = assert(contents:match("r=(%d+)"), "Error: radius not found")
local height = assert(contents:match("h=(%d+)"), "Error: heigth not found")
local width  = assert(contents:match("w=(%d+)"), "Error: width not found")
radius = tonumber(radius)
height = tonumber(height)
width  = tonumber(width)

local max_fitness = height * width;

function evaluate(individuals)
    -- radius, height and width are used by this function.
end

The ugp_environment table

When your script is evaluated from MicroGP, a global variable named ugp_environment is available. It gives information about MicroGP, the number of the current generation and the name of a directory that has been created especially for the script.

print("Running MicroGP version " .. ugp_environment.ugp_version)
print("Complete version info: " .. ugp_environment.ugp_tagline)

-- We can write temporary files to our directory
local file = io.open(ugp_environment.tempdir .. "/my_temporary_file", "w")

function evaluate(individuals)
  print(string.format("Evaluating %d individuals at generation %d",
                      #individuals, ugp_environment.generation))
end

Since this variable is only defined by MicroGP, you can write a part of the script that will be evaluated only outside of MicroGP. This is a nice way to add a test of your evaluator and verify it directly using Lua. Example of a function that counts ones in a string (from the OneMax sample):

function evaluate(individuals)
  local _, count = string.gsub(individuals[1], "1", "1");
  return count
end

-- This test will not be run by MicroGP.
if not ugp_environment then
    print("Testing the evaluator...")
    assert(evaluate({"0000"}) == 0)
    assert(evaluate({"0101"}) == 2)
    print("All tests passed!")
end

You can check the syntax and the results of this evaluator by running luajit eval.lua. Example result:

Testing the evaluator...
All tests passed!

Advanced usage

Fast numerical computation

LuaJIT, the just-in-time compiler for Lua, provides two ways of speeding up numerical computions, which can make Lua very competetive as a primary evaluator, despite its interpreted nature. First, the JIT compiler automatically emits optimized native CPU instructions for the critical sections of the code. This comes for free, you don't have anything special to do to get the benefits.

Then, to further optimize your code you can use the ffi library, which provides a way of using C functions and data structures from Lua. This can bring higher performance when working on arrays. The lamps sample uses the FFI to work on integer arrays.

Calling external evaluators

The OneMax/Assembly sample shows how to call external programs and retrieve their output using Lua. The resulting Lua program is more verbose than the bash equivalent, but the evaluation is snappier and is automatically paralellized by MicroGP:

-- We should get a temporary directory from MicroGP
tempdir = assert(ugp_environment.tempdir, "Lua evaluator: no temporary directory.")

-- Compile main.c
local fmt = string.format;
local ret = os.execute(fmt("gcc -O99 -c main.c -o %s/main.o", tempdir))
if ret ~= 0 then
    error("Lua evaluator: compilation error in main.c.")
end

function evaluate(individuals)
  -- Clean up directory
  os.remove(tempdir .. "/tprog")
  os.remove(tempdir .. "/error.log")
  -- Compile
  local file = io.open(tempdir .. "/code.S", "w")
  assert(file, "Lua evaluator: could not write assembly code.")
  file:write(individuals[1])
  file:close()
  local ret = os.execute(fmt("gcc -o %s/tprog %s/main.o %s/code.S -lm 2>%s/error.log", tempdir, tempdir, tempdir, tempdir))
  if ret ~= 0 then
    local error_log = "No error log."
    local file = io.open(tempdir .. "/error.log")
    if file then
      error_log = file:read("*all")
      file:close()
    end
    error("Lua evaluator: compilation error.\n" .. error_log)
  end
  -- Execute
  local out = io.popen(tempdir .. "/tprog")
  assert(out, "Lua evaluator: unable to execute program")
  local fitness, output = string.match(out:read("*all"), "(%d+) (%x*)")
  out:close()
  -- Get number of lines
  local _, num_lines = string.gsub(individuals[1], "\n", "\n")
  fitness_lines = math.max(0, 10000 - num_lines)

  return fitness, fitness_lines, fmt("0x%s/%d", output, num_lines)
end