Code explanation

- Overview and credits

- How to install

- Demo video

- Code explanation

- Binocular energy model

- Vergence control

- Control Evaluation

Code Explanation

Description of the functions

The Vergence Control is implemented as a C/C++ class, called vergCLASS.

vergCLASS is the default construtor:

vergCLASS POPULATION(int SX, int SY, string filter_filename, string vergence_weights_filename, int channles);

takes five input parameters:

• int SX: is the width of the image from the robot cameras
• int SY: is the height of the image from the robot cameras
• string filter_filename: is the string containing path and name of a .ini file containing the parameters to design the Gabor filters (provided)
• string vergence_weights_filename: is the string containing path and name of a .bin file containing the weights used to compute the Vergence Control
• int channles: defines the number of channels contained in the weights file (not used). In this version of the software the vergence channels are two, one for horizontal vergence and one for vertical vergence

setVergenceGAIN allows to set the gain for the Vergence Control, which will be tuned for the kinematics of the specific head you are using. To improve stability and performance, we recommend to add a PID or a Kalman Filter to your control.

POPULATION.setVergenceGAIN(float *GAIN);

The input is a float array with 2 cells, the first for the horizontal vergence gain, the second for the vertical vergence gain.

getVergenceGAIN(float *GAIN) returns as pointer the gain for the Vergence Control

POPULATION.getVergenceGAIN(float *GAIN);

The input is a float array with 2 cells, where the vergence gain is copied, the first for the horizontal vergence gain, the second for the vertical vergence gain.

loadImg and loadImgFile allow to load an image in the class from a OpenCV Mat or from an image.

POPULATION.loadImg(Mat IMG, char tag);
POPULATION.loadImgFile(string img_filename, char tag)

The input parameters are:

• Mat IMG: the image to be loaded from a Mat
• string img_filename: the string containing path and name of a image to be loaded
• char tag: a character that defines if you are uploading the let ('L') or the right ('R') image.

At each iteration of the Vergence Control you have to read both the left and the right image from the stereo cameras.

computeVergenceControl this function actually computes the Vergence Control.

POPULATION.computeVergenceControl();

getVergence, getVergenceH, and getVergenceV, these functions return the computed Vergence Control.

Scalar* VC = POPULATION.getVergence();
float VH = POPULATION.getVergenceH();
float VV = POPULATION.getVergenceV();

The output of these functions is a Scalar of floats with two cells for getVergence, and a float for getVergenceH and getVergenceV.

printVergence this function prints to screen the computed horizontal and vertical vergence control, together with the estimated FPS

POPULATION.printVergence();

setCenter: the Vergence control is computed to align the cameras on the center of the stereo image. This function allows to set the area on the image where to compute the Vergence Control in a position different form the center. This feature is used for instance to compute the vergence demand at target location before executing a saccade in depth. The theory is explained in the related paper. In the DEMO it is used to verge on different parts of the stereo image (see video.

POPULATION.setCenter(Point2d cntr)

The provided demo

* Use the .exe file*

From the command windows navigate to the folder containing the compiled project. To compute the vergence ontrol on a single image pair, use the following command:

VergenceControl.exe image_left.format image_right.format

* test_single_image()* (see video)

This demo loads a stereo pair of size 540 x 960 pixels and computes the Vergence Control 20 times in a for loop. At the end it shows the computation time and the achieved FPS

* test_mouse_scale()* (see video)

This demo loads a stereo pair of size 540 x 960 pixels and rescales them to a smalle size. The images are shown as a single anaglyph image. Saccadic eye movements to a target are simulated allowing the user to click with the mouse on a image location. The Vergence Control is then computed at target location, and vergence movements are simulated warping the left and right image according to the vergence control.
Exemplifying, if the Vergence Control provides a convergence movement, the left image is shifted towards right, while the right image is shifted toward left, of a quantity proportional to the Vergence Control.
The effectiveness of the Vergence Control can be directly evaluated because, is the algorithm is able to provide the correct Vergence Control, the binocular disparity at target location is nullified, and the anaglyph image at target turns to gray scale.