UsersGuideIntroduction

Introduction

This guide will give an overview about how to set up and run batch jobs within the batch system. The guide covers the user interface, considerations about structuring your batch jobs and advanced options to use batch jobs as templates to analyse multiple datasets in a efficient way.
If you are using another MATLAB application and want to run it through this batch system, you have to find out whether it has support (i.e. configuration files and program interfaces) for the batch system. If it does not, you should ask the authors of your application to add the necessary program files. If you have basic MATLAB programming skills, you may even want to do it yourself. The batch system does help to create the necessary interface configuration files using the ConfGUI? application.

Graphical user interface

Overview

The GUI main window consists of

• a menu bar
• a toolbar
• four interaction and information areas.

Menus and toolbar

The "File" and "Edit" menu are built into the GUI and do not change. These menus contain standard functionality to load, save and run batch scripts, to add new applications to the configuration and to edit batch modules. Depending on the applications on your MATLAB path that use matlabbatch, the order of the other menu items may vary - these menus are created when cfg_ui (the GUI) and cfg_util (the configuration management tool) are started for the first time in a MATLAB session. Each application will appear in a separate menu, the menu items correspond to executable modules of this application.
The toolbar contains shortcuts to some of the functionality from the "File" menu.

Main window

The left box in the main window lists all modules that are part of the current job, in the top right part information about all inputs in the current module (the one highlighted in the module list) is displayed. Next to the module and item names there are indicators whether a module has inputs that are not yet set (<-X) or whether it depends on output from another module (DEP). The middle section shows the value of the current input item (highlighted in the top right box). For multidimensional data, struct or cell arrays, the value will be displayed as a set of MATLAB commands to generate it.
The bottom box displays a context sensitive help text describing the purpose of the currently selected module and input item.

Build a batch job

General considerations for job setup

Before you start entering values in the GUI, you should have a detailed plan of your data analysis process. You should

• list all data, which is available before analysis and which will be needed at some stage in your analysis process,
• list all necessary processing steps,
• make a note about the data flow in the analysis process (output data from one processing step that enter subsequent steps).

In most cases, this information is described best in a flow diagram:

In this diagram, there are three input items which are available before the analysis begins. Input 1 and Input 2 are required to run Processing A. Input 1 and some output of Processing A are then passed on to Processing B. Input 3 and output of Processing A are used to run Processing C. Finally, the output of Processing B is used as input to Processing D. This dependency graph shows the relationships between the different processing step. In this example, Processing A has to be run first. Once Processing A has finished, Processing B and Processing C can be run in any order. Processing D has to wait until execution of Processing B has finished.
In the user interface, batch modules have to be entered in sequential order. Modules at the top of the list will be run before those at the bottom. To save you time while setting up a job, it is recommended you draw a full dependency graph before you start adding modules in the GUI. This is because

• The GUI enforces an sequential way to setup a job. Modules can be deleted from any place in a job, but new modules will always be added at the end.
• To reorder parts of a job, the only way to do so is to split the jobs into parts, save them independent of each other and reload them.

The following sections suggest this strategy for creating a batch template or job:

1. Assemble all modules in the correct order.
2. Link modules that depend on each other.
3. Enter settings which are constant for all data you want to analyse using this batch. At this point you can save the batch as a template, leaving only the inputs unset that need to be filled with your actual data.
4. Load the batch template (if necessary) and enter the missing input data.
5. Run the job.

Assemble modules

A new module can be added to the batch by choosing from the application's menu. This module will then show up in the module list. It can be deleted via "Delete Module" in the "Edit" menu or in the module list context menu.

Enter settings

To enter values, highlight the module in the left pane which you want to modify. A summary of all inputs and their current values will be displayed on the right. <-X means that this input has yet to be set before the module can be run. DEP indicates that this input has been linked to an output from a previous processing step (see below). You should now enter the values which do not depend on the actual data set you want to analyse (e.g. algorithm parameters, constants). To enter values, hit <RETURN> on an item, double click with the mouse or hit the <Edit>/<Select Files> button. For menus, choices and repeated inputs, you can also select the choice directly from the value display pane.

Dependencies

If configured by the application developer, each module will be able to determine what output arguments will be produced after the module has been run. No computation is done and no data is being read at this time. The number and structure of the expected outputs can only depend on the structure of the inputs. If there are outputs available, a <Dependencies> button will appear in addition to the <Edit>/<Select Files> button. Select the appropriate output from the dialog window, and the dependency will be entered. Actual values will then be passed from one module to another at run time.

Prepare a template

To reuse a batch for the analysis of multiple similar datasets, it is recommended to

• enter all common settings
• set all dependencies
• mark all data inputs with <-X

If a data input is not yet marked with <-X, use "Clear Value" in the "Edit" menu to unset it. This ensures, that no necessary inputs can be missed when the template is loaded to analyse real dataset. After all settings have been checked, the batch can be saved as template. The batch is a special MATLAB variable, it can be saved either as a MATLAB .mat file or a MATLAB .m file.

Run a job

After the template has been created or loaded, the remaining data can be entered. Once everything is set, the "Play" button should turn green and the job can be started. The MATLAB command window will show the progress of job execution, unless the job is executed in parallel on a cluster. If an error occurs in a module, the batch system will try to continue with other modules that do not depend on outputs of the failed module. After job execution, an error log is printed to the command window. If a batch exits with some error, please inspect the command window output to find out which module failed for what reason.

Working on many datasets

GUI options

There are several ways to run a batch template for multiple data sets. If the datasets are analysed as they become available, it may be sufficient to open the GUI, load the template and run it for one dataset at a time. For a small number of datasets one can load the template multiple times (when loading the template, click on the "Ed" button and copy/paste the filename as often as necessary). The template will be loaded as many times as selected, and data for each dataset can be entered at the appropriate places.

MATLAB Scripts

If there are many datasets to analyse, or if the analysis should be triggered automatically, the batch system can be controlled from MATLAB command line or MATLAB scripts. See the Toy Example introduction for example code.