Manual

Manual Content:

• Name
• Synopsis
• Description
• Options
• Controls
  • Keyboard Controls
  • Mouse Controls
• Neural Network Models
• Learning Algorithms
• Activation Functions
• Visualization Method
• Interaction
• Performance Analysis
• Files
• Directories
• Patterns
  • Training patterns
  • Testing patterns
  • Visualization patterns
  • Extrapolation patterns
• Examples
  • Patterns Preparation
  • Running bnns
  • Performance Analysis
  • Configuration File
• Bugs
• Author
• See Also

Name

bnns - blue's neural network simulator

Synopsis

bnns [options] nn_topology

Description

Research tool for interactive training of artificial neural networks based on the Response Function Plots visualization method. It enables users to simulate, visualize and interact in the learning process of a Multi-Layer Perceptron on tasks which have a 2D character. Tasks like the famous two-spirals task or classification of satellite image data.

Options

--version
    Print version information and exit.

--help
    Print options summary and exit.

-v, --verbose
    Print verbose messages.

-c, --conflicts-preview
    Display conflicts between output layer neurons.

-d, --disable-input-rfps
    Disable input layer visualization.

-e, --ee-preview
    Display error-energy on output layer neurons.

-i, --user-interaction
    Enable user interaction.

-f, --conf-file=file
    Configuration file.

-l, --log-path=dirpath
    Path to the directory containing logfiles.

-p, --patterns-path=dirpath
    path to the directory containing patterns

-s, --scale-factor=val
    RFPs scale factor 1/val for val in {2,3,4,5}.

-u, --update-period=val
    RFPs update period in N.

--input-row-size=val
    Number of RFPs to display in a row of input layer x-window.

--hidden-row-size=val
    Number of RFPs to display in a row of hidden layer x-window.

--output-row-size=val
    Number of RFPs to display in a row of output layer x-window.

Mandatory Options

-p, --patterns-path=dirpath
    Path to the directory containing patterns.

-t, --task-name=val
    Common root of filenames of patterns.

Controls

Keyboard Controls

Space - Freeze/Unfreeze weights to the neuron below mouse pointer.
C-c - Stop training process.
C-d - Exit bnns.

Mouse Controls

Left button - If scaling of RFPs active, then display RFP in orig. size.
Middle button - Reset weight coming into the neuron below Mouse pointer.
Right Button - Display sum of posterior probabilities on output layer.

Neural Network Models

Multi-Layer Perceptron (MLP)

Learning Algorithms

Vanilla Backpropagation learning

Activation Functions

• Sigmoidal activation function with adjustable steepness.

• Softmax activation function on the output layer. Useful in classification tasks, because it ensures an actual sum of posterior probabilities across the output layer equal to one.

Visualization Method

Response Function Plots (RFP)

Interaction

User interaction with the NN model is only allowed when the training process is not active. It is at the moment limited to:

1. Reset of weights coming into a neuron by the Middle Mouse button.

2. Freeze and Unfreeze of weights coming into the neuron below Mouse pointer by the Space key. The RFP becomes highlighted to inform a user of weights being frozen. The learning rate parameter of the neuron is set to zero and to the default value respectively.

Performance Analysis

• Logging features menu

Enables to log the sum of posterior probabilities and the average squared error-energy on the output layer. Use provided gnuplot(1) scripts to display corresponding graphs.

• Error-energy

The average and the maximum of instantaneous error-energy on training/testing patterns can be displayed below RFPs of output neurons.

• Error-energy preview

Combines the instantaneous error-energy information with RFPs of the output layer.

• Conflicts Preview

Combines the sum of posterior probabilities across the output layer with RFPs of the output layer. Conflict, a strong response of several neurons is rep. by red colored regions. A weak network response is rep. by blue colored regions

• Conflicts view

If the output layer consists of at least a pair of neurons, then a Right Mouse button click on any of the output layer RFPs opens a window displaying the sum of posterior probabilities across the output layer. Strong response of several neurons is rep. by pink colored regions. A weak network response is rep. by cyan colored regions.

• Detail view

If scaling of RFPs is enabled, then a Left Mouse button click on any of the RFPs opens a window displaying the selected RFP in orig. size. A click on another RFP updates the window.

• Extrapolation view

An RFP of the output neuron on extrapolation patterns assists in extrapolation performance analysis.

Files

The main configuration file global.h provides macros customizing the compilation of bnns. It enables to select bnns features and to set global constants.

Directories

bnns/
|- data - images in PGM format, the source for bnns patterns
|- doc - manual page
|- patterns - default location of patterns in bnns format
|- scripts - gnuplot scripts to analyze NN performance and perl scripts to prepare patterns
|- src - bnns source code

Patterns

Input patterns must use a bnns specific format. The following scripts assist in patterns preparation. Use the -h command line option for an options summary.

scripts/
|- pgm2bnns.pl - Prepare patterns from a PGM image.
|- pgmck.pl - PGM image by Gimp -> PGM as described by **pgm**(5).
|- db2bnns.pl - Prepare patterns from d_Boston/ErdasExport.

Training patterns

Patterns used to train a NN, randomly selected from the set of labeled patterns.

File name: ${task}_train.pat

File structure:

[patterns #] [inputs #] [expected outputs #] [task dimension]
2
2
2
1
[X1 coordinate] [Y1 coordinate]
[input vector_1]
[expected output vector]

[X2 coordinate] [Y2 coordinate]
[input_vector_2]
[expected output vector]

...

[X coordinate] [Y coordinate]
[input_vector_k]
[expected output vector]

Testing patterns

Patterns used to evaluate the NN performance, usually a set of labeled patterns not included in the training set.

File name: ${task}_test.pat

File structure:

[patterns #] [inputs #] [expected outputs #] [task dimension]
2
2
2
1
[X1 coordinate] [Y1 coordinate]
[input_vector_1]
[expected output vector]

[X2 coordinate] [Y2 coordinate]
[input_vector_2]
[expected output vector]

...

[Xn coordinate] [Yn coordinate]
[input_vector_n]
[expected output vector]

Visualization patterns

Patterns required to compute RFPs, represent all known patters from the input space.

File name: ${task}_visualize.pat

File structure:

[RFP width] [RFP height] [inputs #]
2
2
2
1
[input_vector_1]

[input_vector_2]

...

[input_vector_n]

Extrapolation patterns

Set of patterns to test the extrapolation performance of a NN, usually a superset of visualization patterns.

File name: ${task}_extrapolation.pat

File structure: See Visualization patterns.

Examples

The bnns binary and perl scripts are self documented, use either the --help option or no options to get help.

Patterns Preparation

A pgm(5) image represents a labeled set of data that is mapped onto a predefine feature space with default origin in (0,0). The labeled data is divided into a randomly chosen training set of predefined size and a testing set. In real world tasks not all data is labeled and a visualization set is required to use the RFP visualization method. In order to study the extrapolation performance of a NN an extrapolation set that extends the predefined feature space is required.

Prepare patterns for the selected task in verbose mode including extrapolation patterns:

$ ./scripts/pgm2bnns.pl -v -e ./data/pgm/${task}.pgm

Use additional options to set the pct. of patterns to include into the training set and extrapolation patterns size.

$ ./scripts/pgm2bnns.pl -v -e -s 4 -p 20 ./data/pgm/${task}.pgm

Modify the settings section of pgm2bnns.pl in order to further customize the patterns preparation process.

Note: pgm2bnns.pl expects PGM files as defined by pgm(5). Gimp for example stores PGM pictures in a different way, so pgmck.pl is used to fix format issues. The pgmck.sh scripts MUST have executable flag set in order to have pgm2bnns.sh function properly on all input data.

To prepare a TAR archive storing patterns prepared from the Boston Remote Sensing Testbed data located in dirpath:

$ ./scripts/db2bnns.pl -v -t -p -d dirpath

Running bnns

Run bnns in verbose mode on a selected task with MLP topology of 2 input units, 4 hidden units and 1 output unit:

$ bnns -v ${task} 2 4 1

Run bnns on a selected task with MLP topology of 2 input units, 10, 4 units in the 1st, 2nd hidden layer respectively and 1 output unit. RFPs scaled by a factor of 0.5 and input layer visualization disabled:

$ bnns -d -s 0.5 ${task} 2 10 4 1

Performance Analysis

Use gnuplot(1) to analyze logs of the average error-energy and the sum of posterior probabilities on output layer:

$ gnuplot ./scripts/rspns_plot.gp
$ gnuplot ./scripts/ee_plot.gp

Configuration File

# bnns config file
conflicts-preview 1
disable-input-rfps 1
ee-preview 0
log-path /tmp/test
patterns-path ~/projects/bnns/patterns/
task-name spiral
scale-factor 1
user-interaction 1
update-period 40
verbose 1
input-row-size 10
hidden-row-size 5
output-row-size 10

Bugs

The bnns project is far from being extensively tested so bugs may occur. Please send bug reports to bnns-devel@lists.sourceforge.net

Author

Matus Uzak (matus.uzak@gmail.com)

See Also

gnuplot(1), perl(1), pgm(5)