[Moose-g3-devel] March 2006 GENESIS development Workshop Summary

[Dave B. <db...@do...>]

This summary of the WAM-BAMM*06 GENESIS 3 development workshop was
originally prepared for the genesis-dev mailing list.  The actual posting
was shorter, and pointed to the genesis-dev web site.  This site will
continue to exist, but we are gradually moving over to the Sourceforge
MOOSE/GENESIS 3 site at http://sourceforge.net/projects/moose-g3.

Now that this mailing list has been replaced by the moose-g3-devel mailing
list, I am reposting it here, so that it will be archived at the
moose-g3 site.  Other subsequent discussions may be seen in the mailing
list archives, through http://sourceforge.net/projects/moose-g3.
These may modify some of the suggested priorities in this original
March 2006 document.  (e.g., we have decided to use Sourceforge for
the mailing list, rather than set up Mailman ourselves.)

-----------------------------------------------------------------

GENESIS 3.0 Development Workshop Summary
----------------------------------------


A GENESIS 3.0 Development Workshop was held on Thursday, March 23 2006
during the WAM-BAMM conference in San Antonio.  The format of the workshop
was several short presentations on the status and the design issues of
various aspects of GENESIS 3 development, with each one followed by a
period of moderated open discussion on related topics.  The purpose was to
solict opinions, ideas, and suggested development priorities from GENESIS
users and developers.  The schedule and suggested discussion topics are
available at http://www.wam-bamm.org/program.html.  The powerpoint slides
and the following meeting summary will be available on the GENESIS
developers web site.  I am circulating this summary to GENESIS developers
and interested collaborators in the hope that all of you, those who attended
the meeting and those who were unable to attend, can add your comments
and to help us refine and prioritize our development goals.

During the workshop, Dave Beeman gave a quick summary of the present status
of GENESIS 2, followed by the a review of our overall plans involving MOOSE
as the core of GENESIS 3.  GENESIS 2.3 was released March 20, and will
hopefully be the last GENESIS 2 release.  The overview of GENESIS 3 plans
can be seen at http://www.genesis-sim.org/GENESIS/newgrant/G3/index.html.

Upi Bhalla gave an update of the current status of MOOSE.  The most recent
version 0.06 (February 27, 2006) was emailed to several developers.  Sometime
in the near future it will be generally available through a Sourceforge
repository.  In the meantime it is available on the GENESIS developers web
page.  The discussion, sumamrized below centered around the issues of
non-graphical aspects of GENESIS 3 and the MOOSE core, including desired
new features, binary file formats, and optional loading of modules.

Joe Svitak led a discussion of issues related to the design of the
graphical interface with the MOOSE core, and source code maintenance, the
development process, etc.

Sharon Crook and Padraig Gleeson gave an overview of NeuroML and its
relation to other neural databases or model descriptions (e.g. SBML,
BrainML, CellML, NeuronDB), and to the neuroConstruct project.

The Neural Open Markup Language (NeuroML) project (http://www.neuroml.org/)
is an ongoing collaborative effort to facilitate cooperation in building,
simulating, testing and publishing models of channels, neurons and networks
of neurons.  It is not a single standard XML language; rather a collection
of related XML projects for modelling different aspects and levels of
neural systems, from intracellular mechanisms and ion channels to networks
of reconstructed neurons.  The NeuroML Validation service at
http://morphml.org:8080/NeuroMLValidator/ allows validation of XML files
against the various XML Schema documents that define the NeuroML
specification.

The neuroConstruct project
(http://www.physiol.ucl.ac.uk/research/silver_a/neuroConstruct/) has been
designed to simplify development of complex networks of biologically
realistic neurons, i.e. models incorporating dendritic morphologies and
realistic cell membrane conductances. It is implemented in Java and
generates output which can be used by the NEURON and GENESIS simulators. It
uses the latest XML based specifications for neuronal information exchange
developed through the NeuroML project. Both GENESIS and NEURON developers
collaborate with the neuroConstruct project through the NeuroML Sourceforge
development site (http://sourceforge.net/projects/neuroml).

For more information about these projects, see the summary of the Workshop
on XML for Model Specification held during WAM-BAMM*05
(http://www.wam-bamm.org/WB05/Tutorials/advanced-tutorials/crook/index.html).

Greg Hood, from the Pittsburgh Supercomnputing Center, disscussed issues
relevant to parallelization of simulations in GENESIS 3.  There will be no
separate "PGENESIS 3" as an add-on to GENESIS 3.  Parallelization will be
built in from the outset.

Discussion summary and development priorities
---------------------------------------------

On Thursday Evening, March 23, 2006, we held a meeting of the GENESIS
developers and interested collaborators at the Menger Hotel to further
discuss issues that were brought up at the Workshop.  It was attended by
GENESIS developers Upi Bhalla, Dave Beeman, Joe Svitak, Sharon Crook, Greg
Hood, and Dieter Jaeger, and collaborators from the NeuroML project Padraig
Gleeson (NeuroConstruct) and Tom Morse (NEURON).  In the following days,
there were additional discussions with developers Hugo Cornelis, Kay
Robbins, Rodrigo Oliveira, Zhiwei Wang, Michael Edwards, and Jim Bower, as
well as several GENESIS users.

There were discussions on the following eight categories of development
tasks and issues:

1. MOOSE and the non-graphical aspects of GENESIS 3
2. The GENESIS 3 graphical interface
3. Project management, the development process, source code maintenance,
    coding and documentation standards.
4. Model representation with XML
5. The possible role of Neurospaces as an internal representation within
    GENESIS 3.
6. Parallel computation within GENESIS 3.
7. Implementing specialized solvers, e.g. hsolve for compartmental models.
8. Communication with users and developers.

The following summary is a synthesis of notes taken by Dave Beeman and
Sharon Crook.  It attempts to present what I (Dave Beeman) believe is our
current understanding of the important issues to be resolved and
development tasks to be preformed.  I have attempted to assign priorities
to some of these, so that we can assess manpower requirements and develop a
timeline for carrying them out.  Generally, design decisions that will
affect implementation of critical parts of GENESIS 3 have been given a High
Priority.  Implementation of features that give compatibilty with the most
often used features of GENESIS 2 are also of High Priority, with
implementation of new features given a Medium or Low Priority.  The priority
ranking is intended to be a measure of when these tasks will be done in
the development process, not necessarily a measure of their relative
importance.  Generally, we would expect that High Priority features will be
implemented in the first public beta release, and certainly in the initial
GENESIS 3.0 release.  Ones of Low Priority may wait until a later 3.1
release.

I would like to encourage all GENESIS developers and interested
collaborators to discuss these priorities and make corrections and
additions to this summary, preferably through the GENESIS developers
mailing list.  This is long!  But, I've resisted the urge to produce
a short summary of major isssues and priorities until I get your
feedback.

1. MOOSE and the non-graphical aspects of GENESIS 3

One big challenge is modeling multiple levels and therefore multiple time
scales.  Network specification is not well developed at this time

* Binary file format compatibility, Netcdf or alternatives

We need a binary file format for import and export of data that is platform
independent.  According to Dieter, even with today's large capacity drives,
an ASCII file format is not acceptable for the enormous data sets used in
his simulations.  The traditional FMT1 format used by GENESIS 2 is neither
efficient nor platform independent.  GENESIS 2 also incorporates NetCDF (a
vary large package that is much too slow). Maybe on the fly compression of
ASCII data when the file is saved is sufficient?  This is the approach used
in the Open Ducument Format, and it generally results in files that are
smaller than Microsoft Office format.  People will always want to save all
kinds of data. Will compressed ascii be enough?  Dieter has a program for
lossless compression of the GENESIS binary format which is important for
saving data from 100,000 simulations. It results in file sizes that are 10%
of the size of FMT1.  In this case filesize is a huge issue. Dieter would
love to have good header info with parameter values etc.

* Modules

Greg proposed that we have automatic download/compilation/linking of
required modules based on a module dependency graph.  This would enable one
to add new classes without recompiling (using dynamic linking).  For
example, one may want a lean version of GENESIS 3 that optionally loads the
kinetics or compartmental modeling modules.  We discussed what is needed
for the MOOSE core for this to be possible.  If successive versions of a
module cannot coexist, this suggests that there should be an interface
module for that functionality

Should parameter searching should be a separate module, or considered a
higher-level problem for which our job is to provide a good GENESIS API?
Currently parameter searching is provides with a set of customizable
GENESIS scripts with the GENESIS 2 distribution (genesis/Scripts/param).

Modular approach to specifying equations.  ???

* GENESIS 2 non-graphical simulation classes and commands to re-implement

Dave will make a first pass at dividing existing GENESIS non-XODUS classes
and commands into priority categories of High, Medium, Low, and "Do Not
Implement" and circulate it for comment.

* New features for GENESIS 3?

Data acquisition:  (Low Priority for implementation - it can come at any
time, to be done by those who need it the most.)  It is simple in MOOSE -
just write a class.  Upi does this in his lab.  Dieter would like to
simulate a robot arm, rat in maze, etc, and connect to a network
simulation.  Upi says that it just requires writing the classes.

Markov channels:  NeuroML describes them; NEURON has them; GENESIS needs
them.  Dieter can offer some expertise, but someone else has to do the
programming.

* MOOSE interfaces to other software

Bruce McCormick suggested that the MOOSE interface, with relatively minor
extensions, could permit the reconstruction and geometrical modeling of cells
(neurons and glia) and their networks including neuroavasculature).  The
advantage to GENESIS 3.0 would be that of having close links to cellular
neuroanatomy modeling and the visibility that the rapid advances in biomedical
imaging can bring to computational neuroscience.

2. The GENESIS 3 graphical interface
------------------------------------

* Discussion:  GUIs - Two views

Dieter: Serious modelers don't care about GUIs.

Dave: Young grad students and postdocs of the "point and click" generation
tend to choose NEURON over GENESIS at modeling courses because it is easy
to get started with the Cell Builder and other GUI tools.

* Interfaces between JAVA code and MOOSE/computational core

Design decision: (Priority: High) How does MOOSE talk to GUI's? What is the
interface?  This should be done first, and then decide on the graphical
toolkits/languages to be used for the implementation.

RMI and overhead issues:  Padraig believes that C++/Java communication has
a large overhead, and will hurt performance.  Joe is not convinced.  Some
tests will be necessary.

* SWING/non-SWING, JAVA version, etc. (compromises betweeen usablility
     in a web browser and features available with standalone operation)

Design decision: (Priority: High) What tools will be used for GUI's?
(Java?, If Java, use Swing or AWT?, Python, FLTK, or another?

Our goal will be to run the Purkinje tutorial, complete with the color
compartment Vm display, either when running GENESIS locally with the
built-in GUI, or remotely with a browser.  The only way to be sure that
that the web interface display will be visible to a user of Internet
Explorer or other popular browser would be to either require the user to
download and install a plugin, or to develop the interface as a Java 1.1
(AWT) applet, with considerable restrictions on the graphical tools
available.

Jim mentioned that UTHSCSA Virtual uses the VNC virtual desktop with an open
source plugin in order to peek at a specified area of another users screen.
This could provide any easy way to let users run a GENESIS simulation via a
web browser without any special implementation of a browser-friendly GUI.
Greg's experience with VNC has been that this is so slow as to be unusable
with anything less than a 10 Mb/sec connectionl.  It wouldn't have the
response needed to interact with a running simulation.  Perhaps some tests
should be run.

Jim believes that we should first address the needs of researchers and
educators who would be willing to install a browser plugin to view
simulations over the web.  This widens the choice beyond Java with AWT.
In two years or so, we can write an educational proposal to fund development
of an alternate GUI for web browser use with whatever is available in
IE and other popular browsers that doesn't require installing additional
plugins.  Of course, our MOOSE/GUI interface should allow this to be done.

Thus, for the initial GUI implementation, we should pick a language/toolkit
that (a) permits the reimplementation of existing XODUS widgets, (b) eases
development efforts, (c) can run in a browser with a plugin that is no
harder for a user to install than Sun's Java runtime environment.

* Issues related to Microsoft Windows

GENESIS 2 is started from a console window under UNIX, or within a
Cygwin-X xterm under Windows.  It should be possible to launch GENESIS 3
graphically under Windows, which does not provide a console window, other
than the deprecated "DOS Window".  The GUI should provide a console window
with a genesis prompt for entering commands interactively, or debugging.
The parser should implement unix-like commands such as piping to "more",
"cd", etc. without access to the OS shell.

* What XODUS 2 widgets and commands to reimplement, and how faithfully?

Dave will make a first pass at dividing existing XODUS classes (widgets)
and commands into priority categories of High, Medium, Low, and "Do Not
Implement" and circulate it for comment.

We might later relax the standards for backwards compatibility of graphical
classes, but initially assume that GENESIS 2 scripts that create graphics
will run.

The Scripts/piriform simulation will not run under GENESIS 3, because it
uses GENESIS 1 oldconn library classes that will not be implemented.  Jim
says that Fabio will write a modern version, using GENESIS 2 newconn
classes that will be implemented in GENESIS 3.

Other new graphical classes (Priority Low): Scatter plots of spike times in
a collection of cells.

* IDE (Integrated Design Environment) for constructing and debugging
   simulations

Task: Decide what we want in an IDE.  (Medium Priority, as long as it does
not place any additional constraints on our design.  High Priority, to see
if Neurospaces can provide a solution.)

Kay Robbins stressed that we should provide a graphical way to inspect and
debug a model.  What can we offer in the GUI for inspection of GENESIS
elements?  This is related to the Neurospaces issue, below.

3. Project management issues
----------------------------

Project management, the development process, source code maintenance,
  coding and documentation standards.

* Management structure:

MOOSE has been managed and implemented by Upi Bhalla, with the aid of one or
two hired programmers who are assigned to specific tasks.  This will
continue to be the case for MOOSE.  GENESIS 3 will consist of the MOOSE
core, plus the contributiions of many other programmers and
user/programmers.

Zhiwei Wang (Director of the UTSA Bioinformatics/Computational Biology
Facility) and Michael (Hardware/software coordinator for the Computationsl
Biology facility) will gradually take over the management of the GENESIS
project.

Documentation standards: Who will set them?

New hires:  We will need to generate job descriptions for needed new
personnel.


* Collaborations:

Management infrastructure will support coordinating input from "volunteer"
developers

Future funding may support collaborations by funding travel and salary for
user/developers

Need to support a publication/communication system built around the model
that allows replication and extension (Upi has experience with this in the
signaling pathway community.)

Can we keep track of the heritage of models as different groups contribute
to the model over time?


* Documentation during development and coding standards. Quality
control for code.


MOOSE is written in ANSI C++ with extensive use of the STL coding standard.

How to assure the quality of user-contributed code and documentation.  This
will generally be implementation of new classes, which will be cleanly
separated from core simulator code.  Can the standards be lower?

* Documentation nomenclature:

GENESIS 2 uses the unfortunate terminology of referring to the templates that
are used to create simulation elements as "objects", and their specific
instantiations as "elements".  Common usage in object-oriented langauges
such as C++ and Java refer to the these templates as "classes" and their
instantiations as "objects".  To avoid confusing both old and new GENESIS
users, it is probably best to avoid the use of the term "objects", and to
refer to "classes" and  "elements".  The term "messages" is used in GENESIS
to refer to persistent connections for the transfer of information between
simulation elements.  This differs from the usual meaning in
object-oriented languages.  Should we refer to them as "connections"?
If so, what about the many GENESIS commands such as "addmsg", "showmsg",
etc.?

* Source code maintenance, repositories, etc.

   o CVS or Subversion for version control? - We will migrate to subversion

   o Use of GNU autoconfigure instead of editing Makefiles? - Yes
     How would this apply to Windows? - We will have precompiled Windows
     binaries without configure

For now we will continue to use
http://sourceforge.net/projects/genesis-sim/ for the CVS (or Subversion)
repository and create a genesis3 tree when it is appropriate.  Later, the
repository may be moved to San Antonio, but this is presently a Low
Priority.  In the very short term, before Upi sets up a Sourceforge
repository for MOOSE, a recent developers version of MOOSE will be
available on the GENESIS developers website, along with material that is
intended for developers rather than users and the general public.

4. Model representation with XML
--------------------------------

Padraig has implemented a NeuroML description of the Maex and De Schutter
granule cell model with 7 types of channels.  NeuroConstruct produces
NEURON and GENESIS scripts from this description that give the same results
when run.  (But a very small time step is needed to get complete agreement.
Which simulator needs the smallest?  Why is this so?)

How to generate XML representation of GENESIS simulation elements?  A
one-to-one mapping between XML and internal objects is the simplest
solution.  Public parts in XML are the things needed for an instantiation
of the object. You can limit the XML to the things needed by physiologists.

We need traversal of messages to produce an XML export of the model.

What about automatic generation of objects from XML? Direct generation
won't work in this case, because GENESIS elements will have fields that are
used in the simulation that are not needed in the XML representation.  Fred
Howell's NeuroML development kit implemention suffers from this problem.

Serialization for saving the state of a simulation is a different issue.
Should we use XML for this? We just need a standard text format that is
fixed that will aid with updates. Something like "simdump" will work--ugly
but practical, and has already been implemened in MOOSE.  This is related
to the issue of binary file formats.

Padraig recommends the intermediate layer with XML as only a format for
model specification for things that will be needed external to the
simulator. Currently messaging really defines a cell. Padraig asks if a
higher order concept of a cell is needed? This would be a object-oriented
concept at that level. Upi says that would be creating a object where one
is not needed. There is currently no object concept falling out of
readcell, but this is redundant.

What about using an XML internally to hold additional information that isn't
needed for the external reperesentation?  There isn't any need for this
complexity, if the representation is only internal to the simulator.
(This is relevant to the discussion of Neurospaces.)

What about using XML for saving the state of an element?  At present,
simdump and simundump are adequate, and they are already implemented in
MOOSE.

Import and meshing of experimental morphology files (e.g. Neurolucida)
(Priority: Low)  This can be done with external tools such as CVAPP and
NeuroConstruct.  This feature can always be added later.

XML model specification as way to use multiple solvers: The XML model
description could be used to determine the specialized solver required by
the model.

5. Possible role of Neurospaces as an internal model representation

Design Decision: (Priority High) The role of Neurospaces as an internal
model representation within GENESIS 3.

Late Friday afternoon and during the evening poster session, Hugo Cornelis
began showing his Neurospaces demonstration to people.  You can read his
abstract in the printed WB06 program, or read a short description of
Neurospaces from a link on the GENESIS developers web page.  In the demo,
Neurospaces was incorporated as an object within a customized GENESIS 2
version.  His GUI, along with the internal representation, allowed
inspection of the details of a large network, providing the sort of
information that Kay Robbins was asking for in an IDE.  She encouraged us
to take a serious look at it.  It also nicely handled the use of hsolve in
the array of cells composing the network.  All the details of specifying
chanmodes and using findsolvefield to access fields of hsolved elements
seemed to be hidden from the user, as we would want it them to be.

I believe that the general consensus of the developers who looked at
it was that it was worth considering, but we we need to quickly decide
whether Hugo should continue in this direction, or to concentrate on
a separate way to implement an hsolver that does not involve Neurospaces.

The question facing us is - Would some version of Neurospaces incorporated
as a GENESIS 3 class provide an easy way to translate between the external
model representation in NeuroML, and an internal representation for the
simulator?  Note that we are not suggesting that Neurospaces replace
NeuroML as the way that models are represented outside of GENESIS.

What are the advantages and disadvantages of incorporating Neurospaces
compared to solving these problems (model inspection, translation from
NeuroML to GENESIS simulation elements, and user-transparent hsolve
implementation) by other means?  Of course, it would have to be rewritten
in C++ and made to work in the context of MOOSE.  The GUI for imspection of
models would have to be redone using the same toolkit that is used in the
XODUS replacement.  Implementation of this last part can be a low priority
in the timeline, but the decision needs to be made soon, so that Hugo
doesn't waste his time on something we decide not to use.

6. Parallel computation within GENESIS 3.


* Implications of parallelization:

Elements should be independently runnable, serializable, and migratable.
No direct method calls will be used, as is the case with the GENESIS 2
ACTIONS.  Pointers will not be used for inter-element refereneces.

User interface should be the same whether serial or parallel
Objects must be independent with possibilities for serialization and migration

Using object tree representation, certain subtrees can be restricted to one
processor for efficiency.

For multiple time scales, a parallel discrete event simulation (as used in
MOOSE) is better than multiple clocks (as used in GENESIS 2 to invoke the
PROCESS action on elements).  Parallel discrete event simulation (PDES)
methods support variable timestep methods in a more natural way.

* Multithreading: Currently, MOOSE does not use multiple threads, but this
should be implemented soon (by whom?).  (Priority: Medium).

Should parsers, GUIs, shells, etc. each be run in their own thread?
There should be separate hreads for interfaces and solvers for numerical
methods.  The GUI and the parser (at least) should have separate threads.

Avoid one thread per element, due to stack size issues.  Reproduceability
of results requires deterministic results from parallel operations.

Need to set up queue-based API for parsers, shell and GUI to run on
threads.

Thread support is not present on some supercomputers; Java even more
problematic.

SMP support - just for solvers or for everything?  (Yet to be resolved.)

Do we need to do anything special to use GENESIS 3 in a GRID computing
environment?  According to Greg, our plans are GRID-compatible.  The GRID
standards are very flexible, and put few constraints on our design.  But
high latency will probably make GRID computing useful for parameter search
or other nearly independent processes.


* Debugging parallel simulations:

Developers spend a lot of time chasing down bugs -- Let's make our lives
easier.  Users, when confronted by core dumps, or mysterious behavior, get
very frustrated and, especially in the early stages, are apt to abandon the
software.  Our goal is to blur distinction between user and developers by
allowing easy extensibility through modules.  Thus many people may be
writing code.

o Make Debugging Easy -- Challenges

- Commercial debuggers are oriented towards programmers.
- Parallel debugging is hard even for programmers.
- Preprocessors put extra distance between what a user/developer writes and
   what a standard debugger displays.

o Make Debugging Easy -- Ideas

   - Take advantage of C++ type checking and avoid making the
     users/developers put casts into their own code.

    - Make sure preprocessors preserve line number information for the
      user-written code.

    - Build sanity-checking methods into most classes.

    - Possibly automate the iterative binary search that developers go
      through to isolate offending code (e.g. on the 946th invocation of
      method y() on object x, memory corruption occurred).

   - Build graphical tools to allow visual inspection of virtually every data
     structure.  (See discussion of IDEs.)

7. Implementing specialized solvers, e.g. hsolve for compartmental models.

Solvers should be fast, transparent, and easy-to-use.  How should these be
handled in GENESIS 3?

Should we restrict SMP to solvers?

How to add new channels or other new classes that should be solved?  How
much generality is needed?

What will be the API for solvers?

We may need a numerical analyst/scientific computation person in order to
resolve issues of multiscale (time and space) computation and multiple
solvers

Doing field potentials (e.g. the GENESIS 2 efield object) will be hard with
hsolve.  But, Hugo has done it with Neurospaces and GENESIS 2.  It will be hard
to parallelize the field potential display.

Which solvers do we need most and what are their features? (hsolve, kinetic
solvers, mechanics solvers, Markov solver)

* kinetics solvers (Some are done, others under development.)
* hsolve equivalent (Priority: High)
* markov solver (Priority: Medium)


* Numerical Analysis: Validation of numerical methods. Timestep vs error.

Automatic picking of time step:  Picking an appropriate time step is hard
for a beginner.  Can error estimates allow the solver to pick a reasonable
default time step?  But, the solver might pick one unrealistically low.
Upi suggests that the user specify a desired accuracy.

8. Communication with users and developers.

Replacement for current manually edited BABEL web page and hand-maintained
mailing list.  Michael Edwards will install GNU Mailman in the coming months
to run the mailing list.  (Priority: Medium)

Tom Morse mentioned that the NEURON users discusssion forums use the open
source phpBB package, which provides for automatic emailing of major
announcements.  We should look into it, or the prototype JBoss forum set up
by Danny Thornton. (Priority: Low)