[SOSlib-discuss] splitting soslib functionality, and high-performance computing

[Rainer M. <ra...@tb...>]

Hi Ben,

moving this discussion to the list, in case anyone else is interested.

On Mon, 2009-09-28 at 11:33 -0400, Ben Allan wrote:
> I'm glad to hear there's still an active community around SOSlib.
> 
> I'm working on a project which really wants to provide its own
> drivers for montecarlo and ode/dae solutions, but to use
> the fairly nice model evaluation engine you all have developed.

So you are only interested in the SBML -> ODEModel part, but not the
Sundials wrapper, correct?

Two things, first for the SBML -> ODEModel: we plan to formalize that
approach, and introduce a stoichiometryMatrix object as an intermediate
between SBML and ODE system. This should allow easier integration with
reaction-network-based model reduction algorithms, or general reaction
network analysis tools. Don't know if you would be interested in this
topic too. Probably most of these algorithms already exist somewhere
with SBML-to-SBML interfaces, but I think it would be a natural
extension for SOSlib.

Second a bit of advertising for the integrator part of SOSlib - although
from what you say, you probably really want to use your own solver
wrappers. Anyways, we do have a separate library which for unresolved
licensing issues is not part of SOSlib (yet?), but uses our sundials
wrappers for efficient parameter estimation (fitting experimental data
stored in the same format as the soslib's commandline tool odeSolver
output), both global and local.
Once the error handling is moved to isntances instead of global, all
should be set up for nice parallel usage.

In principle, the IntegratorInstance was designed so that ideally any
integrator can be plugged in (currently it only distinguishes SBML
models which have only assignment rules and no ODEs, but we'd like to
wrap sundials' IDA (DAE) solver, and keep it open for stochastic solvers
as well). Also, if the current wrapper of SUNDIALS is too overloaded for
high-performance stuff one could think of a stripped down version not
capable of re-initiation or the sensitivity and adjoint solvers (or
three dedicated wrappers for forward, sensitivity and adjoint
integration).

> 
> I did notice the code generation bit in your system; I did some fairly
> similar work for the ascend4 system back in the late 1990s while
> I was at school.

For clarification, there are currently two code generation systems. The
old one just writes the sundials right-hand-side functions as C-code
into a string buffer and compiles this as a dynamic library using the C
compiler used for compilation of the code (gcc et al.). Then the dynamic
library is loaded and pointers to these functions are passed to sundials
(see src/compiler.c). 
The second is really generation of internal compiler machine code
(currently only for intel processors, 32 and 64bit) for each equation
(from libSBML abstract syntax tree), and the hard-coded right-hand-side
function calls this dynamically compiled code for each equation of the
system (see src/arithmeticCompiler.c).

> 
> Is there a proper forum (mailing list? wiki?) for soslib that i might
> join, or is the email reply chain the preferred option to continue
discussing?

Yes, there is, and there are a few people subscribed who were using
soslib on or are interested in high-performance computation and might
also be interested in the above info. If you don't mind I could re-send
this email to sbm...@li...
or you could reply to this email directly to the list after subscribing
at
https://lists.sourceforge.net/lists/listinfo/sbmlsolver-discuss

> 
> A little report on what i've done so far in modifying the code in my
copy:
> () getting it to work with libsbml 4 (2.4 sbml spec).

Nice, are there many changes required?

> () making most of the ODEModel header private instead of public (C-
> style-- splitting it into
> ODEModel_pub.h with more accessor functions, and ODEModel_priv.h with
> the struct bodies).

We once had a complete separation of structures, but due to some
problems with compilation we got rid of it again. Can't remember what
the problem exactly was, but in principle we wouldn't mind splitting
into public and private headers again. 

> () running continously with valgrind to check for insanities.

I use valgrind to, on the command-line tool and the programs in the
example folder, but we unfortunately don't have a dedicated testing
framework.

> 
> After I get the application prototype finished for my client, I'm
hoping
> they will approve further development to make it more maintainable
> and perhaps to upgrade to sbml 3. In particular, the downstream
> users are NOT going to be strong C programmers, so conversion/wrapping
> to a completely c++ interface (preferably complete implementation)
> is of great interest. Then moving on to run on something like CUDA
> would require exactly the code generation approach you already have in
> place.
> 

BTW, we have biology, chemistry, physics and mathematics background,
resp., so there is no dedicated computer scientist involved. Andrew
Finney was the computer scientist (involved in SBML and libSBML once)
who pushed the whole thing towards a library and would also have
preferred C++ interfaces. Unfortunately none of us really speaks C++
currently. But especially the interfaces (input settings and result
structures) would probably be much easier to maintain and wrap in C++.
CUDA sound great.

> Is there a more active code repository than the one at sourceforge, or
is that
> the master?  

Sourceforge repository is master. Do you want developer access? I'd have
to wait for internal discussion though. 
And in principle significant changes should be discussed on the list
before submitting of course.

> Would you be interested in splitting soslib internally
> (or publically)
> into an smbl evaluation engine library with a separate library of
> integrations/statistical
> methods on top of it?

Do you mean separate packages and build structures for these
functionalities, or separate CVS branches? I wouldn't know how that
works. 
In principle, we wouldn't oppose splitting into two libraries. As
mentioned above, we have plans for extending the SBML<->ODE part and
linking it with pure reaction network analysis, separate from the
sundials wrappers, and on the other hand we already have an additional
separate library build on top of the sundials integration and like to
keep it general. Making it more modular is certainly in our interest. 

But as said in the first email, our main limitation currently is time :(

> 
> As an aside, I'm fairly expert in the SIDL/babel tool which produces
fully 2-way
> wrappers to any of the c/c++/fortran/python/java languages to an
implementation
> language (which may also be any of  c/c++/fortran/python/java). It
doesn't have
> the spread of language support (unless you then run swig on the C++
> binding headers)
> that swig does, but it preserves/exposes array and struct arguments
well
> compared to swig. If that kind of technology is of interest, I may be
> able to contribute
> such a wrapper. I've also done work in swig (creating new but
> unpublished language bindings, not
> just using swig) on top of sidl/babel.

This sounds great, but personally I don't know much about neither swig
nor SIDL/babel. Maybe Christoph can comment on that.

Thanks,
Rainer

> 
> Thanks,
> Ben
> 
> On Mon, Sep 28, 2009 at 3:05 AM, Rainer Machne <ra...@tb...>
wrote:
> > Hi Ben,
> >
> > Yes, we do plan some further extensions of SOSlib before a next
release:
> > most importantly
> > * attaching the currenlty global error handling into individual
ODEmodel
> > and integratorInstance objects,
> > * separating the current ODEmodel structures for inverse analysis
> > (objectiveFunction etc.) into separate structures,
> > * making the interface the our internal equation compiler public,
and
> > * providing some wrappers for scripting languages.
> >
> > But since we are in the final months of another project, we
currently
> > can't estimate when this will  the case. The current CVS version is
> > rather stable.
> >
> > In the long run, one possiblity would be to move some of the
interface
> > functionality to C++.
> >
> > You might especially be interested in our internal compilation
> > functionality, which speeds up integration times significantly.
> > Currently, it can be activated by defining "ARITHMETIC_TEST" on
compile
> > time and NOT use the "compileFunctions" option in cvodeSettings.
> >
> > We are open for further discussions and contributions!
> >
> > Rainer
> >
> >
> >
> > On Thu, 2009-09-24 at 15:36 -0400, Ben Allan wrote:
> >> I'm still interested in knowing the development status of soslib,
> >> but I see that changing the version listed in the sbml input
changes
> >> what libsbml thinks the model contains, including the
(dis)appearance
> >> of parameters.
> >> Now I'm off to solve the next bug that comes up...
> >>
> >> thanks,
> >> Ben
> >>
> >> On Thu, Sep 24, 2009 at 3:05 PM, Ben Allan <ba2...@gm...>
wrote:
> >> > Hi,
> >> >
> >> > I would like to know if you are still using/maintaining SOSLib
(or a
> >> > better open-source alternative with a similar purpose)?
> >> >
> >> > I'm working with the current version from sourceforge CVS and I'm
> >> > trying to figure out the solution
> >> > to errors I get when using a particular input file. I would
prefer to
> >> > fix bugs (if that's what it is) instead
> >> > of trying to rewrite soslib from scratch (probably in c++ instead
of
> >> > C). My ultimate aim is to use sbml
> >> > input in very large scale computations, possibly on gpus, with a
small
> >> > embedded engine for evaluating
> >> > the models.
> >> >
> >> > The current fatal symptom is that while evaluating rate laws with
> >> > parameters in their definitions it cannot find
> >> > names of constant parameter names in the om->names list during
evaluateAST.
> >> >
> >> > thanks,
> >> > Ben
> >> >
> >
> >