[SloppyCell-users] Error when trying to use mpi parallelism

[Ally H. <a....@ed...>]

I have trying to use the MPI parallelism functionality of sloppy cell and the code is hanging and simply never terminates.

I have debugged it a bit and now know a more about the cause but I would need to understand the code much better to go any further.

The Dyamics.integrate_sensitivity function sends out the tasks to the workers, processes some data at the master and then waits for each worker to return.  When worker 1 returns the result it is parsed by _parse_sens_result.  This function is throwing a NameError exception which is not caught here so the code does not try to get a result from workers 2,3,4... and hence these workers never terminate and the whole job hangs.

So why does _parse_sens_result throw an exception?  Because N_dyn_vars is not defined.  If I record the stack trace I get:

Traceback (most recent call last):
File "/exports/work/physics_epcc/ahume/sloppycell_anaconda/lib/python2.7/site-packages/SloppyCell-CVS-py2.7-linux-x86_64.egg/SloppyCell/ReactionNetworks/Dynamics.py", line 958, in integrate_sensitivity
_parse_sens_result(result, net, vars_assigned[worker], yout, youtdt, events_occurred)
File "/exports/work/physics_epcc/ahume/sloppycell_anaconda/lib/python2.7/site-packages/SloppyCell-CVS-py2.7-linux-x86_64.egg/SloppyCell/ReactionNetworks/Dynamics.py", line 869, in _parse_sens_result
e.ysens_fired = scipy.concatenate((e.ysens_fired, result[-1][eii].ysens_fired[N_dyn_vars:]))
NameError: global name 'N_dyn_vars' is not defined

The line numbers do not match the code in git as I have added lots of debug lines to get this far. But you will see the line that is causing the problem is the first line in _parse_sens_result that uses the variable N_dyn_vars.

Could we be doing something wrong or is this a bug?  This all comes from a call to: Optimization.fmin_lm_log_params

If need be I can ask the researcher for more details of the model. My role here is to support him by getting this running on a cluster. 

Any advise?

Regards,

Ally

Ally Hume
Software Architect
EPCC, The University of Edinburgh
Tel: 0131 651 3397   Skype: ally.hume









-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.