relax SVN archive

Merged revisions 25214-25221,25223-25229 via svnmerge from
svn+ssh://bugman@.../svn/relax/trunk

........
r25214 | tlinnet | 2014-08-22 14:48:11 +0200 (Fri, 22 Aug 2014) | 21 lines

Modified following functions:

Time points are now saved at the [ei][mi][oi][di] index level.
At this index Ãlevelall time points are saved for the R2eff point.

- interpolate_disp()
To interpolate time points, all time points through the original dispersion points di, are
collected and then made unique. This time list can potentially be the largest of all time lists.

- interpolate_offset()
To interpolate time points, all time points through the original offset points, and then dispersion points di, are
collected and then made unique. This time list can potentially be the largest of all time lists.

- plot_disp_curves_to_file()
To acquire the original relax_times points.

- return_r2eff_arrays()
To save all time points on the level of [ei][mi][oi][di].
At this index level, it will be a numpy array list with all time values used for fitting.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25215 | tlinnet | 2014-08-22 14:48:18 +0200 (Fri, 22 Aug 2014) | 3 lines

Modified back_calc_r2eff() to accept interpolated timepoints.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25216 | tlinnet | 2014-08-22 14:48:19 +0200 (Fri, 22 Aug 2014) | 3 lines

Modified target function of relax disp, to use the the new list of time points, which are of higher dimension.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25217 | tlinnet | 2014-08-22 14:48:21 +0200 (Fri, 22 Aug 2014) | 8 lines

Fix to systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1()

After the relaxation times have been fixed, this model now return reasonable chi2 values.

The reported parameters are though quite different from all other models, and it seems something may
still be wrong.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25218 | tlinnet | 2014-08-22 14:53:25 +0200 (Fri, 22 Aug 2014) | 3 lines

Fix for time not extracted for CPMG experiments in target_function-

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25219 | tlinnet | 2014-08-22 15:22:36 +0200 (Fri, 22 Aug 2014) | 3 lines

Fix for interpolating time points, when producing xmgrace files for CPMG experiments.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25220 | tlinnet | 2014-08-22 15:22:38 +0200 (Fri, 22 Aug 2014) | 5 lines

Fix for systemtest Relax_disp.test_exp_fit(), where the spin.isotope was not set.

The new call to return_r2eff_arrays(), when producing graphs, raise RelaxSpinTypeError() if no isotope is set.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25221 | bugman | 2014-08-22 15:50:08 +0200 (Fri, 22 Aug 2014) | 5 lines

Modified the Relax_disp.test_r1rho_kjaergaard_missing_r1 system test to pass on 64-bit Linux systems.

The accuracy of the checks of the optimised values have been decreased.
........
r25223 | tlinnet | 2014-08-22 16:27:19 +0200 (Fri, 22 Aug 2014) | 3 lines

Moved the storing of relax time up before check of missing data in return_r2eff_arrays().

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25224 | tlinnet | 2014-08-22 16:27:23 +0200 (Fri, 22 Aug 2014) | 3 lines

Fix for systemtest not adding spin.isotope to setup information.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25225 | tlinnet | 2014-08-22 16:27:25 +0200 (Fri, 22 Aug 2014) | 3 lines

Fix for looping over data indices, where tilt_angles has the si index.

bug #22461(https://gna.org/bugs/?22461): NS R1rho 2-site_fit_r1 has extremely high chi2 value in systemtest Relax_disp.test_r1rho_kjaergaard_missing_r1.
........
r25226 | bugman | 2014-08-22 19:00:56 +0200 (Fri, 22 Aug 2014) | 6 lines

Added Nikolai's original Matlab code to the lib.dispersion.ns_r1rho_2site module docstring.

This is the code taken directly form the original funNumrho.m file, which was the origin of the code
in this module.
........
r25227 | tlinnet | 2014-08-22 19:17:19 +0200 (Fri, 22 Aug 2014) | 7 lines

Further extended the profiling script for curve fitting.

Now profiling is in place for the implemented C code method in relax.

A similar code should now be devised for numpy array for comparing.

But this profiling shows that when contraints=True, is slowing down this procedure by a factor 10 X !
........
r25228 | tlinnet | 2014-08-25 01:08:44 +0200 (Mon, 25 Aug 2014) | 12 lines

Further improved the profiling of relax curve fit.

This profiling shows, that Python code is about twice as slow as the C code implemented.

But it also shows that optimising with scipy.optimize.leastsq is 20 X faster.
It also gives reasonable error values.

Combining a function for a linear fit to guess the initial values, together
with scipy optimise, will be an extreme time win for estimating R2eff values fast.

A further test would be to use relax Monte-Carlo simulations for say 1000-2000 iterations,
and compare to the errors extracted from estimated covariance.
........
r25229 | tlinnet | 2014-08-25 01:08:46 +0200 (Mon, 25 Aug 2014) | 31 lines

Added verification script, that shows that using scipy.optimize.leastsq reaches the exact same parameters as minfx for exponential curve fitting.

The profiling shows that scipy.optimize.leastsq is 10X as fast as using minfx (with no linear constraints.)
scipy.optimize.leastsq is a wrapper around wrapper around MINPACK's lmdif and lmder algorithms.

MINPACK is a FORTRAN90 library which solves systems of nonlinear equations, or carries out the least squares minimization of the residual of a set of linear or nonlinear equations.

The verification script also shows, that a very heavy and time consuming monte carlo simulation of 2000 steps, reaches the same
errors as the errors reported by scipy.optimize.leastsq.

The return from scipy.optimize.leastsq, gives the estimated co-variance.
Taking the square root of the co-variance corresponds with 2X error reported by minfx.

This could be an extremely time saving step, when performing model fitting in R1rho, where
the errors of the R2eff values, are estimited by Monte-Carlo simulations.

The following setup illustrates the problem.
This was analysed on a: MacBook Pro, 13-inch, Late 2011.
Witn no multi-core setup.

Script running is:
test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py

This script analyses just the R2eff values for 15 residues.
It estimates the errors of R2eff based on 2000 Monte Carlo simulations.
For each residues, there is 14 exponential graphs.

The script was broken after 35 simulations.
This was measured to 20 minutes.
So 500 simulations would take about 4.8 Hours.
The R2eff values and errors can by scipy.optimize.leastsq can instead be calculated in: 15 residues * 0.02 seconds = 0.3 seconds.
........

Turned off the optimisation constraints for the 'R2eff' model in the dispersion auto-analysis.

This follows from http://thread.gmane.org/gmane.science.nmr.relax.scm/22977/focus=6829.

This model does not require constraints at all, and the constraints only cause the optimisation to
take 10x longer to complete. Therefore the constraint flag has been set to False for the model.

Fixes for the new target_functions.relax_fit.jacobian() function.

The Python list of lists is now correctly created and returned.

Implementation of the target_functions.relax_fit.jacobian() function.

This follows from the discussions at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807.

The function will calculate the Jacobian matrix for the exponential curve-fitting module. The
Jacobian can be used to directly calculate the covariance matrix, for example as described at
https://www.gnu.org/software/gsl/manual/html_node/Computing-the-covariance-matrix-of-best-fit-parameters.html.
The Jacobian is calculated using the help of the new exponential_dI() and exponential_dR() functions
in the target_functions/exponential.c file. These calculate the partial derivatives of the
exponential curve with respect to each model parameter separately.

The implementation still needs testing and debugging.

Removal of unnecessary imports in specific_analyses.relax_disp.optimisation().

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Fix for missing imports.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Implemented first try to minimise with minfx in estimate_r2eff() function.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Split function to minimise with scipy.optimize.leastsq out in estimate_r2eff module.

This is to prepare for implementing with minfx.

Isolated all code related to user function:

relax_disp.r2eff_estimate() into independent module file.

All has been isolated to: specific_analyses.relax_disp.estimate_r2eff.

Moved target function for curve fitting with scipy into specific_analyses.relax_disp.estimate_r2eff.

This will later include the backend specific_analyses.relax_disp.optimisation.estimate_r2eff()
function and the code in the target_functions package.

The code in target_functions.relax_disp_curve_fit is a lot more than just a target function, so it doesn't really belong in this package.

This is also to isolate this experimental feature.