relax SVN archive

Modified profiling script to use the new estimate R2eff module.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Added initial documentation for multifit_covar.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Merged revisions 25254-25287 via svnmerge from
svn+ssh://bugman@.../svn/relax/trunk

........
r25254 | tlinnet | 2014-08-25 20:03:40 +0200 (Mon, 25 Aug 2014) | 6 lines

Initial try to form the Jacobian and Hessian matrix for exponential decay.

This can be tried with systemtest:
relax -s Relax_disp.test_estimate_r2eff_error

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25255 | tlinnet | 2014-08-25 20:57:15 +0200 (Mon, 25 Aug 2014) | 10 lines

Tried algorithms:

newton
Newton-CG
Steepest descent
Fletcher-Reeves

None of them work.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25256 | tlinnet | 2014-08-25 23:00:39 +0200 (Mon, 25 Aug 2014) | 6 lines

Intermediate step in estimate R2eff module.

It seems that minfx is minimising in a quadratic space because of the power of chi2, while
the general input to scipy.optimize does not do this.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25257 | tlinnet | 2014-08-25 23:00:41 +0200 (Mon, 25 Aug 2014) | 3 lines

Cleaned up target function for leastsq, since arguments to function can be extracted from class.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25258 | tlinnet | 2014-08-25 23:00:43 +0200 (Mon, 25 Aug 2014) | 5 lines

Tried to implement with scipy.optimize.fmin_ncg and scipy.optimize.fmin_cg, but cannot get it to work.

The matrices are not aligned well.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25259 | bugman | 2014-08-26 08:58:39 +0200 (Tue, 26 Aug 2014) | 6 lines

Implemented the chi-squared gradient as a C module for the target functions.

This simply translates the Python code into C to allow any target function C modules to build its
own gradient function.
........
r25260 | bugman | 2014-08-26 09:00:14 +0200 (Tue, 26 Aug 2014) | 6 lines

Implemented the target_functions.relax_fit.dfunc() gradient function.

This is using the Python/C interface to provide a Python function for calculating and returned the
chi-squared gradient for the exponential curves.
........
r25261 | bugman | 2014-08-26 09:06:32 +0200 (Tue, 26 Aug 2014) | 6 lines

Implementation of the specific_analyses.relax_fit.optimisation.dfunc_wrapper() function.

This interfaces with the target_functions.relax_fit C module and converts the gradient from a Python
list to a numpy array.
........
r25262 | bugman | 2014-08-26 09:18:02 +0200 (Tue, 26 Aug 2014) | 3 lines

The exponential curve-fitting gradient is now scaled by the scaling matrix.
........
r25263 | bugman | 2014-08-26 09:18:55 +0200 (Tue, 26 Aug 2014) | 3 lines

Clean up of the end of the target_functions.relax_fit.dfunc() function.
........
r25264 | bugman | 2014-08-26 09:25:32 +0200 (Tue, 26 Aug 2014) | 3 lines

Fixes for the target function chi-squared gradient C function.
........
r25265 | bugman | 2014-08-26 09:31:44 +0200 (Tue, 26 Aug 2014) | 6 lines

Fixes for all of the exponential functions in target_functions/exponential.c.

The condition whereby Rx is zero is now setting the value correctly - the exponential will be 1, not
zero, hence the intensity and gradient values should not be zero.
........
r25266 | bugman | 2014-08-26 10:23:05 +0200 (Tue, 26 Aug 2014) | 3 lines

Clean up of the target function C files (spacing fixes and removal of unused code).
........
r25267 | bugman | 2014-08-26 10:29:38 +0200 (Tue, 26 Aug 2014) | 3 lines

Changed the argument and variable names in the C code chi-squared gradient function.
........
r25268 | bugman | 2014-08-26 11:07:07 +0200 (Tue, 26 Aug 2014) | 6 lines

Modified all of the exponential curve functions and gradients in the C target function module.

Instead of passing in the parameter vector, now the I0 and R values are passed in separately. This
is for greater code flexibility as the parameter order does not need to be known.
........
r25269 | bugman | 2014-08-26 11:09:30 +0200 (Tue, 26 Aug 2014) | 5 lines

The parameter index is now passed into exponential_dI0() and exponential_dR().

This is for the relaxation curve-fitting C module so that the indices are not hardcoded.
........
r25270 | bugman | 2014-08-26 11:16:02 +0200 (Tue, 26 Aug 2014) | 5 lines

The I0 and R parameter indices are now defined in the target_function/relax_fit.h header file.

This is to abstract the exponential curve parameter indices even more.
........
r25271 | tlinnet | 2014-08-26 11:21:36 +0200 (Tue, 26 Aug 2014) | 5 lines

Big cleanup of estimate R2eff module.

This is to make the documentation more easy to read and understand.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25272 | bugman | 2014-08-26 11:35:25 +0200 (Tue, 26 Aug 2014) | 5 lines

Created 2 unit tests for the target_functions.relax_fit relax C module.

These check the func() and dfunc() Python methods exposed by the module.
........
r25273 | bugman | 2014-08-26 11:39:01 +0200 (Tue, 26 Aug 2014) | 3 lines

The relax_fit C module unit tests now check if the parameter scaling is functional.
........
r25274 | tlinnet | 2014-08-26 11:47:24 +0200 (Tue, 26 Aug 2014) | 3 lines

Added several comments to the R2 eff estimate module.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25275 | bugman | 2014-08-26 12:06:18 +0200 (Tue, 26 Aug 2014) | 8 lines

Added a script and log file for calculating the numerical gradient for an exponential curve.

This uses the data at http://thread.gmane.org/gmane.science.nmr.relax.devel/6807/focus=6840 and
calculates the gradient using the scipy.misc.derivative() function both at the minimum and at a
point away from the minimum. The values will be used to construct a unit test to check the C module
implementation.
........
r25276 | bugman | 2014-08-26 12:10:46 +0200 (Tue, 26 Aug 2014) | 3 lines

Created a unit test to check the dfunc() function of the relax_fit C module off the minimum.
........
r25277 | bugman | 2014-08-26 12:14:28 +0200 (Tue, 26 Aug 2014) | 5 lines

Fixes for the target_functions.relax_fit C module unit tests.

All values are now set to floats to avoid integer division issues.
........
r25278 | bugman | 2014-08-26 12:17:23 +0200 (Tue, 26 Aug 2014) | 5 lines

Activated parameter scaling of the gradient in the test_dfunc_off_minimum() unit test.

This is the test class test_suite.unit_tests._target_functions.test_relax_fit.Test_relax_fit.
........
r25279 | bugman | 2014-08-26 12:20:49 +0200 (Tue, 26 Aug 2014) | 5 lines

The exponential curve numeric gradient script now uses only floating point numbers.

This is to avoid integer truncation problems.
........
r25280 | bugman | 2014-08-26 12:24:57 +0200 (Tue, 26 Aug 2014) | 5 lines

Fix for the script for calculating the numerical gradient for an exponential curve.

The off-minimum derivative was not correctly calculated.
........
r25281 | bugman | 2014-08-26 12:29:49 +0200 (Tue, 26 Aug 2014) | 3 lines

Increased the printouts for the script for calculating the numerical gradient for an exponential curve.
........
r25282 | bugman | 2014-08-26 12:44:12 +0200 (Tue, 26 Aug 2014) | 6 lines

Bug fix for the chi-squared gradient calculation in the C module.

The definition of the square() function needed extra brackets so that the 1/error**2 calculation
would be 1/(error*error) rather than the incorrect 1/error*error form.
........
r25283 | bugman | 2014-08-26 12:47:28 +0200 (Tue, 26 Aug 2014) | 6 lines

Fix for the test_dfunc_off_minimum() unit test.

This is the test class test_suite.unit_tests._target_functions.test_relax_fit.Test_relax_fit. The
wrong gradient was being scaled.
........
r25284 | bugman | 2014-08-26 12:51:21 +0200 (Tue, 26 Aug 2014) | 8 lines

Switched the optimisation algorithm in test_suite/system_tests/scripts/relax_fit.py.

This script, used by the Relax_fit.test_curve_fitting_height and Relax_fit.test_curve_fitting_volume
system tests, now uses the BFGS optimisation. This is to demonstrate that the exponential curve
gradient function dfunc() is implemented correctly and that more advanced optimisation algorithms
can be used (excluding those that require the full Hessian d2func() function).
........
r25285 | tlinnet | 2014-08-26 13:12:14 +0200 (Tue, 26 Aug 2014) | 24 lines

Got the method of 'Steepest descent' to work properly, by specifying the Jacobian correctly.

The Jacobian was derived according to the chi2 function.

The key point was to evaluate the two derivative arrays for all times points, and
and then sum each of the two arrays together, before constructing the Jacobian.

This clearly shows the difference between minfx and scipy.optimize.leastsq.

scipy.optimize.leastsq takes as input a function F(x0), which should return the array
of weighted differences between function value and measured values:
" 1. / self.errors * (self.calc_exp(self.times, *params) - self.values) "

This will be an array with number of elements 'i' corresponding to number of elements.
scipy.optimize.leastsq then internally evaluates the sum of squares -> sum[ (O - E)**2 ],
and minimises this. This is the chi2.

Minfx requires the function to minimise before hand.
So, the "func" should be chi2.

Then the dfunc, and d2func, should be derivative of chi2, bum all elements in the array should
still be summed together.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25286 | tlinnet | 2014-08-26 13:23:44 +0200 (Tue, 26 Aug 2014) | 7 lines

Got the Quasi-Newton BFGS to work.

This uses only the gradient, this gets the same results as 2000 Monte-Carlo with simplex and scipy.optimize.leastsq.

Error estimation still not provided.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25287 | tlinnet | 2014-08-26 13:23:47 +0200 (Tue, 26 Aug 2014) | 5 lines

Removed all code regarding scipy.optimize fmin_cg and fmin_ncg.

This problem should soon be able to be solved with minfx.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........

Removed all code regarding scipy.optimize fmin_cg and fmin_ncg.

This problem should soon be able to be solved with minfx.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Got the Quasi-Newton BFGS to work.

This uses only the gradient, this gets the same results as 2000 Monte-Carlo with simplex and scipy.optimize.leastsq.

Error estimation still not provided.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Got the method of 'Steepest descent' to work properly, by specifying the Jacobian correctly.

The Jacobian was derived according to the chi2 function.

The key point was to evaluate the two derivative arrays for all times points, and
and then sum each of the two arrays together, before constructing the Jacobian.

This clearly shows the difference between minfx and scipy.optimize.leastsq.

scipy.optimize.leastsq takes as input a function F(x0), which should return the array
of weighted differences between function value and measured values:
" 1. / self.errors * (self.calc_exp(self.times, *params) - self.values) "

This will be an array with number of elements 'i' corresponding to number of elements.
scipy.optimize.leastsq then internally evaluates the sum of squares -> sum[ (O - E)**2 ],
and minimises this. This is the chi2.

Minfx requires the function to minimise before hand.
So, the "func" should be chi2.

Then the dfunc, and d2func, should be derivative of chi2, bum all elements in the array should
still be summed together.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Switched the optimisation algorithm in test_suite/system_tests/scripts/relax_fit.py.

This script, used by the Relax_fit.test_curve_fitting_height and Relax_fit.test_curve_fitting_volume
system tests, now uses the BFGS optimisation. This is to demonstrate that the exponential curve
gradient function dfunc() is implemented correctly and that more advanced optimisation algorithms
can be used (excluding those that require the full Hessian d2func() function).

Fix for the test_dfunc_off_minimum() unit test.

This is the test class test_suite.unit_tests._target_functions.test_relax_fit.Test_relax_fit. The
wrong gradient was being scaled.

Bug fix for the chi-squared gradient calculation in the C module.

The definition of the square() function needed extra brackets so that the 1/error**2 calculation
would be 1/(error*error) rather than the incorrect 1/error*error form.

Increased the printouts for the script for calculating the numerical gradient for an exponential curve.