relax SVN archive

Removed the user function to estimate the R2eff values and errors with scipy.optimize.leastsq.

With the newly implemented Jacobian and Hessian of the exponential decay function, the front-end to
scipy.optimize.leastsq does not serve a purpose.

This is because minfx is now as fast as scipy.optimize.leastsq, and can estimate the errors from the
Jacobian to the exact same numbers as scipy.optimize.leastsq.

In addition to that, the covariance can be calculated by QR decomposition.
This adds additional feature for checking for a singular matrix.

The back-end wilÃl still be kept in place for the coming tim, but could be removed later.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Merged revisions 25344 via svnmerge from
svn+ssh://bugman@.../svn/relax/trunk

........
r25344 | bugman | 2014-08-27 18:56:05 +0200 (Wed, 27 Aug 2014) | 6 lines

Clean up of the declarations in the target_functions.relax_fit C module.

The Python list objects are now declared at the start of the functions, and then PyList_New() is
called later on. This allows the code to compile on certain Windows systems.
........

Clean up of the declarations in the target_functions.relax_fit C module.

The Python list objects are now declared at the start of the functions, and then PyList_New() is
called later on. This allows the code to compile on certain Windows systems.

Merged revisions 25337-25341 via svnmerge from
svn+ssh://bugman@.../svn/relax/trunk

........
r25337 | tlinnet | 2014-08-27 14:23:41 +0200 (Wed, 27 Aug 2014) | 12 lines

By using minfx, and the reported Jacobian, it is now possible to get the exact same error estimation as scipy.optimize.leastsq.

The fatal error was to set the weighting matrix with diagonal elements as the error.
There weights are 1/errors**2.

There is though some un-answered questions left.

The Jacobian used, is the direct derivative of the function.

It is not the chi2 derivative Jacobian.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25338 | tlinnet | 2014-08-27 17:16:04 +0200 (Wed, 27 Aug 2014) | 3 lines

Fixed naming of functions, to better represent what they do in module of estimating R2eff.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25339 | tlinnet | 2014-08-27 17:16:07 +0200 (Wed, 27 Aug 2014) | 5 lines

Implemented the Jacobian of exponential function in Python Code.

This now also gets the same error as leastsq and C code.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25340 | tlinnet | 2014-08-27 17:16:09 +0200 (Wed, 27 Aug 2014) | 3 lines

Tried to implement a safety test for linearly-dependent columns in the co-variance matrix.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25341 | bugman | 2014-08-27 17:40:27 +0200 (Wed, 27 Aug 2014) | 6 lines

Fixes for the relax_disp.r2eff_estimate user function documentation.

This is to allow the relax manual to compile again as the original documentation was causing LaTeX
failures.
........

Merged revisions 25329-25336 via svnmerge from
svn+ssh://bugman@.../svn/relax/trunk

........
r25329 | tlinnet | 2014-08-27 11:12:50 +0200 (Wed, 27 Aug 2014) | 6 lines

Implemented the first try to compute the Variance of R2eff and i0, by the co-variance.

This uses the Jacobian matrix.
The errors calculated, are though way to small compared 2000 Monte-Carlo simulations.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25330 | tlinnet | 2014-08-27 11:29:24 +0200 (Wed, 27 Aug 2014) | 7 lines

Tried to implement the Jacobian from C-code.

This though also report errors which are to small.

Maybe some scaling is wrong.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25331 | tlinnet | 2014-08-27 11:49:21 +0200 (Wed, 27 Aug 2014) | 90 lines

Modified profiling script to calculate timings.

The timings for C-code are:
Simplex, with constraints = 2.192
Simplex, without constraints = 0.216
BFGS, without constraints = 0.079
Newton, without constraints = 0.031

This is pretty pretty fast.

To this profiling script, I would also now add some verification on calculations.

Profile, with constraints, C code, Simplex
Wed Aug 27 11:45:16 2014 /var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmpvJSNrB

724946 function calls (723444 primitive calls) in 2.192 seconds

Ordered by: cumulative time
List reduced from 307 to 1 due to restriction <1>

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 2.192 2.192 profiling_relax_fit.py:314(verify)

Verify, without constraints, C code, Simplex
Wed Aug 27 11:45:16 2014 /var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmphJ9GV3

76042 function calls (74654 primitive calls) in 0.216 seconds

Ordered by: cumulative time
List reduced from 132 to 1 due to restriction <1>

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.216 0.216 profiling_relax_fit.py:314(verify)

Verify, without constraints, C code BFGS
Wed Aug 27 11:45:16 2014 /var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmp78MZjS

25618 function calls (24230 primitive calls) in 0.079 seconds

Ordered by: cumulative time
List reduced from 153 to 1 due to restriction <1>

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.079 0.079 profiling_relax_fit.py:314(verify)

Verify, without constraints, C code Newton
Wed Aug 27 11:45:16 2014 /var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmp2IPgFC

14572 function calls (13184 primitive calls) in 0.031 seconds

Ordered by: cumulative time
List reduced from 169 to 1 due to restriction <1>

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.031 0.031 profiling_relax_fit.py:314(verify)

Verify, with constraints, Python, Simplex
Wed Aug 27 11:45:20 2014 /var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmpQUUY4L

808444 function calls (806942 primitive calls) in 3.108 seconds

Ordered by: cumulative time
List reduced from 141 to 1 due to restriction <1>

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 3.108 3.108 profiling_relax_fit.py:353(verify_pyt)

Verify, without constraints, Python, Simplex
Wed Aug 27 11:45:20 2014 /var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmp3nFBTq

87492 function calls (86104 primitive calls) in 0.320 seconds

Ordered by: cumulative time
List reduced from 131 to 1 due to restriction <1>

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.320 0.320 profiling_relax_fit.py:353(verify_pyt)

Verify, without constraints, Python Scipy
Wed Aug 27 11:45:20 2014 /var/folders/ww/1jkhkh315x57jglgxnr9g24w0000gp/T/tmpo1HoWn

6600 function calls (5212 primitive calls) in 0.020 seconds

Ordered by: cumulative time
List reduced from 137 to 1 due to restriction <1>

ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.020 0.020 profiling_relax_fit.py:388(verify_sci)

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25332 | tlinnet | 2014-08-27 13:22:41 +0200 (Wed, 27 Aug 2014) | 3 lines

Tried to verify solution to profiling script.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25333 | tlinnet | 2014-08-27 13:22:43 +0200 (Wed, 27 Aug 2014) | 5 lines

Set the constraints=False, when doing Monte-Carlo simulations for R2eff.

This is to speed-up the Monte-Carlo simulations by a factor X10, when estimating the error for R2eff.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25334 | tlinnet | 2014-08-27 13:22:45 +0200 (Wed, 27 Aug 2014) | 9 lines

Implemented the use of "Newton" as minimisation algorithm for R2eff curve fitting instead of simplex.

Running the test script:
test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/2_pre_run_r2eff.py

For 50 Monte-Carlo simulations, the time drop from:
3 minutes and 13 s, to 1 min an 5 seconds.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.
........
r25335 | bugman | 2014-08-27 13:38:24 +0200 (Wed, 27 Aug 2014) | 6 lines

Changed the relax_fit.py sample script to use Newton rather than Simplex optimisation.

This can lead to significantly faster optimisation times, as shown in the commit message
http://article.gmane.org/gmane.science.nmr.relax.scm/23081.
........
r25336 | bugman | 2014-08-27 13:42:43 +0200 (Wed, 27 Aug 2014) | 9 lines

Changed the optimisation description in the relaxation curve-fitting chapter of the manual.

The script example has been converted to match the sample script, replacing the Nelder-Mead simplex
algorithm with Newton optimisation, and removing the argument turning diagonal scaling off. All the
text about only the simplex algorithm being supported due to the missing gradients and Hessians in
the C module have been deleted. The text that linear constraints are not supported has also been
removed - but this was fixed when the logarithmic barrier constraint algorithm was added to minfx.
........

Fixes for the relax_disp.r2eff_estimate user function documentation.

This is to allow the relax manual to compile again as the original documentation was causing LaTeX
failures.

Tried to implement a safety test for linearly-dependent columns in the co-variance matrix.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Implemented the Jacobian of exponential function in Python Code.

This now also gets the same error as leastsq and C code.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

Fixed naming of functions, to better represent what they do in module of estimating R2eff.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.

By using minfx, and the reported Jacobian, it is now possible to get the exact same error estimation as scipy.optimize.leastsq.

The fatal error was to set the weighting matrix with diagonal elements as the error.
There weights are 1/errors**2.

There is though some un-answered questions left.

The Jacobian used, is the direct derivative of the function.

It is not the chi2 derivative Jacobian.

task #7822(https://gna.org/task/index.php?7822): Implement user function to estimate R2eff and associated errors for exponential curve fitting.