relax SVN archive

Merged revisions 23760-23770,23787-23789,23791-23806,23813-23855 via svnmerge from
svn+ssh://bugman@.../svn/relax/branches/disp_spin_speed

................
r23760 | tlinnet | 2014-06-10 01:01:49 +0200 (Tue, 10 Jun 2014) | 5 lines

Moved the calculation of pA and kex out off all loopes.

This was done by having two special 1/0 spin structure arrays.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23761 | tlinnet | 2014-06-10 01:01:51 +0200 (Tue, 10 Jun 2014) | 3 lines

Removed dw_frq_a numpy array, as it was not necessary.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23762 | tlinnet | 2014-06-10 01:01:53 +0200 (Tue, 10 Jun 2014) | 6 lines

Removed all looping over spin and spectrometer frequency.

This is the last loop!
Wuhu.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23763 | tlinnet | 2014-06-10 01:01:55 +0200 (Tue, 10 Jun 2014) | 3 lines

Reordered arrays for beauty of code.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23764 | tlinnet | 2014-06-10 01:01:57 +0200 (Tue, 10 Jun 2014) | 3 lines

Made the back_calc array be initiated as copy of the values array.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23765 | tlinnet | 2014-06-10 01:01:59 +0200 (Tue, 10 Jun 2014) | 3 lines

Small edit to profiling script, to help bug finding.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23766 | tlinnet | 2014-06-10 01:02:00 +0200 (Tue, 10 Jun 2014) | 3 lines

Fixed that arrays are correctly initiated with one or zero values.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23767 | tlinnet | 2014-06-10 01:02:02 +0200 (Tue, 10 Jun 2014) | 5 lines

Very important fix, for only replacing part of data array which have Nan values.

Before, all values were replaced, which was wrong.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23768 | tlinnet | 2014-06-10 01:02:04 +0200 (Tue, 10 Jun 2014) | 8 lines

Needed to increase the relative tolerance when testing if pA array is 1.

Now system test Relax_disp.test_hansen_cpmg_data_missing_auto_analysis passes.

Also added some comments lines, to prepare for mask replace of values.
For example if only some of etapos values should be replaced.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23769 | tlinnet | 2014-06-10 01:02:06 +0200 (Tue, 10 Jun 2014) | 3 lines

Restored profiling script to normal.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23770 | tlinnet | 2014-06-10 01:21:14 +0200 (Tue, 10 Jun 2014) | 3 lines

Made the logic and comments much clearer about how to reshape, expand axis, and tile numpy arrays.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23787 | tlinnet | 2014-06-10 16:51:30 +0200 (Tue, 10 Jun 2014) | 3 lines

Removed line "# -*- coding: utf-8 -*-" which was automatically inserted by new python code editor.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23788 | tlinnet | 2014-06-10 16:51:33 +0200 (Tue, 10 Jun 2014) | 10 lines

Implemented a masked array search for where "missing" array is equal 1.

This makes it possible to replace all values with this mask, from the value array.

This eliminates the last loops over the missing values.

It took over 4 hours to figure out, that the mask should be called with mask.mask,
to return the same fulls structure,

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23789 | tlinnet | 2014-06-10 16:51:35 +0200 (Tue, 10 Jun 2014) | 3 lines

Yet another small improvement for the profiling script.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23791 | tlinnet | 2014-06-10 18:40:55 +0200 (Tue, 10 Jun 2014) | 5 lines

Removed the multi dimensional structure of pA.

pA is not multi-dimensional, and can just be multiplied with numpy arrays.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23792 | tlinnet | 2014-06-10 18:40:59 +0200 (Tue, 10 Jun 2014) | 3 lines

Fix for testing of pA in lib function, when pA is just float.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23793 | tlinnet | 2014-06-10 18:41:01 +0200 (Tue, 10 Jun 2014) | 3 lines

Modified unit tests, so pA is sent to target function as float instead of array.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23794 | tlinnet | 2014-06-10 18:41:03 +0200 (Tue, 10 Jun 2014) | 5 lines

Removed the multi dimensional structure of kex.

kex is not multi-dimensional, and can just be multiplied with numpy arrays.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23795 | tlinnet | 2014-06-10 18:41:05 +0200 (Tue, 10 Jun 2014) | 3 lines

Fix for testing of kex in lib function, when kex is just float.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23796 | tlinnet | 2014-06-10 18:41:06 +0200 (Tue, 10 Jun 2014) | 3 lines

Modified unit tests, so kex is sent to target function as float instead of array.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23797 | tlinnet | 2014-06-10 18:41:08 +0200 (Tue, 10 Jun 2014) | 10 lines

Important fix for replacing values if eta_pos > 700 is violated.

This fixes systemtest: Relax_disp.test_sod1wt_t25_to_cr72, which failed after making kex to a numpy float.

The trick is to make a numpy mask which stores the position where to replace the values.
Then replace the values just before last return.

This makes sure, that not all values are changed.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23798 | tlinnet | 2014-06-10 18:41:10 +0200 (Tue, 10 Jun 2014) | 5 lines

Increased the kex speed to 1e7 in clustered unit tests cases.

This is to demonstrate where there will be no excange.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23799 | tlinnet | 2014-06-10 18:54:28 +0200 (Tue, 10 Jun 2014) | 3 lines

Added a multi-dimensional numpy array chi2 value calculation function.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23800 | tlinnet | 2014-06-10 18:54:31 +0200 (Tue, 10 Jun 2014) | 3 lines

Called the newly created chi2 function to calculate for multi dimensional numpy arrays.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23801 | tlinnet | 2014-06-10 19:11:07 +0200 (Tue, 10 Jun 2014) | 5 lines

Renamed chi2_ND to chi2_rankN.

This is a better name for representing multiple axis calculation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23802 | tlinnet | 2014-06-10 20:58:15 +0200 (Tue, 10 Jun 2014) | 5 lines

Made special ei, si, mi, and oi numpy structure array.

This is for rapid speed-up of numpy array creation in target function.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23803 | tlinnet | 2014-06-10 20:58:19 +0200 (Tue, 10 Jun 2014) | 3 lines

Code clean-up in lib function.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23804 | tlinnet | 2014-06-10 20:58:22 +0200 (Tue, 10 Jun 2014) | 3 lines

Replace np.tile with tile.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23805 | tlinnet | 2014-06-10 20:58:24 +0200 (Tue, 10 Jun 2014) | 3 lines

Replaced self.spins_a with self.disp_struct.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23806 | tlinnet | 2014-06-10 21:06:12 +0200 (Tue, 10 Jun 2014) | 3 lines

Removed un-used self structures in __init__of class.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23813 | tlinnet | 2014-06-11 08:28:25 +0200 (Wed, 11 Jun 2014) | 3 lines

Made iinitialisation structures for dw.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23814 | tlinnet | 2014-06-11 08:28:27 +0200 (Wed, 11 Jun 2014) | 3 lines

Initial try to reshape dw faster.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23815 | tlinnet | 2014-06-11 08:40:38 +0200 (Wed, 11 Jun 2014) | 5 lines

Switched to use self.ei, self.si, self.mi, self.oi, self.di.

This is for better reading of code.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23816 | tlinnet | 2014-06-11 08:40:40 +0200 (Wed, 11 Jun 2014) | 3 lines

Comment out the sys.exit(), which would make the code fail for wrong calculation of dw.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23817 | tlinnet | 2014-06-11 08:46:00 +0200 (Wed, 11 Jun 2014) | 7 lines

Copied profiling script for CPMG model CR72 to R1rho DPL94 model.

The framework of the script will be the same, but the data a little different.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23818 | tlinnet | 2014-06-11 10:08:02 +0200 (Wed, 11 Jun 2014) | 3 lines

Started converting profiling script to DPL94.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23819 | tlinnet | 2014-06-11 10:08:05 +0200 (Wed, 11 Jun 2014) | 5 lines

Replaced self.(ei,si,mi,oi,di) with self.(NE,NS,NM,NO,ND).

These numbers represents the maximum number of dimensions, instead of index.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23820 | tlinnet | 2014-06-11 10:08:06 +0200 (Wed, 11 Jun 2014) | 3 lines

Added the ei index, when creating the first dw_mask.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23821 | tlinnet | 2014-06-11 10:08:08 +0200 (Wed, 11 Jun 2014) | 3 lines

Reordered how the structures dw init structures are created.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23822 | tlinnet | 2014-06-11 10:08:10 +0200 (Wed, 11 Jun 2014) | 3 lines

Clearing the dw_struct before calculation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23823 | tlinnet | 2014-06-11 10:16:27 +0200 (Wed, 11 Jun 2014) | 8 lines

Started using the new way of constructing dw.

This is for running system tests.

Note, somewhere in the dw array, the frequencies will be different between the two implementations.
But apparently, this does not matter.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23824 | tlinnet | 2014-06-11 10:46:42 +0200 (Wed, 11 Jun 2014) | 3 lines

Inserted temporary method to switch for profiling.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23825 | tlinnet | 2014-06-11 10:46:44 +0200 (Wed, 11 Jun 2014) | 3 lines

First try to speed-up the old dw structure calculation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23826 | tlinnet | 2014-06-11 13:12:12 +0200 (Wed, 11 Jun 2014) | 3 lines

Simplified calculation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23827 | tlinnet | 2014-06-11 13:12:14 +0200 (Wed, 11 Jun 2014) | 3 lines

Yet another try to implement a fast dw structure method.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23828 | tlinnet | 2014-06-11 13:12:16 +0200 (Wed, 11 Jun 2014) | 6 lines

Implemented the fastest way to calculate the dw structure.

This uses the numpy ufunc multiply.outer function to create the outer array, and then multiply
with the frqs_structure.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23829 | tlinnet | 2014-06-11 13:12:18 +0200 (Wed, 11 Jun 2014) | 3 lines

Renamed dw temporary structure to generic structure.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23830 | tlinnet | 2014-06-11 13:12:20 +0200 (Wed, 11 Jun 2014) | 3 lines

Restructured the calculation of R20A and R20B to the most efficient way.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23831 | tlinnet | 2014-06-11 13:12:21 +0200 (Wed, 11 Jun 2014) | 3 lines

Made the lib/cr72.py to a numpy multi dimensional numpy array calculation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23832 | tlinnet | 2014-06-11 14:05:35 +0200 (Wed, 11 Jun 2014) | 5 lines

Changed the catching when dw is zero, to use masked array.

Implemented backwards compability with unit tests.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23833 | tlinnet | 2014-06-11 14:05:37 +0200 (Wed, 11 Jun 2014) | 5 lines

Bugfix for testing if kex is zero.

It was tested if kex was equal 1.0.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23834 | tlinnet | 2014-06-11 14:54:37 +0200 (Wed, 11 Jun 2014) | 3 lines

Implemented masked replacement if fact is less that 1.0.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23835 | tlinnet | 2014-06-11 14:55:31 +0200 (Wed, 11 Jun 2014) | 1 line

Replaced isnan mask with function that catches all invalid values.
................
r23836 | tlinnet | 2014-06-11 14:55:32 +0200 (Wed, 11 Jun 2014) | 3 lines

Clean up of comment lines.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23837 | tlinnet | 2014-06-11 14:55:34 +0200 (Wed, 11 Jun 2014) | 8 lines

Removed the masked replacement if fact is less than 1.0.

This is very strange, but otherwise system test:
Relax_disp.test_hansen_cpmg_data_missing_auto_analysis

would fail.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23838 | tlinnet | 2014-06-11 15:09:01 +0200 (Wed, 11 Jun 2014) | 5 lines

Removed the slow allclose() function to test if R20A and R20B is equal.

It is MUCH faster to just subtract and check sum is not 0.0.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23839 | tlinnet | 2014-06-11 15:25:32 +0200 (Wed, 11 Jun 2014) | 3 lines

Replaced the temporary variable R2eff with back_calc, and used numpy subtract to speed up.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23840 | tlinnet | 2014-06-11 15:25:34 +0200 (Wed, 11 Jun 2014) | 5 lines

Made the lib function into a pure numpy array calculation.

This requires, that r20a, r20b and dw has same dimension as the dispersion points.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23841 | tlinnet | 2014-06-11 15:25:36 +0200 (Wed, 11 Jun 2014) | 3 lines

Changes too unit tests, so data is sent to target function in numpy array format.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23842 | tlinnet | 2014-06-11 16:19:10 +0200 (Wed, 11 Jun 2014) | 3 lines

Removed an the creation of a unnessary structure bu using numpy multiply.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23843 | tlinnet | 2014-06-11 16:19:12 +0200 (Wed, 11 Jun 2014) | 3 lines

Moved the mask which finds where to replace values into the __init__ function.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23844 | tlinnet | 2014-06-11 16:43:17 +0200 (Wed, 11 Jun 2014) | 7 lines

Replaced the multiplcation with the disp structure, to a replacement with a mask.

My timings are varying at my computer, so results are within the very same timing.

But for reading the code, having two masks, is essential better.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23845 | tlinnet | 2014-06-11 17:09:23 +0200 (Wed, 11 Jun 2014) | 14 lines

Merged revisions 23743 via svnmerge from
svn+ssh://tlinnet@.../svn/relax/trunk

........
r23743 | tlinnet | 2014-06-08 22:42:55 +0200 (Sun, 08 Jun 2014) | 6 lines

Updated profiling text for CR72 model.

Now it is tested for 3 fields.

This is related to:
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
................
r23846 | tlinnet | 2014-06-11 17:14:09 +0200 (Wed, 11 Jun 2014) | 25 lines

Reverted r23845-r23844 as this was using a wrong commit message.

The command used was:
svn merge -r23845:23844 .

........
r23845 | tlinnet | 2014-06-11 17:09:23 +0200 (Wed, 11 Jun 2014) | 14 lines

Merged revisions 23743 via svnmerge from
svn+ssh://tlinnet@.../svn/relax/trunk

........
r23743 | tlinnet | 2014-06-08 22:42:55 +0200 (Sun, 08 Jun 2014) | 6 lines

Updated profiling text for CR72 model.

Now it is tested for 3 fields.

This is related to:
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
........
................
r23847 | tlinnet | 2014-06-11 17:15:03 +0200 (Wed, 11 Jun 2014) | 5 lines

Copied profiling script for CR72 to B14 model.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23848 | tlinnet | 2014-06-11 19:37:37 +0200 (Wed, 11 Jun 2014) | 3 lines

Modified profiling script for the B14 model.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23849 | tlinnet | 2014-06-11 19:37:40 +0200 (Wed, 11 Jun 2014) | 5 lines

Modified model B14 lib file to faster numpy multidimensional mode.

The implementations comes almost directly from the CR72 model file.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23850 | tlinnet | 2014-06-11 19:37:42 +0200 (Wed, 11 Jun 2014) | 5 lines

Reverted the use of the mask "mask_set_blank".

It did not work, and many system tests started failing.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23851 | tlinnet | 2014-06-11 19:37:43 +0200 (Wed, 11 Jun 2014) | 3 lines

Changed the target function to handle the B14 model for faster numpy computation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23852 | tlinnet | 2014-06-11 19:37:45 +0200 (Wed, 11 Jun 2014) | 3 lines

Changed unit test for B14 to match numpy input requirement.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23853 | tlinnet | 2014-06-11 20:43:45 +0200 (Wed, 11 Jun 2014) | 5 lines

Added additional tests in b14, when math errors can occur.

This is very easy with a conditional masked search in arrays.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23854 | tlinnet | 2014-06-11 20:43:47 +0200 (Wed, 11 Jun 2014) | 3 lines

Comment fix for finding when E0 is above 700 in lib function of B14.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................
r23855 | tlinnet | 2014-06-11 20:47:29 +0200 (Wed, 11 Jun 2014) | 3 lines

Removed use of "asarray", since the variables are already arrays.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
................

Merged revisions 23745-23752,23754-23758 via svnmerge from
svn+ssh://bugman@.../svn/relax/branches/disp_spin_speed

........
r23745 | tlinnet | 2014-06-08 23:44:44 +0200 (Sun, 08 Jun 2014) | 8 lines

Swith the looping from spin->frq to frq->spin.

Since the number of dispersion points are the same for all spins, this
allows to move the calculation of pA and kex array one level up.

This saves alot of computation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23746 | tlinnet | 2014-06-08 23:44:45 +0200 (Sun, 08 Jun 2014) | 3 lines

Changed all the creation of special numpy arrays to be of float64 type.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23747 | tlinnet | 2014-06-08 23:54:41 +0200 (Sun, 08 Jun 2014) | 5 lines

Moved the data filling of special numpy array errors and values, to initialization of Dispersion class.

These values does not change, and can safely be stored outside.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23748 | tlinnet | 2014-06-08 23:56:36 +0200 (Sun, 08 Jun 2014) | 3 lines

Just a tiny little more speed, by removing temporary storage of chi2 calculation.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23749 | tlinnet | 2014-06-09 00:09:59 +0200 (Mon, 09 Jun 2014) | 25 lines

Changed all calls to numpy np.X functions to just the numpy function.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.

Timing is now showing, 17% loss per single spin, but but 277 % gain on 100 spin.
3 fields, 1000 iterations.
1 spin
1 0.000 0.000 0.677 0.677 <string>:1(<module>)
1 0.001 0.001 0.677 0.677 pf:419(single)
1000 0.002 0.000 0.671 0.001 pf:405(calc)
1000 0.009 0.000 0.669 0.001 relax_disp.py:979(func_CR72_full)
1000 0.102 0.000 0.655 0.001 relax_disp.py:507(calc_CR72_chi2)
1003 0.160 0.000 0.365 0.000 cr72.py:101(r2eff_CR72)
23029 0.188 0.000 0.188 0.000 {numpy.core.multiarray.array}
4003 0.119 0.000 0.182 0.000 numeric.py:1862(allclose)

100 spin
1 0.000 0.000 19.783 19.783 <string>:1(<module>)
1 0.002 0.002 19.783 19.783 pf:441(cluster)
1000 0.004 0.000 19.665 0.020 pf:405(calc)
1000 0.013 0.000 19.661 0.020 relax_disp.py:979(func_CR72_full)
1000 6.541 0.007 19.634 0.020 relax_disp.py:507(calc_CR72_chi2)
916108 11.127 0.000 11.127 0.000 {numpy.core.multiarray.array}
1300 1.325 0.001 2.026 0.002 cr72.py:101(r2eff_CR72)
4300 0.495 0.000 0.634 0.000 numeric.py:1862(allclose)
........
r23750 | tlinnet | 2014-06-09 00:49:14 +0200 (Mon, 09 Jun 2014) | 3 lines

Removed unused import of numpy.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23751 | tlinnet | 2014-06-09 00:49:15 +0200 (Mon, 09 Jun 2014) | 3 lines

Changed all calls to numpy np.X functions to just the numpy function in lib/dispersion/cr72.py.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23752 | tlinnet | 2014-06-09 00:49:18 +0200 (Mon, 09 Jun 2014) | 3 lines

Removed unused import of numpy.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23754 | tlinnet | 2014-06-09 19:46:17 +0200 (Mon, 09 Jun 2014) | 3 lines

Made copies of numpy arrays instead of creating from new.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23755 | tlinnet | 2014-06-09 19:46:19 +0200 (Mon, 09 Jun 2014) | 3 lines

Added a self.frqs_a as a multidimensional numpy array.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23756 | tlinnet | 2014-06-09 19:46:20 +0200 (Mon, 09 Jun 2014) | 3 lines

Small fix for the indicies to the errors and values numpy array.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23757 | tlinnet | 2014-06-09 19:46:22 +0200 (Mon, 09 Jun 2014) | 5 lines

Lowered the number of iterations to the profiling scripts.

This is to use the profiling script as bug finder.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23758 | tlinnet | 2014-06-09 19:46:25 +0200 (Mon, 09 Jun 2014) | 7 lines

Moved the calculation of dw_frq out of spin and spectrometer loop.

This is done by having a special 1/0 spin numpy array, which turns on or off the values in the numpy array multiplication.

The multiplication needs to first axis expand dw, and then tile the arrays according to the numpy structure.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........

Merged revisions 23722-23742 via svnmerge from
svn+ssh://bugman@.../svn/relax/branches/disp_spin_speed

........
r23722 | tlinnet | 2014-06-07 21:43:16 +0200 (Sat, 07 Jun 2014) | 6 lines

Modified profiling script to have different number of NCYC points per frequency.

This is to complicate the data, so any errournous reshaping of data is discovered.
It is expected, that experiments can have different number of NCYC points per spectrometer frequency.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23723 | tlinnet | 2014-06-07 21:43:19 +0200 (Sat, 07 Jun 2014) | 55 lines

Initial try to alter the target function calc_CR72_chi2.

This is the first test to restructure the arrays, to allow for higher dimensional computation.
All numpy arrays have to have same shape to allow to multiply together.
The dimensions should be [ei][si][mi][oi][di]. [Experiment][spins][spec. frq][offset][disp points].
This is complicated with number of disp point can change per spectrometer frequency.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.

This implementation brings a high overhead.
The first test shows no winning of time.
The creation of arrays takes all the time.

Checked on MacBook Pro
2.4 GHz Intel Core i5
8 GB 1067 Mhz DDR3 RAM.
Python Distribution -- Python 2.7.3 |EPD 7.3-2 (32-bit)|

Timing for:
3 fields, [600. * 1E6, 800. * 1E6, 900. * 1E6]
('sfrq: ', 600000000.0, 'number of cpmg frq', 15)
('sfrq: ', 800000000.0, 'number of cpmg frq', 20)
('sfrq: ', 900000000.0, 'number of cpmg frq', 22)
iterations of function call: 1000

Timed for simulating 1 or 100 clustered spins.

For TRUNK

1 spin:
ncalls tottime percall cumtime percall filename:lineno(function)
3000 0.267 0.000 0.313 0.000 cr72.py:100(r2eff_CR72)
1000 0.056 0.000 0.434 0.000 relax_disp.py:456(calc_CR72_chi2)
3000 0.045 0.000 0.061 0.000 chi2.py:32(chi2)

100 spins:
ncalls tottime percall cumtime percall filename:lineno(function)
300000 26.315 0.000 30.771 0.000 cr72.py:100(r2eff_CR72)
1000 5.498 0.005 42.660 0.043 relax_disp.py:456(calc_CR72_chi2)
300000 4.438 0.000 6.021 0.000 chi2.py:32(chi2)

TESTING

1 spin:
ncalls tottime percall cumtime percall filename:lineno(function)
19013 0.278 0.000 0.278 0.000 {numpy.core.multiarray.array}
1000 0.191 0.000 0.777 0.001 relax_disp.py:457(calc_CR72_chi2)
1000 0.147 0.000 0.197 0.000 cr72.py:101(r2eff_CR72)
3000 0.044 0.000 0.061 0.000 chi2.py:32(chi2)

100 spins:
ncalls tottime percall cumtime percall filename:lineno(function)
1504904 25.215 0.000 25.215 0.000 {numpy.core.multiarray.array}
1000 17.261 0.017 51.180 0.051 relax_disp.py:457(calc_CR72_chi2)
300000 4.637 0.000 6.310 0.000 chi2.py:32(chi2)
........
r23724 | tlinnet | 2014-06-07 21:43:21 +0200 (Sat, 07 Jun 2014) | 4 lines

Temporary changed the lib/dispersion/cr72.py function to unsafe state.

This change turns-off all the safety measures, since they have to be re-implemented
for higher dimensional structures.
........
r23725 | tlinnet | 2014-06-07 22:36:00 +0200 (Sat, 07 Jun 2014) | 71 lines

Altered profiling script to report cumulative timings and save to temporary files.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.

This indeed shows that the efficiency has gone down:

-----

Checked on MacBook Pro
2.4 GHz Intel Core i5
8 GB 1067 Mhz DDR3 RAM.
Python Distribution -- Python 2.7.3 |EPD 7.3-2 (32-bit)|

Timing for:
3 fields, [600. * 1E6, 800. * 1E6, 900. * 1E6]
('sfrq: ', 600000000.0, 'number of cpmg frq', 15)
('sfrq: ', 800000000.0, 'number of cpmg frq', 20)
('sfrq: ', 900000000.0, 'number of cpmg frq', 22)
iterations of function call: 1000

Timed for simulating 1 or 100 clustered spins.

For TRUNK

1 spin:
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.434 0.434 <string>:1(<module>)
1 0.001 0.001 0.434 0.434 pf:354(single)
1000 0.002 0.000 0.432 0.000 pf:340(calc)
1000 0.007 0.000 0.429 0.000 relax_disp.py:908(func_CR72_full)
1000 0.054 0.000 0.416 0.000 relax_disp.py:456(calc_CR72_chi2)
3000 0.256 0.000 0.300 0.000 cr72.py:100(r2eff_CR

100 spins:
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 42.035 42.035 <string>:1(<module>)
1 0.002 0.002 42.035 42.035 pf:378(cluster)
1000 0.004 0.000 41.957 0.042 pf:340(calc)
1000 0.011 0.000 41.953 0.042 relax_disp.py:908(func_CR72_full)
1000 5.378 0.005 41.928 0.042 relax_disp.py:456(calc_CR72_chi2)
300000 25.942 0.000 30.276 0.000 cr72.py:100(r2eff_CR72)
300000 4.362 0.000 5.903 0.000 chi2.py:32(chi2)

TESTING

1 spin:
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.792 0.792 <string>:1(<module>)
1 0.001 0.001 0.792 0.792 pf:354(single)
1000 0.002 0.000 0.789 0.001 pf:340(calc)
1000 0.011 0.000 0.787 0.001 relax_disp.py:966(func_CR72_full)
1000 0.187 0.000 0.769 0.001 relax_disp.py:457(calc_CR72_chi2)
26013 0.290 0.000 0.290 0.000 {numpy.core.multiarray.array}
1000 0.147 0.000 0.198 0.000 cr72.py:101(r2eff_CR72)
2001 0.002 0.000 0.086 0.000 numeric.py:167(asarray)
3000 0.045 0.000 0.062 0.000 chi2.py:32(chi2)

100 spins:
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 46.821 46.821 <string>:1(<module>)
1 0.002 0.002 46.821 46.821 pf:378(cluster)
1000 0.004 0.000 46.744 0.047 pf:340(calc)
1000 0.017 0.000 46.740 0.047 relax_disp.py:966(func_CR72_full)
1000 15.422 0.015 46.708 0.047 relax_disp.py:457(calc_CR72_chi2)
1511904 23.475 0.000 23.475 0.000 {numpy.core.multiarray.array}
300000 4.175 0.000 5.668 0.000 chi2.py:32(chi2)
2001 0.004 0.000 4.804 0.002 numeric.py:167(asarray)
1000 0.010 0.000 2.438 0.002 fromnumeric.py:1774(amax)
1000 0.006 0.000 2.428 0.002 fromnumeric.py:32(_wrapit)
300000 0.353 0.000 1.493 0.000 fromnumeric.py:1379(sum)
1000 1.281 0.001 1.444 0.001 cr72.py:101(r2eff_CR72)
........
r23726 | tlinnet | 2014-06-07 23:18:15 +0200 (Sat, 07 Jun 2014) | 3 lines

Added print out of chi2 to profile script.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23727 | tlinnet | 2014-06-07 23:18:18 +0200 (Sat, 07 Jun 2014) | 3 lines

Moved the creation of special numpy structures outside target function.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23728 | tlinnet | 2014-06-08 13:14:34 +0200 (Sun, 08 Jun 2014) | 5 lines

Modified profiling script to calculate correct values when setting up R2eff values.

This is to test, that the return of chi2 gets zero.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23729 | tlinnet | 2014-06-08 13:14:36 +0200 (Sun, 08 Jun 2014) | 31 lines

Removing looping over exp and offset indicies in calc_chi2. They are always 0 anyway.

This brings a little speed.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.

----
Profiling.

1000 iterations
100 spins
3 sfrq
('sfrq: ', 600000000.0, 'number of cpmg frq', 15, array([ 2., 6., 10., 14., 18., 22., 26., 30., 34., 38., 42.,
46., 50., 54., 58.]))
('sfrq: ', 800000000.0, 'number of cpmg frq', 20, array([ 2., 6., 10., 14., 18., 22., 26., 30., 34., 38., 42.,
46., 50., 54., 58., 62., 66., 70., 74., 78.]))
('sfrq: ', 900000000.0, 'number of cpmg frq', 22, array([ 2., 6., 10., 14., 18., 22., 26., 30., 34., 38., 42.,
46., 50., 54., 58., 62., 66., 70., 74., 78., 82., 86.]))
('chi2 cluster:', 0.0)

TRUNK
ncalls tottime percall cumtime percall filename:lineno(function)
1000 5.221 0.005 53.578 0.054 relax_disp.py:456(calc_CR72_chi2)

BEFORE
ncalls tottime percall cumtime percall filename:lineno(function)
1000 14.871 0.015 43.084 0.043 relax_disp.py:494(calc_CR72_chi2)

AFTER removing looping over exp and offset indicies. They are always 0.
ncalls tottime percall cumtime percall filename:lineno(function)
1000 12.831 0.013 38.248 0.038 relax_disp.py:494(calc_CR72_chi2)
........
r23730 | tlinnet | 2014-06-08 13:14:38 +0200 (Sun, 08 Jun 2014) | 5 lines

In profiling script, moved up the calculation of values one level.

This is to better see the output of the profiling iterations for cr72.py.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23731 | tlinnet | 2014-06-08 19:48:23 +0200 (Sun, 08 Jun 2014) | 6 lines

Fix for calculation of the larmor frequency per spin in profiling script.

The frq loop should also be up-shifted.
It was now extracted as 0.0.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23732 | tlinnet | 2014-06-08 19:48:25 +0200 (Sun, 08 Jun 2014) | 7 lines

Re-inserted safety checks in lin/dispersion/cr72.py file.

This is re-inserted for the rank_1 cases.

This makes the unit-tests pass again.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23733 | tlinnet | 2014-06-08 19:48:27 +0200 (Sun, 08 Jun 2014) | 8 lines

Important fix for extracting the correct shape to create new arrays.

If using just one field, or having the same number of dispersion points, the shape would extend to the dispersion number.
It would report [ei][si][mi][oi][di] when calling ndarray.shape.

Shape always has to be reported as: [ei][si][mi][oi].

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23734 | tlinnet | 2014-06-08 19:48:29 +0200 (Sun, 08 Jun 2014) | 3 lines

Made it easier to switch between single and cluster reporting in profiling script.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23735 | tlinnet | 2014-06-08 19:48:31 +0200 (Sun, 08 Jun 2014) | 9 lines

Important fix for the creation of the multi dimensional pA numpy array.

It should be created as numpy.zeros([ei][si][mi][oi]) instead of numpy.ones([ei][si][mi][oi]).

This allows for rapid testing of all dimensions with np.allclose(pA, numpy.ones(dw.shape)).
pA can have missing filled out values, when the number of dispersion points are different
per spectrometer frequency.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23736 | tlinnet | 2014-06-08 19:48:33 +0200 (Sun, 08 Jun 2014) | 18 lines

Added unit tests demonstrating edge cases 'no Rex' failures of the model 'CR72 full', for a clustered multi dimensional calculation.

This is implemented for one field.

This is to implement catching of math domain errors, before they occur.

These tests cover all parameter value combinations which result in no exchange:

- dw = 0.0,
- pA = 1.0,
- kex = 0.0,
- dw = 0.0 and pA = 1.0,
- dw = 0.0 and kex = 0.0,
- pA = 1.0 and kex = 0.0,
- dw = 0.0, pA = 1.0, and kex = 0.0.
- kex = 1e5,

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23737 | tlinnet | 2014-06-08 19:48:35 +0200 (Sun, 08 Jun 2014) | 7 lines

Re-implemented safety checks in lib/dispersion/cr72.py.

This is now implemented for both rank-1 float array and of higher dimensions.

This makes the unit tests pass for multi dimensional computing.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23738 | tlinnet | 2014-06-08 19:48:37 +0200 (Sun, 08 Jun 2014) | 17 lines

Added unit tests demonstrating edge cases 'no Rex' failures of the model 'CR72 full', for a clustered multi dimensional calculation.

This is implemented for three fields.

This is to implement catching of math domain errors, before they occur.

These tests cover all parameter value combinations which result in no exchange:

- dw = 0.0,
- pA = 1.0,
- kex = 0.0,
- dw = 0.0 and pA = 1.0,
- dw = 0.0 and kex = 0.0,
- pA = 1.0 and kex = 0.0,
- dw = 0.0, pA = 1.0, and kex = 0.0.
- kex = 1e5,
Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23739 | tlinnet | 2014-06-08 19:48:38 +0200 (Sun, 08 Jun 2014) | 5 lines

Changed that special numpy structure is also created for "CR72".

This makes most system tests pass.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23740 | tlinnet | 2014-06-08 22:22:22 +0200 (Sun, 08 Jun 2014) | 5 lines

Critical fix for the slicing of values in target function.

This makes system test: Relax_disp.test_sod1wt_t25_to_cr72 pass.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23741 | tlinnet | 2014-06-08 22:22:24 +0200 (Sun, 08 Jun 2014) | 6 lines

Added self.has_missing keyword in initialization of the Dispersion class.

This is to test once, per spin or cluster.
This saves a looping over the dipsersion points, when collection the data.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........
r23742 | tlinnet | 2014-06-08 22:22:26 +0200 (Sun, 08 Jun 2014) | 6 lines

Created multi dimensional error and value numpy arrays.

This is to calculate the chi2 sum much faster.
Reordered the loop over missing data points, so it is only iniatiated if missing points is detected.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.
........

Initialized merge tracking via "svnmerge" with revisions "1-23718" from
svn+ssh://bugman@.../svn/relax/branches/disp_spin_speed

Merged revisions 24638 via svnmerge from
svn+ssh://bugman@.../svn/relax/trunk

........
r24638 | bugman | 2014-07-22 17:06:47 +0200 (Tue, 22 Jul 2014) | 6 lines

Modified the align_tensor.init user function so that the parameters are now optional.

This allows alignment tensors to be initialised without specifying the parameter values for that
tensor.
........

Comment space fixing in target_functions/relax_disp.py.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.

Converted the N-state model analysis to the new grid bounds and scaling factor design.

The parameter object now registers the grid bounds and scaling factors for all of the N-state model
parameters.

The specific_analyses.n_state_model.parameters.assemble_scaling_matrix() function has been deleted
as this is now provided by the upstream code in pipe_control.minimise. And the API methods
grid_search() and minimise() has been modified to accept the list of scaling matrices. In addition,
all of the lower bounds defined in the grid_search() API method have been deleted as this is now in
the parameter object.

The new API function print_model_title() has been aliased from _print_model_title_global(). And the
get_param_names() and get_param_values() API methods have been implemented.

API documentation fixes, where a "\" is the last character on the line. There should be a space " ", ending this characater.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.

Removed unnecessary repetitive calculation of kex**2 in model DPL94.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.

Documentation fix for the get_back_calc() function in target_function/relax_disp.py.

Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion models for Clustered analysis.