relax SVN archive

svn+ssh://bugman@.../svn/relax/branches/frame_order_cleanup

........
r23241 | bugman | 2014-05-19 15:10:15 +0200 (Mon, 19 May 2014) | 5 lines

Increased the precision of all the data in the CaM frame order test data generation base script.

These have all been converted from float16 to float64 numpy types.
........
r23242 | bugman | 2014-05-19 16:13:24 +0200 (Mon, 19 May 2014) | 5 lines

Fix for the RDC error setting in the CaM frame order test data generation base script.

The rdc_err data structure is located in the interatomic data containers, no the spin containers.
........
r23243 | bugman | 2014-05-19 16:54:19 +0200 (Mon, 19 May 2014) | 4 lines

Modification of the structure loading part of the CaM frame order data generation base script.

The structures are now only loaded if the DIST_PDB flag is set, as they are only used for generating
the 3D distribution of structures. This saves a lot of time and computer memory.
........
r23244 | bugman | 2014-05-19 18:11:12 +0200 (Mon, 19 May 2014) | 9 lines

Huge speed up of the CaM frame order test data generation base script.

By using multidimensional numpy arrays to store the atomic positions and XH unit vectors of all
spins, and performing the rotations on these structures using numpy.tensordot(), the calculations
are now a factor of 10 times faster. The progress meter had to be changed to show every 1000 rather
than 100 iterations.

The rotations of the positions and vectors are now performed sequentially, accidentally fixing a bug
with the double motion models (i.e. the 'double rotor' model).
........
r23245 | bugman | 2014-05-19 20:10:33 +0200 (Mon, 19 May 2014) | 5 lines

Modified the CaM frame order test data generation base script to conserve computer RAM.

The XH vector and atomic position data structures for all N rotations are now of the numpy.float32
rather than numpy.float64 type. The main change is to calculate the averaged RDCs and averaged PCSs
separately, deleting the N-sized data structures once the data files are written.
........
r23251 | bugman | 2014-05-20 11:09:21 +0200 (Tue, 20 May 2014) | 21 lines

Complete redesign of the CaM frame order data generation base script for speed and memory savings.

Although the rotated XH bond vector and atomic position code was very fast, the amount of memory
needed to store these in the spin containers and interatomic data containers was huge when N > 1e6.
The subsequent rdc.back_calc and pcs.back_calc user function calls would also take far too long.

Therefore the base script has been redesigned. The _create_distribution() method has been split
into four: _calculate_pcs(), _calculate_rdc(), _create_distribution(), and _pipe_setup().
The _pipe_setup() method is called first to set up the data pipe with all required data. Then the
_calculate_rdc() and _calculate_pcs() methods, and finally _create_distribution() if the DIST_PDB
flag is set.

The calls to the rdc.back_calc and pcs.back_calc user functions have been eliminated. Instead the
_calculate_rdc() and _calculate_pcs() methods calculate the averaged RDC and PCS themselves as numpy
array structures. Rather than storing the huge rotated vectors and atomic positions data
structures, the RDCs and PCSs are summed. These are then divided by self.N at the end to average
the values.

Compared to the old code, when N is set to 20 million the RAM usage drops from ~20 GB to ~65 MB.
The total run time is also decreased on one system from a few days to a few hours (an order or two
of magnitude).
........
r23252 | bugman | 2014-05-20 11:48:11 +0200 (Tue, 20 May 2014) | 7 lines

Large increase in accuracy of the RDC and PCS averaging.

This is for the CaM frame order test data generation base script. By storing the RDCs and PCSs in
numpy.float128 arrays, a 64-bit system is required, and then using numpy.average(), the average
value can be calculated with a much higher accuracy. As N becomes larger, the averaging introduces
greater and greater amounts of truncation artifacts. So this change alleviates this. However a lot
more RAM is now required.
........
r23253 | bugman | 2014-05-20 11:48:13 +0200 (Tue, 20 May 2014) | 4 lines

Changed the progress meter updating for the CaM frame order test data generation base script.

The spinner was far too fast, updating every 5 increments, and is now updated every 250. And the
total number is now only printed every 10,000 increments.
........
r23256 | bugman | 2014-05-20 13:47:33 +0200 (Tue, 20 May 2014) | 20 lines

Reverted r23252 as this uses far, far too much RAM, and there is a better solution.

The command used was:
svn merge -r23252:r23251 .

.....
r23252 | bugman | 2014-05-20 11:48:11 +0200 (Tue, 20 May 2014) | 7 lines
Changed paths:
M /branches/frame_order_cleanup/test_suite/shared_data/frame_order/cam/generate_base.py

Large increase in accuracy of the RDC and PCS averaging.

This is for the CaM frame order test data generation base script. By storing the RDCs and PCSs in
numpy.float128 arrays, a 64-bit system is required, and then using numpy.average(), the average
value can be calculated with a much higher accuracy. As N becomes larger, the averaging introduces
greater and greater amounts of truncation artifacts. So this change alleviates this. However a lot
more RAM is now required.
.....
........
r23257 | bugman | 2014-05-20 13:47:59 +0200 (Tue, 20 May 2014) | 3 lines

Improvements to the progress meter for the CaM frame order test data generation base script.

Commas are now printed between the thousands and the numbers are now right justified.
........
r23258 | bugman | 2014-05-20 13:57:31 +0200 (Tue, 20 May 2014) | 9 lines

Large increase in accuracy of the RDC and PCS averaging.

This is for the CaM frame order test data generation base script. By summing the RDCs and PCSs into
1D numpy.float128 arrays (for this, a 64-bit system is required), and then dividing by N at the end,
the average value can be calculated with a much higher accuracy. As N becomes larger, the
numerical averaging introduces greater and greater amounts of truncation artifacts. So this change
alleviates this.

This change is what r23252 should have been.
........
r23280 | bugman | 2014-05-21 09:24:28 +0200 (Wed, 21 May 2014) | 5 lines

Fix for the RDC and PCS averaging in the CaM frame order test data generation base script.

For the double rotor model, or any multiple motional mode model, the averaging was incorrect.
Instead of dividing by N, the values should be divided by N**M, where M is the number of motional
modes.
........
r23281 | bugman | 2014-05-21 09:24:31 +0200 (Wed, 21 May 2014) | 9 lines

Huge increase in precision for the CaM frame order free rotor model test data.

The higher precision is because the number structures in the distribution is now twenty million
rather than one million, and the much higher precision numpy.float128 averaging of the updated data
generation base script has been used.

This data should allow for a much better estimate of the beta and gamma average domain position
parameter values for the free rotor models which are affected by the collapse of the alpha parameter
to zero.
........
r23282 | bugman | 2014-05-21 09:24:34 +0200 (Wed, 21 May 2014) | 5 lines

Huge increase in precision for the CaM frame order double rotor model test data.

The higher precision is because the number structures in the distribution is now over twenty million
(4500**2) rather than a quarter of a million (500**2). And the much higher precision numpy.float128
averaging of the updated data generation base script has been used.
........
r23283 | bugman | 2014-05-21 09:59:51 +0200 (Wed, 21 May 2014) | 3 lines

Fix for the constraint deactivation in the frame order minimisation when no constraints are present.

This problem was introduced at r23211.
........
r23309 | bugman | 2014-05-21 17:06:17 +0200 (Wed, 21 May 2014) | 4 lines

Deletion of the frame_order.quad_int user function.

This no longer has a purpose, as the incredibly slow quadratic integration code has been removed
from relax.
........
r23321 | bugman | 2014-05-22 08:58:54 +0200 (Thu, 22 May 2014) | 4 lines

Huge increase in precision for the CaM frame order rotor model test data.

The higher precision is because the number structures in the distribution is now 20 million rather
than 166,666, and the numpy.float128 data averaging has been used.
........
r23322 | bugman | 2014-05-22 08:59:00 +0200 (Thu, 22 May 2014) | 4 lines

Large increase in precision for the 2nd CaM frame order rotor model test data set.

The higher precision is because the number structures in the distribution is now 20 million rather
than 1,000,001 and the numpy.float128 data averaging has been used.
........