https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/Recent changes to Data Processing OutlineenSat, 19 Oct 2013 20:31:05 -0000https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v32 +++ v33 @@ -123,7 +123,7 @@  - + ##### When this is done, click 'Calculate'; ##### </pre> </div>Anna MakalSat, 19 Oct 2013 20:31:05 -0000https://sourceforge.net242cf280be3c2329d27cc5205ef21f5b0bcf833dhttps://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v31 +++ v32 @@ -121,9 +121,9 @@ ##### Expt Clusters Manager always has to be set so as the cluster number in console was in between 30 and 40 ##### - - - + + + ##### When this is done, click 'Calculate'; ##### </pre> </div>Anna MakalSat, 19 Oct 2013 20:26:49 -0000https://sourceforge.net8499f313e2fb639535aeadd93ed035ff9f0434d4https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v30 +++ v31 @@ -115,14 +115,15 @@ ##### ... or in 'long' dataset (>20 phi angles) ##### - +  ##### Expt Clusters Manager always has to be set so as the cluster number in console was in between 30 and 40 ##### - + + ##### When this is done, click 'Calculate'; ##### </pre> </div>Anna MakalSat, 19 Oct 2013 20:25:16 -0000https://sourceforge.net371fd6f6b1fd91b117ea5f38a31105f6f98c4999https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v29 +++ v30 @@ -120,9 +120,9 @@ ##### Expt Clusters Manager always has to be set so as the cluster number in console was in between 30 and 40 ##### - -##### 6. orientation matrix refinement ##### +##### 7. orientation matrix refinement ##### $ lu__refine_orientmatrix __expt.h5 ../../2012-02-mono/__K.h5 ../../14IDB_10892.mda.0001 __initial.h5 _output:_ `__expt_rot.h5` @@ -142,12 +153,12 @@  -##### 6a. cell parameters / Euler angle control ##### +##### 7a. cell parameters / Euler angle control ##### $ lu__view_matrix_parameters __expt_rot.h5 $ lu__view_matrix_parameters __expt_full.h5 -##### 7. hkl indices assignment ##### +##### 8. hkl indices assignment ##### $ lu__hkl_assign_monolc __expt.h5 ../../2012-02-mono/__K.h5 ../../14IDB_10892.mda.0001 __expt_rot.h5 __expt_full.h5> <-P delay> @@ -168,7 +179,7 @@  -##### 8. SORTAV processing - in **ratio** subdirectory ##### +##### 9. SORTAV processing - in **ratio** subdirectory ##### __just ratios for SORTAV__ @@ -182,7 +193,7 @@ $ bash ../../processing_scripts/run_sortav.sh __assign_FU 9 0.5 -##### 9. correlation plot visualization ##### +##### 10. correlation plot visualization ##### __If hkl unmerged,__ $ lu__view_correlation_merged_ratios_phi /__ratios.h5 /__ratios.h5 @@ -191,7 +202,7 @@ $ lu__view_correlation_merged_ratios /__sortav.hkl /__sortav.hkl -##### 10. Optional 'sensitive' hkl retrieval: ##### +##### 11. Optional 'sensitive' hkl retrieval: ##### $ $PYTHON /space-md2/lab/_WORK/python-bin/hkl-h5.py __ratios.h5 ~/Desktop/RhPNP_significant_positive_experimental_eta.txt '(h,k,l), (-h,-k,-l), (h, -k, l), (-h, k, -l)' _output:_ `selected_hkl.lst` </pre> </div>Anna MakalSat, 19 Oct 2013 18:30:33 -0000https://sourceforge.net735bb411164a25497cb1bcb0e782c059de9c8122https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v26 +++ v27 @@ -25,7 +25,7 @@ ##### 0. in specified directory, for each compound convert reference monochromatic data to hdf5 format: ##### - $ lu_fcf2mono -o __XXXK.h5 -e '<(h,k,l),(-h,-k,-l),...>' /.fcf + $ lu_fcf2mono -o __XXXK.h5 -e '[(h,k,l),(-h,-k,-l),...]' /.fcf ##### 1. in new terminal switch to bash: ##### </pre> </div>Anna MakalWed, 12 Jun 2013 18:46:30 -0000https://sourceforge.net1c10be7eaec4ae060f99803d0ef07f1d69e7fdd3https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v25 +++ v26 @@ -1,17 +1,17 @@ ### directory structure ### -`[compound]` is a name of investigated compound -`[dataset]` is a name of directory for a single crystal specimen - it should be the same as the name of directory containing raw frames -`[scan]` is a name of a certain scan for a crystal, like 'laser01' or 'powerscan01' -`[path]` is a path to a given file +`` is a name of investigated compound +`` is a name of directory for a single crystal specimen - it should be the same as the name of directory containing raw frames +`` is a name of a certain scan for a crystal, like 'laser01' or 'powerscan01' +`` is a path to a given file _HDF5 files, i.e. processed files_ - ...../WORK/APS-YYYY-MM/[dataset]/h5 + ...../WORK/APS-YYYY-MM//h5 _SORTAV and text files_ - ...../WORK/APS-YYYY-MM/[dataset]/ratio + ...../WORK/APS-YYYY-MM//ratio _monochromatic data in HDF5 format_ @@ -25,7 +25,7 @@ ##### 0. in specified directory, for each compound convert reference monochromatic data to hdf5 format: ##### - $ lu_fcf2mono -o [scan]__XXXK.h5 -e '[(h,k,l),(-h,-k,-l),...]' [path]/[compound].fcf + $ lu_fcf2mono -o __XXXK.h5 -e '<(h,k,l),(-h,-k,-l),...>' /.fcf ##### 1. in new terminal switch to bash: ##### @@ -35,11 +35,11 @@ $ source laue_util.sh -##### 3. each time a new crystal is mounted, create a new `[dataset]` directory:##### +##### 3. each time a new crystal is mounted, create a new `` directory:##### $ cd WORK/APS-2012-06/ - $ mkdir [dataset] - $ cd [dataset] + $ mkdir + $ cd $ mkdir h5 $ cd h5 @@ -47,55 +47,55 @@ __mccd2hdf__ - $ lu_mccd2hdf -o [scan]__mccd.h5 -p /data/coppens_1202/[dataset]/ /data/coppens_1202/[dataset]/[scan].log - - _output:_ `[scan]__mccd.h5` + $ lu_mccd2hdf -o __mccd.h5 -p /data/coppens_1202// /data/coppens_1202//.log + + _output:_ `__mccd.h5` __hdf2spots__ __a) dark__ - $ lu_hdf2spots -o [scan]__spots.h5 [scan]__mccd.h5 + $ lu_hdf2spots -o __spots.h5 __mccd.h5 - _output:_ `[scan]__spots.h5` + _output:_ `__spots.h5` __b) laser__ - $ lu_hdf2spots -o [scan]__spots.h5 [scan]__mccd.h5 -m kruskal -S blocks - _output:_ `[scan]__spots.h5` + $ lu_hdf2spots -o __spots.h5 __mccd.h5 -m kruskal -S blocks + _output:_ `__spots.h5` __spots2expt__ __a) default__ - $ lu_spots2expt -o [scan]__expt.h5 -c [beam center] -d 65.0 [scan]__spots.h5 - _output:_ `[scan]__expt.h5` + $ lu_spots2expt -o __expt.h5 -c -d 65.0 __spots.h5 + _output:_ `__expt.h5` __b) with refinement of detector distance & beam-center__ - $ lu_spots2expt -o [scan]__expt.h5 -c [beam center] -d 65.0 -r yes [scan]__spots.h5 - _output:_ `[scan]__expt.h5` + $ lu_spots2expt -o __expt.h5 -c -d 65.0 -r yes __spots.h5 + _output:_ `__expt.h5` __c) 'laser' dataset__ - $ lu_spots2expt -o [scan]__expt.h5 -c [beam center] -d 65.0 -r yes -l yes [scan]__spots.h5 - _output:_ `[scan]__expt.h5` + $ lu_spots2expt -o __expt.h5 -c -d 65.0 -r yes -l yes __spots.h5 + _output:_ `__expt.h5` ##### at the current setup, these three steps can be performed together with appropriate bash scripts: __dark__ - $ bash ../../processing_scripts/dark_integration.sh [dataset] [scan] + $ bash ../../processing_scripts/dark_integration.sh __laser__ - $ bash ../../processing_scripts/laser_integration.sh [dataset] [scan] + $ bash ../../processing_scripts/laser_integration.sh __multidelay__ - $ bash ../../processing_scripts/multi_integration.sh [dataset] [scan] + $ bash ../../processing_scripts/multi_integration.sh ##### 5. orientation matrix search ##### - lu__match_pairs [scan]__expt.h5 ../../2012-02-mono/[compound]__[temp]K.h5 - _output:_ `[scan]__initial.h5` + lu__match_pairs __expt.h5 ../../2012-02-mono/__K.h5 + _output:_ `__initial.h5` ##### parameters to be changed in 'short' dataset (<20 phi angles) ##### @@ -104,14 +104,14 @@ ##### ... or in 'long' dataset (>20 phi angles) ##### - + ##### Expt Clusters Manager always has to be set so as the cluster number in console was in between 30 and 40 ##### - - - +![Ooops, no figure ExptClustersManager_long.png alt=aaargh!>(https://sourceforge.net/p/laueutil/wiki/Images/attachment/ExptClustersManager_long.png) + +![Ooops, no figure console.png alt=aaargh!>(https://sourceforge.net/p/laueutil/wiki/Images/attachment/console.png) ##### When this is done, click 'Calculate'; ##### @@ -127,44 +127,44 @@ ##### Save the output. ##### - _output:_ `[scan]__initial.h5` + _output:_ `__initial.h5`  ##### 6. orientation matrix refinement ##### - $ lu__refine_orientmatrix [scan]__expt.h5 ../../2012-02-mono/[compound]__[temp]K.h5 ../../14IDB_10892.mda.0001 [scan]__initial.h5 - _output:_ `[scan]__expt_rot.h5` + $ lu__refine_orientmatrix __expt.h5 ../../2012-02-mono/__K.h5 ../../14IDB_10892.mda.0001 __initial.h5 + _output:_ `__expt_rot.h5`  - _output:_ `[scan]__expt_full.h5` if full optimization was used + _output:_ `__expt_full.h5` if full optimization was used  ##### 6a. cell parameters / Euler angle control ##### - $ lu__view_matrix_parameters [scan]__expt_rot.h5 - $ lu__view_matrix_parameters [scan]__expt_full.h5 + $ lu__view_matrix_parameters __expt_rot.h5 + $ lu__view_matrix_parameters __expt_full.h5 ##### 7. hkl indices assignment ##### - $ lu__hkl_assign_monolc [scan]__expt.h5 ../../2012-02-mono/[compound]__[temp]K.h5 ../../14IDB_10892.mda.0001 [scan]__expt_rot.h5 [OR [scan]__expt_full.h5] [-P delay] - - _output:_ `[scan]__assign_IU.h5` - for unrefined global orientation matrix - OR `[scan]__assign_CU.h5` - for frame by frame refined orientation matrix - OR `[scan]__assign_FU.h5` - for full refinement + $ lu__hkl_assign_monolc __expt.h5 ../../2012-02-mono/__K.h5 ../../14IDB_10892.mda.0001 __expt_rot.h5 __expt_full.h5> <-P delay> + + _output:_ `__assign_IU.h5` - for unrefined global orientation matrix + OR `__assign_CU.h5` - for frame by frame refined orientation matrix + OR `__assign_FU.h5` - for full refinement  ##### -P delay option is used for 'multi-delay' datasets ##### ##### Alternative, new, presumably more precise algorithm of assignment, is executed as: ##### - $ lu__hkl_assign_monolc_sph [scan]__expt.h5 ../../2012-02-mono/[compound]__[temp]K.h5 ../../14IDB_10892.mda.0001 [scan]__expt_rot.h5 [OR [scan]__expt_full.h5] + $ lu__hkl_assign_monolc_sph __expt.h5 ../../2012-02-mono/__K.h5 ../../14IDB_10892.mda.0001 __expt_rot.h5 __expt_full.h5> - _output:_ `[scan]__assign_sph_IU.h5` - for unrefined global orientation matrix - OR `[scan]__assign_sph_CU.h5` - for frame by frame refined orientation matrix - OR `[scan]__assign_sph_FU.h5` - for full refinement + _output:_ `__assign_sph_IU.h5` - for unrefined global orientation matrix + OR `__assign_sph_CU.h5` - for frame by frame refined orientation matrix + OR `__assign_sph_FU.h5` - for full refinement  @@ -172,28 +172,28 @@ __just ratios for SORTAV__ - $ lu_indexing2ratios_V2 -r ../h5/[scan]__ratios.h5 -R [scan]__ratios.txt -o [scan]__ratios.hkl -n 10 -s 0.5 ../h5/[scan]__assign_FU.h5 + $ lu_indexing2ratios_V2 -r ../h5/__ratios.h5 -R __ratios.txt -o __ratios.hkl -n 10 -s 0.5 ../h5/__assign_FU.h5 __automatic SORTAV run: WINGX SORTAV version__ - ../../processing_scripts/run_windows_sortav.sh [scan]__assign_FU [scan] 9 0.5 + ../../processing_scripts/run_windows_sortav.sh __assign_FU 9 0.5 __automatic SORTAV run LINUX__ - $ bash ../../processing_scripts/run_sortav.sh [scan]__assign_FU [scan] 9 0.5 + $ bash ../../processing_scripts/run_sortav.sh __assign_FU 9 0.5 ##### 9. correlation plot visualization ##### __If hkl unmerged,__ - $ lu__view_correlation_merged_ratios_phi [path]/[scan]__ratios.h5 [path]/[scan]__ratios.h5 + $ lu__view_correlation_merged_ratios_phi /__ratios.h5 /__ratios.h5 __If sortav merging done,__ - $ lu__view_correlation_merged_ratios [path]/[scan]__sortav.hkl [path]/[scan]__sortav.hkl + $ lu__view_correlation_merged_ratios /__sortav.hkl /__sortav.hkl ##### 10. Optional 'sensitive' hkl retrieval: ##### - $ $PYTHON /space-md2/lab/_WORK/python-bin/hkl-h5.py [scan]__ratios.h5 ~/Desktop/RhPNP_significant_positive_experimental_eta.txt '(h,k,l), (-h,-k,-l), (h, -k, l), (-h, k, -l)' + $ $PYTHON /space-md2/lab/_WORK/python-bin/hkl-h5.py __ratios.h5 ~/Desktop/RhPNP_significant_positive_experimental_eta.txt '(h,k,l), (-h,-k,-l), (h, -k, l), (-h, k, -l)' _output:_ `selected_hkl.lst` </pre> </div>Anna MakalWed, 12 Jun 2013 18:45:46 -0000https://sourceforge.netb86cc4e97a2c8c91491520f22738cb376e928376https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v24 +++ v25 @@ -1,7 +1,9 @@ ### directory structure ### +`[compound]` is a name of investigated compound `[dataset]` is a name of directory for a single crystal specimen - it should be the same as the name of directory containing raw frames `[scan]` is a name of a certain scan for a crystal, like 'laser01' or 'powerscan01' +`[path]` is a path to a given file _HDF5 files, i.e. processed files_ @@ -23,7 +25,7 @@ ##### 0. in specified directory, for each compound convert reference monochromatic data to hdf5 format: ##### - $ lu_fcf2mono -o [scan]__XXXK.h5 -e '[(h,k,l),(-h,-k,-l),...]' [compound].fcf + $ lu_fcf2mono -o [scan]__XXXK.h5 -e '[(h,k,l),(-h,-k,-l),...]' [path]/[compound].fcf ##### 1. in new terminal switch to bash: ##### </pre> </div>Anna MakalTue, 11 Jun 2013 22:21:53 -0000https://sourceforge.netbb21860b5992442fb65a99da96ae915be51dd2c5https://sourceforge.net/p/laueutil/wiki/Data%2520Processing%2520Outline/<div class="markdown_content"><pre>--- v23 +++ v24 @@ -19,7 +19,12 @@ ...../WORK/APS-YYYY-MM -### data processing flow ### +### data processing ### + +##### 0. in specified directory, for each compound convert reference monochromatic data to hdf5 format: ##### + + $ lu_fcf2mono -o [scan]__XXXK.h5 -e '[(h,k,l),(-h,-k,-l),...]' [compound].fcf + ##### 1. in new terminal switch to bash: ##### % bash </pre> </div>Anna MakalTue, 11 Jun 2013 22:20:15 -0000https://sourceforge.netcbd7b754a02734adbc49286481db53935c713107