lu_hdf2spots

lu_hdf2spots

Usage: lu_hdf2spots_v2_lib.py [options] file__mccd.h5

Options:
  -h, --help            show this help message and exit
  -o FO, --output=FO    output file (HDF5) name [default: file__spots.h5]
  -m METHOD, --method=METHOD
                        [default: n_removed] method used to estimate
                        background level (allowed values: n_removed, kruskal)
  -n N, --n=N           percent of extreme values to skip while calculating
                        median and IQR when fixed percent of values is removed
                        [default: 25]
  -A ALPHA, --alpha=ALPHA
                        Alpha-value to use during Kruskal-Wallis test
                        [default: 0.05]
  -c CUTOFF, --cutoff=CUTOFF
                        cutoff used during mask definition [default: 3.0]
  -e N_EROSIONS, --erosions=N_EROSIONS
                        number of erosions [default: 1]
  -d N_DILATIONS, --dilations=N_DILATIONS
                        number of dilations [default: 2]
  -s STRUCTURE, --structure=STRUCTURE
                        [default: 2,2] see:
                        scipy.ndimage.morphology.generate_binary_structure
  -a ANGLES, --angles=ANGLES
                        check for minimum number of angle values present in
                        lauecollect/angle [default: 10]
  -S SYNC               [default: none] synchronize masks (allowed values:
                        none, pairs, blocks)
  -C COMPRESSION, --compression=COMPRESSION
                        compression algorithm [default: gzip]

Additional options were added with respect to lu_hdf2spots program. The -m option offers choice between simple (n strongest outliers) and more precise method of Kruskall-Wallis test-based outlier rejection in initial background estimation. The -m kruskal option can only be applied for 'laser' datasets. The additional -A option sets the alpha value for the statistical procedure.

Option -n N controls process of gathering pixel statistics. Its meaning is the following: for each pixel coordinates values from all frames are collected and sorted and than N% largest values are removed before calculating intensity statistics (avg, std, iqr). N = 0 means that the default value 25% is used. The precise number of points removed is stored as attribute 'n_removed' of 'info' group.

A reasonable value of N would be: 10..20% x number of frames

The following values are calculated and are available in HDF5 file in group stats:
avg:: average intensity (over trimmed sample)
std:: standard deviation (over trimmed sample)
iqr:: interquartile range Q75 – Q25 (over trimmed sample)
max:: maximum value (over full sample)
arg:: “arg max” - frame number of maximum intensity (over full sample)

This arrays can be visualized using hdfview by right-clicking on the array, choosing '€œOpen As' and than setting €œ'Display As' to 'Image' and choosing some color palette for example '€œNature' or '€œRainbow'€. No spots should be visible on 'avg' array - only the slowly changing beam background (with beamstop shadow) and some detector-related grid-like pattern. If spots do show than N should be increased. 'std' should show nearly only the detector-related pattern.

Option -c CUTOFF controls process of calculating raw masks. Pixel [X,Y] on frame i is considered to be part of a reflexion if:

     I[i,X,Y]   >=   `_avg_`[X,Y]    +    CUTOFF * `_std_`[X,Y]

The lower value of CUTOFF the more noisy masks, the higher value the more weak reflexions are possibly ignored.

Options -e N_EROSIONS and -d N_DILATIONS control filtering masks. N_EROSIONS determines number of €œborder pixel removing operations (effectively removing single pixels or lines) and N_DILATIONS determines number of spot re-growing operations.

Option -S SYNC can be used only for laser data. It will calculate synchronized masks for either each pair of off/on frames (-S pairs) or for whole blocks (-S blocks), where blocks is defined as sets of all off/on pairs of frames taken at the same angle. A pixel is marked as belonging to a spot (nonzero value of mask) on a synchronized mask if it belongs to a spot on any of frames that are being synchronized.

Masks can be visualized using hdfview by right-clicking on the masks arrays ('masks_raw'[i,X,Y] and 'masks_filtered'[i,X,Y] with i = N_masks-1), choosing '€œOpen As' €and than setting '€œDisplay As'€ to 'Image'€ and choosing some color palette for example 'Nature' or '€œRainbow'€. 'Height' should be set to 'dim1', '€œWidth'€ to 'dim 2' and 'Depth'€ to 'dim 0'.

Group 'masks' contains two arrays being the result of applying spot finding&labeling algorithm to 'masks_synced' if present or 'masks_filtered' otherwise:
labels:: an array (labels[i,X,Y] with i = N_masks-1) containing for each pixel a spot label (integer number > 0, 0 <==> pixel is not an integrable spot). The array can be visualized in the same manner as mask arrays,
counts:: a 1D array (counts[i] with i = N_masks-1) containing information about number of spots that has been found (and labeled 1, 2, ...) on each mask. It can be visualized by opening and selecting the only possible column and than clicking on chart icon in toolbar at the top of the window.

Group spots contains arrays with the data on all spots on all frames combined:
I:: integrated intensity
XY:: position of most intense pixel in the spot
box:: box on frame containing whole spot
frame:: frame number (index)
label:: assigned label, note: labels for spots obtained with different masks are not related!
mask:: index of mask in masks_filtered_' or '_masks_synced and also for the corresponding slice in labels
pixel_list_BE:: auxiliary data: indices for slicing pixel_list array containing pixel indices (in a flattened image) for all pixels belonging to a given spot

frame2spotsBE:: auxiliary data: indices for slicing arrays in spots group to obtain spots for a given frame.

Related

Wiki: Programs