FitAllB modified by

Mon, 07 Jul 2014 17:27:05 -0000

FitAllB - fitting centre-of-mass grain positions, orientation and elastic strain tensors

The FitAllB suite of programs is designed to refine not only the centre-of-mass positions, orientations and elastic strain tensors of each grain, but also a number of global parameters relating to the experimental setup. This can be done for a maximum of two detectors, one detector at the time, hereby making it possible to obtain much better grain positions by refining these against nearfield data prior to fitting the orientations and strains to farfield data. Furthermore FitAllB includes an error estimation routine to give the standard deviations of all refined parameters.
Starting from the orientations, positions and assigned reflections from Grainspotter, the program performs a 12 parameter (position 3, orientation 3 and strain 6) fit for each grain. The output of the program is two grain format files (.gff, simply ascii column files), one containing the refined centre of mass, position, orientation, strain and stress tensor for each grain, the other containing the estimated errors on all of these. In addition the relative grain volumes, calculated based on the intensity of the diffraction peaks, are output, so by the aid of a tesselation routine a 3D orientation and stress/strain map of the polycrystal can in principle be obtained.
The name FitAllB was chosen because the information about the perturbation of the crystal lattices of the individual grains, and thus the grain resolved elastic strains, is contained in the grain specific reciprocal space metric B. By fitting all of these the grain resolved elastic strains, and therefore also - in the components of the stiffness tensor are provided - the type II stresses, can be determined.

References

For further information the user is refered to the FitAllB manual that is automatically downloaded with the .SVN version of FitAllB.
In addition the 3DXRD geometry and image orientation documents mentioned in the users corner are useful reading in order to understand all relevant parameters.

For publications using FitAllB please cite:
Oddershede, J., Schmidt, S., Poulsen, H. F., SĂ¸rensen, H. O., Wright, J., and Reimers, W. (2010). J. Appl. Cryst., 43(3):539â€"549.

Code

All scripts in the FitAllB package are written in python. The central fitting is performed as a least squares minimisation using pyminuit, a python wrapper for the original Minuit variable metric minimisation developed for solving problems in the field of high energy physics at CERN.

Installation

The newest development version can be obtained by checking it out from the subversion (SVN) repository:

svn co https://fable.svn.sourceforge.net/svnroot/fable/FitAllB

Go to the trunk and install FitAllB (most likely this will need to be done with root privileges, e.g. put sudo in front of the command):

python setup.py install

Alternatively the program can be downloaded from the Fable page on sourceforge.net.
Presently the code has been distributed as a zip package of the source code. Download and unpack the latest version, FitAllB-1.1.1 (19/07/12):

unzip FitAllB-1.1.1.zip

All files are unpacked into the directory FitAllB-1.1.1. To install these do:
(NB! Most likely the final step will need to be done with root privileges, e.g. put sudo in front of the command):

cd FitAllB-1.1.1
python setup.py install

Required packages

python, version 2.4 or later
pyminuit, variable metric minimisation module
PIL (python imaging library) - http://www.pythonware.com
numpy - http://www.numpy.org
scipy - http://www.scipy.org/Download
PyCifRW - python library for reading and writing CIF files - http://pycifrw.berlios.de/ (I you install this on an Linux distribution, you might have to install the Python development libraries as well. On Ubuntu (or distributions build on Ubuntu) this can be done on the command line: > sudo apt-get install python-dev)

Required Fable modules

FabIO (FABle Input Output library) - OBS You have to use the numpy version of FabIO - found presently only on subversion. Do the checkout shown below and go to the directory fabio/branch/numpy and run python setyp.py install.
[xfab] (Xtallographic library for FABle) - Presently only available on SVN
[PolyXSim]
ImageD11

Running FitAllB

The FitAllB program package contains 3 different script which basically run the same refinement routine in 3 different preset modes. The reason for this is that in order to obtain accurate grain resolved strain tensors it is of the utmost importance that the global parameters relating to the experimental setup are well calibrated. The calibration or refinement of global parameters must be performed using an undeformed polycrystalline sample.

fitallb.py
The main script for refining centre of mass positions, orientation and strain tensors after having calibrated the global parameters. Only grain specific parameters are refined.
If nearfield diffraction data are available, these are initially used to fit the grain positions, which are then kept fixed in the subsequent farfield refinement of orientations and strains.

fitglobalgrain.py
This is used to refine the global parameters for each grain. The final values and estimated errors are then obtained as the volume weighted average and spread over all grains.

fitgloball.py
This is used to refine the global parameters for all grains in one go. This procedure is more time consuming, but less affected by outlier grains.

Execution
The above scripts are run in the following way:

fitallb.py -i input.inp
fitglobalgrain.py -i input.inp
fitgloball.py -i input.inp

Input

The input in the above examples is an ascii text file with any name and extension, but the extension ".fit" is the default extension taken in account in the Graphical User Interface Fable. The contents of the file can be split into a mandatory set of instructions valid for all script in the FitAllB suite, and some options specific for each mode of application. Flowcharts for the different modes of application are found in the FitAllB documentation that comes with the program. Below a descriptive list of all commands for the input file is given.

Mandatory instructions
The file must contain the following instructions, which are common for fitallb.py, fitglobalgrain.py and fitgloball.py:
NB! Note that the order of the instructions is irrelevant and that extra comments can be added using #, see the example input files.

log_file        name of grainspotter.log from GrainSpotter containing indexing information
   or
res_file        name of .gff file from which to read starting grain parameters
                NB! only one of log_file and res_file are needed.
                    If both are given the information in res_file overrules that from log_file
flt_file        name of peaksearch.flt from peaksearch containing filtered peaks information
                NB! As of September 2011 a .flt.new from ImageD11/makemap containing grain labels for each reflection
                    (labels column) can be used as input along with the .gff (res_file) with the corresponding grain numbers
                    to bypass the slow forward projection step when using res_file. Remember to run makemap with the --no_sort option!
par_file        name of detector.par from ImageD11 containing information about experimental setup

w_step          step size in omega [deg], value in ]0;inf[
w_limit         limits of omega intervals [deg], order irrelevant but there must be an even number of limits,
                values in ]-180;180]
bg              average background level [counts] on detector, value in ]0;2e16], default: 1000
dety_size       detector dimension in y [pixels], default: 2048 (ESRF ID11 Frelon4M)
detz_size       detector dimension in z [pixels], default: 2048
beampol_factor  beam polarisation factor: 1 = fully plane polarised (default), 0 = unpolarised
beampol_direct  direction of the normal to the plane of the primary beam polarisation
                with respect to the sample rotation axis [deg]
                e.g. if the omega rotation axis is parallel to the laboratory z-axis the value is 0.0 (default)
                and if the rotation is along the y-axis it is 90.0
crystal_system  value in {isotropic, cubic, hexagonal, trigonal_high, trigonal_low,
                          tetragonal_high, tetragonal_low, orthorhombic, monoclinic, triclinic}
                NB! isotropic simply means cubic with isotropic elasticity
                trigonal_high: 32, 3m, -3m
                trigonal_low: 3, -3
                tetragonal_high: 422, 4mm, -42m, 4/mmm
                tetragonal_low: 4, -4, 4/m

FitAllB far-field options
For running fitallb.py using only the far-field detector the following instructions might also be useful:

title            anything to specify refinement, but remember quotes:
                 'my very special fitallb farfield refinement title'
structure_file   name of file containing crystallographic information, possible formats: .cif or .pdf
sgno             space group number. Onle needed a structure file is not given,
                 and if the grain info is given as a res_file in stead of a log_file
stress           convert strain to stress (0=no/1=yes), default: 0
c11,c12,c44,..   stiffness constants [MPa], symmetry given by crystal_system will ensure setup
                 of entire tensor from minimal set, which must be given for stress 1, defaults: None
                 crystal_system  required stiffness constants
                 isotropic       c11,c12
                 cubic           c11,c12,c44
                 hexagonal       c11,c12,c13,c33,c44
                 trigonal_high   c11,c12,c13,c14,c33,c44
                 trigonal_low    c11,c12,c13,c14,c25,c33,c44
                 tetragonal_high c11,c12,c13,c33,c44,c66
                 tetragonal_low  c11,c12,c13,c16,c33,c44,c66
                 orthorhombic    c11,c12,c13,c22,c23,c33,c44,c55,c66
                 monoclinic      c11,c12,c13,c15,c22,c23,c25,c33,c35,c44,c46,c55,c66
                 triclinic       c11,c12,c13,c14,c15,c16,c22,c23,c24,c25,c26,c33,c34,c35,c36,c44,c45,c46,c55,c56,c66
abs_mu           linear absorption coefficient for the material of interest at the given wavelength in mm-1, default:0
                 if abs_mu <= 0 no absorption correction is performed, 
                 else the sample is assumed to have rectangular cross section given by abs_xlim and abs_ylim
                 Example for Zr at 80 keV given in test directory
abs_xlim         xmin xmax, default: None, dimensions in mm
abs_ylim         ymin ymax, default: None, dimensions in mm

xyz              fit grain positions (0=no/1=yes), default: 1
rod              fit orientations in Rodrigues representation (0=no/1=yes), default: 1
eps              fit strain tensors (0=no/1=yes), default: 1
fixx             fix grain positions along x (0=no/1=yes), default: 0
fixy             fix grain positions along y (0=no/1=yes), default: 0
fixz             fix grain positions along z (0=no/1=yes), default: 0
                 Useful for planar beam (fixz) and limited scan ranges (fixx or fixy).
tol_grain        tolerance for grain refinement, default: 1e-3
tol_fw_proj      tolerance for forward projection when using res_file 
                 along with an flt_file that does not contain a labels column
                 Must be integer, default: 2
                 Indidates how many steps of w_step along omega and (4,3) pixels
                 along the detector y- and z-directions to search for peaks.

rej_ia           outlier rejection limit [deg] for gvector misorientations, value in ]0;inf],
                 default: 0.2 (undeformed material)
rej_vol          outlier limit for volume/intensity based rejection, value in [3;inf], default: 5
rej_resmedian    outlier limit for robust residual based rejection, value in [1;inf], default: 5
rej_resmean      outlier rejection limit in terms of average residual, value in [3;inf], default: 10
rej_multi        reject peaks assigned to multiple grains (-1=yes all/0=no/1=yes poorest), default: 1
                 -1 removes all multiple assigned reflections (applicable eg in connection with twinning),
                 0 removes none and 1 lets the grain with the best fit in terms of volume and residual keep the spot.
overlap          fraction of identical peaks assigned to grains before merging is considered, value in ]0;1],
                 default: 1 (no merging)
min_refl         minimum number of reflections to considered refinement meaningful, value in [1,inf], default: 9
skip             skip refinement of the named grains from the grainspotter.log, values in {1,2,...}, default: None

Example input file: if10_res.inp
10 grains of IF steel simulated using PolyXSim within an illuminated volume of 0.7x0.7x0.01 mm with random orientations, grain sizes and Gaussian strains with a mean of 0 and a spread of 0.001. The starting values of all grain parameters is read from the if10_near_rotpos.gff generated in the if10 near- and farfield refinement described below to show how resuming a refinement can be done. Note that the grain positions are kept fixed in this farfield refinement as it can be assumed that the position in if10_near_rotpos.gff are better than what can be refined on a far-field detector. Runtime approx.~40 sec per grain on a 2.2 GHzm 3.5 GB RAM PC

FitAllB near-field options
If near-field refinement of grain position should be performed in fitallb.py, the above far-field options can still be used, but xyz must be 0, near_xyz must be 1 and the following options must be added:

near_flt_file     name of peaksearch.flt filtered peaks file from peaksearch containing near-field data
near_par_file     name of detector.par file from ImageD11 containing near-field global parameters
near_bg           average background level [counts] on near-field detector, value in ]0;2e16], default: 100
near_dety_size    near-field detector dimension in y [pixels], default: 1536 (ESRF ID11 quantix)
near_detz_size    near-field detector dimension in z [pixels], default: 1024

In addition the following commands may be useful:

near_xyz              fit grain positions for near-field data (0=no/1=yes), default: 0
near_rod              fit orientations in Rodrigues representation for near-field data (0=no/1=yes), default: 0
near_eps              fit strain tensors for near-field data, 0=no only option
tol_rotpos            tolerance for orientation and position refinement, default: 1e-2

near_rej_ia           near-field outlier rejection limits same meaning and defaults as far-field conterparts
near_rej_vol
near_rej_resmedian
near_rej_resmean
near_rej_multi
near_overlap
near_min_refl         minimum number of reflections to considered near_field refinement meaningful,
                      value in [1,inf], default: 3

Example input file: if10.inp
Same data as for if10_res described above.
The refinement is an example of using both near- and far-field data, and the runtime is approx.~1 min per grain on a 2.2 GHzm 3.5 GB RAM PC.

Global parameter refinements
For running fitglobalgrain.py or fitgloball.py, to refine the global parameter of the far-field detector from an undeformed polycrystalline sample the following commands are mandatory:

xyz              1    # refine positions
rod              1    # refine orientaions
eps              0    # do not refine strains, use undeformed material!

Furthermore it must be specified which global parameters to refine:

w          fit omega stage tilt parameter wy (0=no/1=yes), default: 0
tilt       fit detector tilt parameters tx, ty, tx (0=no/1=yes), default: 0
center     fit detector centre along (0=no/1=yes), default: 0
           In fitglobalgrain only the centre along y is refined, 
           while in fitgloball both y- and z-coordinates are fitted
           (assuming that the centre-of-mass of the indexed grains is zero along z).
pixel      fit pixel size py and pz (0=no/1=yes), default: 0
L          fit sample-to-detector distance (0=no/1=yes), default: 0
d0         fit lattice parameters of undeformed reference unit cell (0=no/1=yes), default: 0
           NB only one of pixel, L and d0 can be refined because of 100% correlations.
           d0 fit only valid for undeformed samples.

For fitgloball.py (fit global parameters for all grains simultaneously) it is possible to refine one or more of the grain positional parameters to be the same for all grains. This may be useful if for instance the grains are known to be situated in a plane perpendicular to the incoming beam at omega=0 (then x should be the same for all grains). This is done by the following commands:

constrx    fit the same position for all grains along the x-axis (0=no/1=yes), default: 0    
constry    fit the same position for all grains along the y-axis (0=no/1=yes), default: 0    
constrz    fit the same position for all grains along the z-axis (0=no/1=yes), default: 0

In addition the following optional commands can be used:

title            same use as stated earlier: 'my very special farfield global parameter fit'
structure_file   name of file containing crystallographic information, possible formats: .cif or .pdf
res_file         name of .gff file from which to read starting grain parameters
                 (optional, NB! overrules information from log_file)

tol_rotpos       tolerance for orientation and position refinement, default: 1e-2
tol_global       tolerance for global parameter refinement, default: 1e-2

rej_ia           outlier rejection limits, rej_multi and overlap same meaning and defaults as stated for fitallb.py
rej_vol
rej_resmedian
rej_resmean
rej_multi
overlap
min_refl         minimum number of reflections to considered refinement meaningful, value in [1,inf],
                 default: 9, but should be increased to around 50% of the expected average number of reflections
skip             skip refinement of the named grains from the grainspotter.log, values in {1,2,...}, default: None

Example input files: fitglobal5_5.inp, fitglobal5_res2.inp
The above input files can be used for both fitglobalgrain.py and fitgloball.py. The fit of the global parameters is performed for 5 grains of undeformed IF steel simulated within an illuminated volume of 0.7x0.7x0.01 mm with random orientations and grain sizes. The input file of global parameters, if100_globals_wedge.par, has a slight offset in all of these as compared to the values used to simulate the diffraction images. The first file, fitglobal5_5.inp, refines 5 cycles of positions, orientations and global paramters starting from the global parameters in if100_globals_wedge.par and the grain position, orientations and assigned reflections of if100_globals_wedge_5.log. The second file, fitglobal5_res2.inp, resumes the refinement from the global parameters, fitglobal5_5_globals4_fab.par, and grains, fitglobal5_5_rotpos4.gff, obtained after the first 5 cycles to illustrate that more cycle can always be done later if it is judged that the refinement hasn't converged yet.
NB! Note that min_refl is set to 45 for the global parameter refinements while it was 36 for the strain refinement examples above that are performed for the same material and omega ranges. This is done to only refine the global parameters from the largest and by assumption best determined grains.

Output

All output files will be saved in a directory with the name of the used input file up to the first punctuation, so in the example using input.inp as input file, the output directory will be named input. This directory name will also be used as the stem of the generated output files which are the following:

General output files

input_log.log             Main output files for far- and near-field:
input_near_log.log        Global parameter values and standard deviations
                          Residual values, refinement times and number of
                          outliers for each refinement step

input_rej.txt             List of rejected peaks and skipped grains for each refinement step.
input_near_rej.txt        The reason to reject the peak, i.e. intensity or residual, is specified

The quality of the fit can be judged from the input_log.log and the standard output where the number of assigned reflections per grain and the residuals are given.
Likewise the mean_IA given in the .gff can be used as an indication in this direction.

Grain refinement specific output files

input_cor.txt             Correlation matrix of the refined parameters for each grain
input_near_cor.txt        New matrices appended after each grain and final step

input_cov.txt             As input_cor.txt, except this is the covariance matrices
input_near_cov.txt

input_cov_eps_sig.gff     Covariance matrices of strain and stress enabling the 
                          calculation of error bars along any direction, for instance
                          for resolved shear stresses

nearfield:                For each step in the algorithm the values of the parameters
input_near_rotpos.gff     for each grain is listed in these (grain file format) files.
                          Parameter order: grainno mean_IA grainvolume x y z
                          rodx rody rodz U11 U12 U13 U21 U22 U23 U31 U32 U33
                          eps11 eps22 eps33 eps23 eps13 eps12
farfield:                 eps11_s eps22_s eps33_s eps23_s eps13_s eps12_s
input_grain.gff           sig11 sig22 sig33 sig23 sig13 sig12
input_final.gff           sig11_s sig22_s sig33_s sig23_s sig13_s sig12_s
                          NB! If more than one final step is required the output file is
                          overwritten in each step

input_errors.txt          Estimated standard deviation of all the grain parameters
                          given in the same order as listed above for the values.
                          The file is updated after each refinement step.
                          Step specific output files are names input_step_errors.txt

The input_grain.gff refers to the first refinement of orientation, strains and possibly positions for all grains,
the input_final.gff collects the information from all subsequent refinement cycles. The most important files are input_final.gff and input_errors.gff.
The parameters eps11, eps22,... are the components of the strain tensor in the Cartesian grain coordinate system;
eps11_s, eps22_s,... are the strain tensor components in the sample system;
sig11, sig22,... and the the components of the stress tensor in the Cartesian grain coordinate system;
sig11_s, sig22_s,... are the strain tensor components in the sample system;
and finally sig_eta and sig_tth are peak shapes in degrees taken as the median values for the assigned reflections for each grain.

Global parameter refinement specific output files

input_global.txt          Correlation and covariance matrices of the refined
                          global parameters are appended to this file after each cycle

input_globals#_fab.par    ImageD11 type detector.par file with refined global parameters

input_rotpos#.gff         Grain file format with refined grain parameters

Here the # means the cycle number, thus anything between 0 and cycle-1, where cycle is the specified number of refinement cycles.
The most important file is input_globalsX_fab.par, where X = cycle-1.

Outlier rejection

For a thorough description of the applied outlier rejection schemes and the related parameters please refer to the FitAllB documentation.
Note, however, that the higher a value given for the rej_ia, rej_vol, rej_resmedian and rej_resmean commands, the more less peaks are rejected.

Recent changes to FitAllB