Mapping quality reported by BFAST is a scaled edit distance. It is assumed that the "-q" option in "bfast localalign" is Q.
Let AS1 be the best (greatest) alignment score and AS2 the second best alignment score (if it exists). Let X be the score of the minimum unit of edit. For nucleotide data this is the score of a base mismatch, while for color space data this is the score of a color mismatch.
- If AS2 does not exist, then the mapping quality is 255.
- Else if AS1 is equal to AS2, then the mapping quality is 0.
- Else iff AS2 exists, then the mapping quality is:
MAPQ = Q * (AS1 - AS2) / X
The latter calculation is can be interpreted as a log-odds ratio of AS1 versus AS2 when AS1, AS2, and X are log-odds ratios themselves. Another interpretation is that it measures the number of edits the next best alignment is away from the best alignment, scaled by Q.
ABI SOLiD Base Qualities
For ABI SOLiD data, base qualities are calculated using the following formula:
- If the base is the last decoded base (last base sequenced), then the base quality is exactly equal to the color quality of the last color.
- Else if the two colors observing the base are not called sequencing errors, then the base quality is the sum of the two color qualities plus ten.
- Else if exactly one out of the two colors observing the base are called sequencing errors, then the base quality is the difference between the qualities non-sequencing-error-color and the sequencing-error-color.
- Else the base quality is zero.
This follows the MAQ 0.7.1 convention.
Note that if a read base is called a SNP (differs from the reference), then the two color differences implied by this SNP are not sequencing errors (they are caused by a SNP!). Therefore, the quality value for a SNP is not zero.