Re: [Psidev-qc-dev] quality metrics

[walzer <wa...@in...>]

Dear all,

as interest in a telco is not putting doodle or me in a difficult spot 
to pick a date, maybe a little digest of the newest metrics we could 
discuss, will kindle some more interest 
(http://doodle.com/poll/bv36kwe3hfmgwm8v).

CPTAC labeling QC
We haven't touched upon QC for labeling techniques, and the CPTAC study 
considerations seem like a good start (thanks to Karl Clauser). 
Summarised, these would cover ideally the completeness of labellings 
(counting the number of labeled vs. unlabeled termini) Also, considering 
the S/N of not only all spectra (as we would now with MS2-2) but also of 
only those with reporter ions seems to be a good idea. MassUP - QC 
MassUp covers base with a number of metrics that also we use (e.g. the 
name of the sample, number of spectra, min. Mass, max. Mass, avg. 
Masses, ...) but also ask for a classas label of the sample - as already 
mentioned this is highly interesting to bring the experimental design 
into the loop to do proper QC for a set of several measurements.They 
also define POPXX as a the number of peaks with Percentage of Presence- 
so I am not that familiar with MALDI, but it seems to be a a good idea 
in general to also look at the replication success on a spectrum and 
peaks level! though I am not sure which kind of peaks they are 
considering in this MALID setting with MassUp analysis. msPurity Lawson 
et al. published an interesting metric to assist with spectral library 
search result assessments in metabolomics. The metric assesses the 
purity of the precursor producing a tandem spectrum. Having more than 
just the targeted ion species in your selection for fragmentation messes 
with the lookup of your spectrum in a library with (assumedly) pure 
spectra. But I think this is also useful in other context, at least I 
come across some mixed spectra sometimes, especially when sample 
abundance or ionisation yield is low and the fill times go up to the 
max. set. They calculate this metric as a ratio of precursor intensity 
and sum. intensity of the respective isolation window. As I read it, it 
is assumed that the highest peak in the isolation window is the targeted 
precursor peak, and it's intensity for the MS2 is interpolated from the 
intensity in the neighboring MS1. I think it would also be good to have 
a classification of analysis metrics and acquisition metrics. I think it 
came up in Ghent, but our metrics list was not long enough to make such 
separations. This goes in the same direction as what Robert was saying 
in his mail, that there might be as well a lot of good/reliable results 
in bad runs. After all, often times analysis algorithms are robust 
enough to navigate around bits of bad data. The ID(free) classification 
goes in that direction, but I think analysis metrics would comprise more.

best,

   Mathias

On 07.02.2017 21:05, Mathias Walzer wrote:
> Dear all,
>
> let's discuss new QC metrics and definition refinement of old ones here. We might start here:
>
> There are some interesting new developments:
> There is the Mass-Up framework ( http://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-015-0752-4 ) for Maldi TOF QC, see Davids post on Quality control in Mass-Up
>
> And also new developments from the metabolomics side with msPurity (http://pubs.acs.org/doi/abs/10.1021/acs.analchem.6b04358)
>
> For labelled experiments, Spectrum Mill is using quality control metrics proven on CPTAC breast cancer proteome dataset (http://www.nature.com/nature/journal/v534/n7605/abs/nature18003.html , see Karl Clausers Re: [Psidev-qc-dev] HUPO PSI Quality Control Working Group post with the SpectrumMill documentation attached)
>
> Or, which would be much to my delight, revise the collection of more general LC-MS/MS metrics already in use in qcML (https://github.com/mwalzer/qcML-development/blob/feature/QC-CV-Metric_overview/cv/qc-metric-collection.md).
>
> So to the matter, how do you think do any metrics from the above fit in our CV definition schema? Would some need generalisation for the sake of minimising redundancy/overlap, or on the other side are some not specific enough?
> Do you have a great metric that is missing in our discussions?
>
> best,
>    Mathias
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Psidev-qc-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-qc-dev