psidev-ms-dev

[Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Eric D. <eri...@gm...>]

From: Joshua Tasman 

Hi all,

There are cases where the m/z array data is contiguous (standard profile
mode) vs noncontiguous (centroided, or thresholded profile).  Could we add
an attribute to the binary array section (or above) describing this?  Or is
this best left to the parsers to automatically determine?

One tricky case that might not be quickly/easily scanned for: imagine a mode
that preserves profile data points around detected peaks; so the first large
# of data points might appear to be the same m/z delta.  Hence, a quick
check by the parser might not get this correct.

Thanks for considering,

Josh

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Pierre-Alain B. <pie...@is...>]

Hi all,

Eric Deutsch wrote:
> From: Joshua Tasman 
>
> Hi all,
>
> There are cases where the m/z array data is contiguous (standard profile
> mode) vs noncontiguous (centroided, or thresholded profile).  Could we add
> an attribute to the binary array section (or above) describing this?  Or is
> this best left to the parsers to automatically determine?
>   
good suggestion, why not having the possibility to have one attribute
> One tricky case that might not be quickly/easily scanned for: imagine a mode
> that preserves profile data points around detected peaks; so the first large
> # of data points might appear to be the same m/z delta.  Hence, a quick
> check by the parser might not get this correct.
>   
but this tricky get less tricky if the parser considers the start and 
end m/z values of the spectrum.  It is relartively rare that a peak is 
starting to appear in a m/z scale right at the start and end of the m/z 
range. Therefore one can expect a set of empty values in these edges. 
This happens in some Waters QTOFs that do a low intensity filter in the 
profile mode exported data.

Pierre-Alain
> Thanks for considering,
>
> Josh
>
>
>
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>   

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Darren K. <Dar...@cs...>]

What is the use case for this?

I think everyone has an intuitive understanding of "contiguous", but I  
think it is a little more complicated to define properly.  The  
intuitive definition is "regularly spaced with no missing data points".

When accessing profile data in RAW files from Thermo FT instruments,  
the spectra are thresholded, and this gives rise to the islands that  
Josh mentioned during the conference call.  However, in FT data  
without islands (e.g. if thresholding is disabled, or perhaps with  
preprocessing to extract a single island or fill in missing data  
points), the data points are still not regularly spaced in m/z.  (The  
data points are regularly spaced in frequency space, but the frequency- 
 >m/z calibration function is non-linear).  Because the data points  
are not regularly spaced, any processing must use some form of  
interpolation, so I'm not sure how a "contiguous" flag will help.


Darren



On Apr 16, 2008, at 6:48 AM, Eric Deutsch wrote:

>
> From: Joshua Tasman
>
> Hi all,
>
> There are cases where the m/z array data is contiguous (standard  
> profile
> mode) vs noncontiguous (centroided, or thresholded profile).  Could  
> we add
> an attribute to the binary array section (or above) describing  
> this?  Or is
> this best left to the parsers to automatically determine?
>
> One tricky case that might not be quickly/easily scanned for:  
> imagine a mode
> that preserves profile data points around detected peaks; so the  
> first large
> # of data points might appear to be the same m/z delta.  Hence, a  
> quick
> check by the parser might not get this correct.
>
> Thanks for considering,
>
> Josh
>
>
>
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save  
> $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Joshua T. <jt...@sy...>]

Hi Darren,

You did understand what I meant.  I've been working with the Agilent MassHunter-format machines which give huge non-thresholded profile arrays (at least currently), where I'd like to make note of data reduction that might be applied at conversion time-- and make it easier for the parsers to handle "choppy profile" data.  However, if Thermo data already looks that way (I didn't realize), maybe it's not necessary.

I incorrectly believed that "profile" implied "no gaps in m/z spacing".

-Josh


Darren Kessner wrote:
> What is the use case for this?
> 
> I think everyone has an intuitive understanding of "contiguous", but I  
> think it is a little more complicated to define properly.  The  
> intuitive definition is "regularly spaced with no missing data points".
> 
> When accessing profile data in RAW files from Thermo FT instruments,  
> the spectra are thresholded, and this gives rise to the islands that  
> Josh mentioned during the conference call.  However, in FT data  
> without islands (e.g. if thresholding is disabled, or perhaps with  
> preprocessing to extract a single island or fill in missing data  
> points), the data points are still not regularly spaced in m/z.  (The  
> data points are regularly spaced in frequency space, but the frequency- 
>  >m/z calibration function is non-linear).  Because the data points  
> are not regularly spaced, any processing must use some form of  
> interpolation, so I'm not sure how a "contiguous" flag will help.
> 
> 
> Darren
> 
> 
> 
> On Apr 16, 2008, at 6:48 AM, Eric Deutsch wrote:
> 
>> From: Joshua Tasman
>>
>> Hi all,
>>
>> There are cases where the m/z array data is contiguous (standard  
>> profile
>> mode) vs noncontiguous (centroided, or thresholded profile).  Could  
>> we add
>> an attribute to the binary array section (or above) describing  
>> this?  Or is
>> this best left to the parsers to automatically determine?
>>
>> One tricky case that might not be quickly/easily scanned for:  
>> imagine a mode
>> that preserves profile data points around detected peaks; so the  
>> first large
>> # of data points might appear to be the same m/z delta.  Hence, a  
>> quick
>> check by the parser might not get this correct.
>>
>> Thanks for considering,
>>
>> Josh
>>
>>
>>
>>
>>
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save  
>> $100.
>> Use priority code J8TL2D2.
>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>> _______________________________________________
>> Psidev-ms-dev mailing list
>> Psi...@li...
>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Darren K. <Dar...@cs...>]

I see -- I definitely think it's useful to flag the data reduction.   
Perhaps this should be encoded as a cvParam in <dataProcessing>?

Darren



On Apr 16, 2008, at 10:15 AM, Joshua Tasman wrote:

> Hi Darren,
>
> You did understand what I meant.  I've been working with the Agilent  
> MassHunter-format machines which give huge non-thresholded profile  
> arrays (at least currently), where I'd like to make note of data  
> reduction that might be applied at conversion time-- and make it  
> easier for the parsers to handle "choppy profile" data.  However, if  
> Thermo data already looks that way (I didn't realize), maybe it's  
> not necessary.
>
> I incorrectly believed that "profile" implied "no gaps in m/z  
> spacing".
>
> -Josh
>
>
> Darren Kessner wrote:
>> What is the use case for this?
>>
>> I think everyone has an intuitive understanding of "contiguous",  
>> but I
>> think it is a little more complicated to define properly.  The
>> intuitive definition is "regularly spaced with no missing data  
>> points".
>>
>> When accessing profile data in RAW files from Thermo FT instruments,
>> the spectra are thresholded, and this gives rise to the islands that
>> Josh mentioned during the conference call.  However, in FT data
>> without islands (e.g. if thresholding is disabled, or perhaps with
>> preprocessing to extract a single island or fill in missing data
>> points), the data points are still not regularly spaced in m/z.  (The
>> data points are regularly spaced in frequency space, but the  
>> frequency-
>>> m/z calibration function is non-linear).  Because the data points
>> are not regularly spaced, any processing must use some form of
>> interpolation, so I'm not sure how a "contiguous" flag will help.
>>
>>
>> Darren
>>
>>
>>
>> On Apr 16, 2008, at 6:48 AM, Eric Deutsch wrote:
>>
>>> From: Joshua Tasman
>>>
>>> Hi all,
>>>
>>> There are cases where the m/z array data is contiguous (standard
>>> profile
>>> mode) vs noncontiguous (centroided, or thresholded profile).  Could
>>> we add
>>> an attribute to the binary array section (or above) describing
>>> this?  Or is
>>> this best left to the parsers to automatically determine?
>>>
>>> One tricky case that might not be quickly/easily scanned for:
>>> imagine a mode
>>> that preserves profile data points around detected peaks; so the
>>> first large
>>> # of data points might appear to be the same m/z delta.  Hence, a
>>> quick
>>> check by the parser might not get this correct.
>>>
>>> Thanks for considering,
>>>
>>> Josh
>>>
>>>
>>>
>>>
>>>
>>>
>>> -------------------------------------------------------------------------
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save
>>> $100.
>>> Use priority code J8TL2D2.
>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>> _______________________________________________
>>> Psidev-ms-dev mailing list
>>> Psi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save  
>> $100.
>> Use priority code J8TL2D2.
>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>> _______________________________________________
>> Psidev-ms-dev mailing list
>> Psi...@li...
>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save  
> $100.
> Use priority code J8TL2D2.
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Matthew C. <mat...@va...>]

Yes, I agree too. Intensity thresholding should most definitely be 
encoded in mzML. It is one of those basic kinds of data processing like 
peak picking that we just have to support even though it's not strictly 
speaking the raw data.

-Matt


Darren Kessner wrote:
> I see -- I definitely think it's useful to flag the data reduction.   
> Perhaps this should be encoded as a cvParam in <dataProcessing>?
>
> Darren
>
>
>
> On Apr 16, 2008, at 10:15 AM, Joshua Tasman wrote:
>
>   
>> Hi Darren,
>>
>> You did understand what I meant.  I've been working with the Agilent  
>> MassHunter-format machines which give huge non-thresholded profile  
>> arrays (at least currently), where I'd like to make note of data  
>> reduction that might be applied at conversion time-- and make it  
>> easier for the parsers to handle "choppy profile" data.  However, if  
>> Thermo data already looks that way (I didn't realize), maybe it's  
>> not necessary.
>>
>> I incorrectly believed that "profile" implied "no gaps in m/z  
>> spacing".
>>
>> -Josh
>>
>>
>> Darren Kessner wrote:
>>     
>>> What is the use case for this?
>>>
>>> I think everyone has an intuitive understanding of "contiguous",  
>>> but I
>>> think it is a little more complicated to define properly.  The
>>> intuitive definition is "regularly spaced with no missing data  
>>> points".
>>>
>>> When accessing profile data in RAW files from Thermo FT instruments,
>>> the spectra are thresholded, and this gives rise to the islands that
>>> Josh mentioned during the conference call.  However, in FT data
>>> without islands (e.g. if thresholding is disabled, or perhaps with
>>> preprocessing to extract a single island or fill in missing data
>>> points), the data points are still not regularly spaced in m/z.  (The
>>> data points are regularly spaced in frequency space, but the  
>>> frequency-
>>>       
>>>> m/z calibration function is non-linear).  Because the data points
>>>>         
>>> are not regularly spaced, any processing must use some form of
>>> interpolation, so I'm not sure how a "contiguous" flag will help.
>>>
>>>
>>> Darren
>>>
>>>
>>>
>>> On Apr 16, 2008, at 6:48 AM, Eric Deutsch wrote:
>>>
>>>       
>>>> From: Joshua Tasman
>>>>
>>>> Hi all,
>>>>
>>>> There are cases where the m/z array data is contiguous (standard
>>>> profile
>>>> mode) vs noncontiguous (centroided, or thresholded profile).  Could
>>>> we add
>>>> an attribute to the binary array section (or above) describing
>>>> this?  Or is
>>>> this best left to the parsers to automatically determine?
>>>>
>>>> One tricky case that might not be quickly/easily scanned for:
>>>> imagine a mode
>>>> that preserves profile data points around detected peaks; so the
>>>> first large
>>>> # of data points might appear to be the same m/z delta.  Hence, a
>>>> quick
>>>> check by the parser might not get this correct.
>>>>
>>>> Thanks for considering,
>>>>
>>>> Josh
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save
>>>> $100.
>>>> Use priority code J8TL2D2.
>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>> _______________________________________________
>>>> Psidev-ms-dev mailing list
>>>> Psi...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>         
>>> -------------------------------------------------------------------------
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save  
>>> $100.
>>> Use priority code J8TL2D2.
>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>> _______________________________________________
>>> Psidev-ms-dev mailing list
>>> Psi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>       
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>> Don't miss this year's exciting event. There's still time to save  
>> $100.
>> Use priority code J8TL2D2.
>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>> _______________________________________________
>> Psidev-ms-dev mailing list
>> Psi...@li...
>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>     
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>   

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Joshua T. <jt...@sy...>]

Cool.  So, no need to denote "choppy profile" vs "contiguous profile" at the binary array level?

(Yay, seems like I can post again! :) )

Josh


Matthew Chambers wrote:
> Yes, I agree too. Intensity thresholding should most definitely be 
> encoded in mzML. It is one of those basic kinds of data processing like 
> peak picking that we just have to support even though it's not strictly 
> speaking the raw data.
> 
> -Matt
> 
> 
> Darren Kessner wrote:
>> I see -- I definitely think it's useful to flag the data reduction.   
>> Perhaps this should be encoded as a cvParam in <dataProcessing>?
>>
>> Darren
>>
>>
>>
>> On Apr 16, 2008, at 10:15 AM, Joshua Tasman wrote:
>>
>>   
>>> Hi Darren,
>>>
>>> You did understand what I meant.  I've been working with the Agilent  
>>> MassHunter-format machines which give huge non-thresholded profile  
>>> arrays (at least currently), where I'd like to make note of data  
>>> reduction that might be applied at conversion time-- and make it  
>>> easier for the parsers to handle "choppy profile" data.  However, if  
>>> Thermo data already looks that way (I didn't realize), maybe it's  
>>> not necessary.
>>>
>>> I incorrectly believed that "profile" implied "no gaps in m/z  
>>> spacing".
>>>
>>> -Josh
>>>
>>>
>>> Darren Kessner wrote:
>>>     
>>>> What is the use case for this?
>>>>
>>>> I think everyone has an intuitive understanding of "contiguous",  
>>>> but I
>>>> think it is a little more complicated to define properly.  The
>>>> intuitive definition is "regularly spaced with no missing data  
>>>> points".
>>>>
>>>> When accessing profile data in RAW files from Thermo FT instruments,
>>>> the spectra are thresholded, and this gives rise to the islands that
>>>> Josh mentioned during the conference call.  However, in FT data
>>>> without islands (e.g. if thresholding is disabled, or perhaps with
>>>> preprocessing to extract a single island or fill in missing data
>>>> points), the data points are still not regularly spaced in m/z.  (The
>>>> data points are regularly spaced in frequency space, but the  
>>>> frequency-
>>>>       
>>>>> m/z calibration function is non-linear).  Because the data points
>>>>>         
>>>> are not regularly spaced, any processing must use some form of
>>>> interpolation, so I'm not sure how a "contiguous" flag will help.
>>>>
>>>>
>>>> Darren
>>>>
>>>>
>>>>
>>>> On Apr 16, 2008, at 6:48 AM, Eric Deutsch wrote:
>>>>
>>>>       
>>>>> From: Joshua Tasman
>>>>>
>>>>> Hi all,
>>>>>
>>>>> There are cases where the m/z array data is contiguous (standard
>>>>> profile
>>>>> mode) vs noncontiguous (centroided, or thresholded profile).  Could
>>>>> we add
>>>>> an attribute to the binary array section (or above) describing
>>>>> this?  Or is
>>>>> this best left to the parsers to automatically determine?
>>>>>
>>>>> One tricky case that might not be quickly/easily scanned for:
>>>>> imagine a mode
>>>>> that preserves profile data points around detected peaks; so the
>>>>> first large
>>>>> # of data points might appear to be the same m/z delta.  Hence, a
>>>>> quick
>>>>> check by the parser might not get this correct.
>>>>>
>>>>> Thanks for considering,
>>>>>
>>>>> Josh
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------------------------
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save
>>>>> $100.
>>>>> Use priority code J8TL2D2.
>>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>>> _______________________________________________
>>>>> Psidev-ms-dev mailing list
>>>>> Psi...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>>         
>>>> -------------------------------------------------------------------------
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save  
>>>> $100.
>>>> Use priority code J8TL2D2.
>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>> _______________________________________________
>>>> Psidev-ms-dev mailing list
>>>> Psi...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>       
>>> -------------------------------------------------------------------------
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>> Don't miss this year's exciting event. There's still time to save  
>>> $100.
>>> Use priority code J8TL2D2.
>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>> _______________________________________________
>>> Psidev-ms-dev mailing list
>>> Psi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>     
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
>> Don't miss this year's exciting event. There's still time to save $100. 
>> Use priority code J8TL2D2. 
>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>> _______________________________________________
>> Psidev-ms-dev mailing list
>> Psi...@li...
>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>
>>   
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[Pierre-Alain B. <pie...@is...>]

I aggree too with the annotation of this thresholding as processing.
Important for some LC-MS "image" analysis tools that can make a 
algorithmic different analysis between those kind of data
Pierre-Alain

Joshua Tasman wrote:
> Cool.  So, no need to denote "choppy profile" vs "contiguous profile" at the binary array level?
>
> (Yay, seems like I can post again! :) )
>
> Josh
>
>
> Matthew Chambers wrote:
>   
>> Yes, I agree too. Intensity thresholding should most definitely be 
>> encoded in mzML. It is one of those basic kinds of data processing like 
>> peak picking that we just have to support even though it's not strictly 
>> speaking the raw data.
>>
>> -Matt
>>
>>
>> Darren Kessner wrote:
>>     
>>> I see -- I definitely think it's useful to flag the data reduction.   
>>> Perhaps this should be encoded as a cvParam in <dataProcessing>?
>>>
>>> Darren
>>>
>>>
>>>
>>> On Apr 16, 2008, at 10:15 AM, Joshua Tasman wrote:
>>>
>>>   
>>>       
>>>> Hi Darren,
>>>>
>>>> You did understand what I meant.  I've been working with the Agilent  
>>>> MassHunter-format machines which give huge non-thresholded profile  
>>>> arrays (at least currently), where I'd like to make note of data  
>>>> reduction that might be applied at conversion time-- and make it  
>>>> easier for the parsers to handle "choppy profile" data.  However, if  
>>>> Thermo data already looks that way (I didn't realize), maybe it's  
>>>> not necessary.
>>>>
>>>> I incorrectly believed that "profile" implied "no gaps in m/z  
>>>> spacing".
>>>>
>>>> -Josh
>>>>
>>>>
>>>> Darren Kessner wrote:
>>>>     
>>>>         
>>>>> What is the use case for this?
>>>>>
>>>>> I think everyone has an intuitive understanding of "contiguous",  
>>>>> but I
>>>>> think it is a little more complicated to define properly.  The
>>>>> intuitive definition is "regularly spaced with no missing data  
>>>>> points".
>>>>>
>>>>> When accessing profile data in RAW files from Thermo FT instruments,
>>>>> the spectra are thresholded, and this gives rise to the islands that
>>>>> Josh mentioned during the conference call.  However, in FT data
>>>>> without islands (e.g. if thresholding is disabled, or perhaps with
>>>>> preprocessing to extract a single island or fill in missing data
>>>>> points), the data points are still not regularly spaced in m/z.  (The
>>>>> data points are regularly spaced in frequency space, but the  
>>>>> frequency-
>>>>>       
>>>>>           
>>>>>> m/z calibration function is non-linear).  Because the data points
>>>>>>         
>>>>>>             
>>>>> are not regularly spaced, any processing must use some form of
>>>>> interpolation, so I'm not sure how a "contiguous" flag will help.
>>>>>
>>>>>
>>>>> Darren
>>>>>
>>>>>
>>>>>
>>>>> On Apr 16, 2008, at 6:48 AM, Eric Deutsch wrote:
>>>>>
>>>>>       
>>>>>           
>>>>>> From: Joshua Tasman
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> There are cases where the m/z array data is contiguous (standard
>>>>>> profile
>>>>>> mode) vs noncontiguous (centroided, or thresholded profile).  Could
>>>>>> we add
>>>>>> an attribute to the binary array section (or above) describing
>>>>>> this?  Or is
>>>>>> this best left to the parsers to automatically determine?
>>>>>>
>>>>>> One tricky case that might not be quickly/easily scanned for:
>>>>>> imagine a mode
>>>>>> that preserves profile data points around detected peaks; so the
>>>>>> first large
>>>>>> # of data points might appear to be the same m/z delta.  Hence, a
>>>>>> quick
>>>>>> check by the parser might not get this correct.
>>>>>>
>>>>>> Thanks for considering,
>>>>>>
>>>>>> Josh
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------------------------
>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>>> $100.
>>>>>> Use priority code J8TL2D2.
>>>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>>>> _______________________________________________
>>>>>> Psidev-ms-dev mailing list
>>>>>> Psi...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>>>         
>>>>>>             
>>>>> -------------------------------------------------------------------------
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save  
>>>>> $100.
>>>>> Use priority code J8TL2D2.
>>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>>> _______________________________________________
>>>>> Psidev-ms-dev mailing list
>>>>> Psi...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>>       
>>>>>           
>>>> -------------------------------------------------------------------------
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>> Don't miss this year's exciting event. There's still time to save  
>>>> $100.
>>>> Use priority code J8TL2D2.
>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>> _______________________________________________
>>>> Psidev-ms-dev mailing list
>>>> Psi...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>     
>>>>         
>>> -------------------------------------------------------------------------
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
>>> Don't miss this year's exciting event. There's still time to save $100. 
>>> Use priority code J8TL2D2. 
>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>> _______________________________________________
>>> Psidev-ms-dev mailing list
>>> Psi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>
>>>   
>>>       
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
>> Don't miss this year's exciting event. There's still time to save $100. 
>> Use priority code J8TL2D2. 
>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>> _______________________________________________
>> Psidev-ms-dev mailing list
>> Psi...@li...
>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>     
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>   

Re: [Psidev-ms-dev] additional feature request: contiguous/noncontigous marker for m/z array data?

[David C. <dc...@ma...>]

 > What is the use case for this?
Some peak detection algorithms required the m/z values to be equally 
spaced, while others require equally spaced or fitting to a polynomial 
(for example quadratic TOF data).

For a database search engine, it's useful to know instantly whether it's 
a peak list or profile data.

Other examples of 'non contiguous' data are stitched PSD data where the 
segments aren't all on the same scale. And some exotic?? instruments may 
apply different calibration parameters to different parts of the 
spectrum, so values may initially appear to be quadratically spaced, and 
then they suddenly change to a different polynomial fit.

It can be surprisingly difficult to determine/reconstruct profile data 
- for example TOF data with zero intensity values thresholded out.

Any converter / writer should know what type of data is being written, 
so the more information for the consumer the better?

The following would certainly be useful for us:

centroided
profile
   linear
   quadratic
   higher order polynomial
   FT
   other
      contiguous
      missing zeros  (thresholded)
      multiple calibration coefficients
      multiple segments (stitched psd)

David

Pierre-Alain Binz wrote:
> I aggree too with the annotation of this thresholding as processing.
> Important for some LC-MS "image" analysis tools that can make a 
> algorithmic different analysis between those kind of data
> Pierre-Alain
> 
> Joshua Tasman wrote:
>> Cool.  So, no need to denote "choppy profile" vs "contiguous profile" at the binary array level?
>>
>> (Yay, seems like I can post again! :) )
>>
>> Josh
>>
>>
>> Matthew Chambers wrote:
>>   
>>> Yes, I agree too. Intensity thresholding should most definitely be 
>>> encoded in mzML. It is one of those basic kinds of data processing like 
>>> peak picking that we just have to support even though it's not strictly 
>>> speaking the raw data.
>>>
>>> -Matt
>>>
>>>
>>> Darren Kessner wrote:
>>>     
>>>> I see -- I definitely think it's useful to flag the data reduction.   
>>>> Perhaps this should be encoded as a cvParam in <dataProcessing>?
>>>>
>>>> Darren
>>>>
>>>>
>>>>
>>>> On Apr 16, 2008, at 10:15 AM, Joshua Tasman wrote:
>>>>
>>>>   
>>>>       
>>>>> Hi Darren,
>>>>>
>>>>> You did understand what I meant.  I've been working with the Agilent  
>>>>> MassHunter-format machines which give huge non-thresholded profile  
>>>>> arrays (at least currently), where I'd like to make note of data  
>>>>> reduction that might be applied at conversion time-- and make it  
>>>>> easier for the parsers to handle "choppy profile" data.  However, if  
>>>>> Thermo data already looks that way (I didn't realize), maybe it's  
>>>>> not necessary.
>>>>>
>>>>> I incorrectly believed that "profile" implied "no gaps in m/z  
>>>>> spacing".
>>>>>
>>>>> -Josh
>>>>>
>>>>>
>>>>> Darren Kessner wrote:
>>>>>     
>>>>>         
>>>>>> What is the use case for this?
>>>>>>
>>>>>> I think everyone has an intuitive understanding of "contiguous",  
>>>>>> but I
>>>>>> think it is a little more complicated to define properly.  The
>>>>>> intuitive definition is "regularly spaced with no missing data  
>>>>>> points".
>>>>>>
>>>>>> When accessing profile data in RAW files from Thermo FT instruments,
>>>>>> the spectra are thresholded, and this gives rise to the islands that
>>>>>> Josh mentioned during the conference call.  However, in FT data
>>>>>> without islands (e.g. if thresholding is disabled, or perhaps with
>>>>>> preprocessing to extract a single island or fill in missing data
>>>>>> points), the data points are still not regularly spaced in m/z.  (The
>>>>>> data points are regularly spaced in frequency space, but the  
>>>>>> frequency-
>>>>>>       
>>>>>>           
>>>>>>> m/z calibration function is non-linear).  Because the data points
>>>>>>>         
>>>>>>>             
>>>>>> are not regularly spaced, any processing must use some form of
>>>>>> interpolation, so I'm not sure how a "contiguous" flag will help.
>>>>>>
>>>>>>
>>>>>> Darren
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Apr 16, 2008, at 6:48 AM, Eric Deutsch wrote:
>>>>>>
>>>>>>       
>>>>>>           
>>>>>>> From: Joshua Tasman
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> There are cases where the m/z array data is contiguous (standard
>>>>>>> profile
>>>>>>> mode) vs noncontiguous (centroided, or thresholded profile).  Could
>>>>>>> we add
>>>>>>> an attribute to the binary array section (or above) describing
>>>>>>> this?  Or is
>>>>>>> this best left to the parsers to automatically determine?
>>>>>>>
>>>>>>> One tricky case that might not be quickly/easily scanned for:
>>>>>>> imagine a mode
>>>>>>> that preserves profile data points around detected peaks; so the
>>>>>>> first large
>>>>>>> # of data points might appear to be the same m/z delta.  Hence, a
>>>>>>> quick
>>>>>>> check by the parser might not get this correct.
>>>>>>>
>>>>>>> Thanks for considering,
>>>>>>>
>>>>>>> Josh
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -------------------------------------------------------------------------
>>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>>> Don't miss this year's exciting event. There's still time to save
>>>>>>> $100.
>>>>>>> Use priority code J8TL2D2.
>>>>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>>>>> _______________________________________________
>>>>>>> Psidev-ms-dev mailing list
>>>>>>> Psi...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>>>>         
>>>>>>>             
>>>>>> -------------------------------------------------------------------------
>>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>>> Don't miss this year's exciting event. There's still time to save  
>>>>>> $100.
>>>>>> Use priority code J8TL2D2.
>>>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>>>> _______________________________________________
>>>>>> Psidev-ms-dev mailing list
>>>>>> Psi...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>>>       
>>>>>>           
>>>>> -------------------------------------------------------------------------
>>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
>>>>> Don't miss this year's exciting event. There's still time to save  
>>>>> $100.
>>>>> Use priority code J8TL2D2.
>>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>>> _______________________________________________
>>>>> Psidev-ms-dev mailing list
>>>>> Psi...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>>     
>>>>>         
>>>> -------------------------------------------------------------------------
>>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
>>>> Don't miss this year's exciting event. There's still time to save $100. 
>>>> Use priority code J8TL2D2. 
>>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>>> _______________________________________________
>>>> Psidev-ms-dev mailing list
>>>> Psi...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>>
>>>>   
>>>>       
>>> -------------------------------------------------------------------------
>>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
>>> Don't miss this year's exciting event. There's still time to save $100. 
>>> Use priority code J8TL2D2. 
>>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>>> _______________________________________________
>>> Psidev-ms-dev mailing list
>>> Psi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>>     
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
>> Don't miss this year's exciting event. There's still time to save $100. 
>> Use priority code J8TL2D2. 
>> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
>> _______________________________________________
>> Psidev-ms-dev mailing list
>> Psi...@li...
>> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>>
>>   
> 
> ------------------------------------------------------------------------
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev

-- 
David Creasy
Matrix Science
64 Baker Street
London W1U 7GB, UK
Tel: +44 (0)20 7486 1050
Fax: +44 (0)20 7224 1344

dc...@ma...
http://www.matrixscience.com

Matrix Science Ltd. is registered in England and Wales
Company number 3533898