psidev-ms-dev

Re: [Psidev-ms-dev] Empty mzML files

[John C. <chi...@um...>]

Perhaps specification development is different than application
development. But in application development if you are processing
lists of things, it is better to allow the empty list as input unless
you have a good reason not to. So I think it would be more appropriate
to ask, "why exclude the possibility for an empty index" rather than
"why would you want an empty index"? Do you have an answer for that,
what is being harmed by an empty index?

Though I don't feel it is the right question to ask, I can respond to
the question of why allow an empty index. There are definitely good
reasons. For instances, there are programs (outside of msconvert) that
take in indexed mzML files but not unindexed mzML files (I think one
of the quant programs in the TPP for instance). Also, suppose I tell
msconvert (or really any program that produces mzML files) that I want
to do some filtering and I specify I want the output as an indexed
mzML file, it would be a defect in that program if it produces an
empty unindexed mzML file.

If I were an analyst, manually running desktop software and inspecting
the outputs are each step, I would never do anything with empty mzML
files, I would get to that point and stop. But that is not what I do,
I develop pipelines on top of applications. If I have to write special
logic to account for handling empty mzML files differently, and so do
the application programmers who write the applications I wrap, that
ends up being a lot of logic added by many different people with no
gain. To me this is a sign of a defect in specification.

Thanks again for your time,
-John

> Hello John,
>
> Thanks for your input. I might be missing something, but I don't see
> much point in writing out the index if it doesn't contain any entries.
> Wouldn't it be better to just write the mzML file without index in cases
> where there are no spectra? Maybe you can ask the msconvert developers
> to allow for that option, if it is not already possible.
>
> Best Regards
>
> Fredrik

Re: [Psidev-ms-dev] Empty mzML files

[Matt C. <mat...@gm...>]

Hi Brian,

I'm moving this back to psidev-ms-dev because I want to wait and see how the specification 
discussion works out. I'm inclined to agree with John that there isn't a strong reason to disallow 
empty indexed mzML files. After a schema change, a 1.1.2 empty indexed mzML file submitted to a 
1.1.1 validator would fail. But it's only going to happen on an empty file, so it's not a big deal.

On the other hand, AFAICT, even in a "pipeline", ending up with an empty file (indexed or not) is 
pretty much an error condition that should abort the pipeline. The abort need not be an ugly one: it 
could simply report to the client that the result at step X was empty and nothing further could be 
done. This error condition can be detected either by getting an error code from the mzML-producing 
application which tried to write an empty indexed mzML (using the existing schema), or by examining 
the produced mzML to see if it's empty (using a revised schema). The former is easier, but only 
works for indexed mzMLs, since non-indexed mzMLs can already be empty and won't produce an error code.

I think we should move toward consistency in allowing emptiness. Either both indexed and non-indexed 
mzMLs should be allowed to be empty, or neither. That way applications can handle them uniformly. 
And I prefer changing the index schema instead of the core schema (even though I'm actually 
indifferent as to whether emptiness is valid or not).

-Matt


On 12/17/2012 12:11 PM, Brian Pratt wrote:
> It sounds like the proper proteowizard response is yet another msconvert argument to indicate what
> to do in the face of an empty output for an indexed file - write unindexed with a warning, write a
> technically invalid empty index with a warning, or fail.
>
> Currently there is the --noindex option to turn off indexing, how about we add
> --emptyindex=<force|none> to indicate that the user prefers an invalid empty index or none at all.
>   The default behavior will continue to be failure, but we'll tweak the error message to indicate
> how to work around it.  Not in love with that name "emptyindex", any better ideas?
>
> Or we could just emit an empty index with a warning.  I'm guessing that's actually the most useful
> thing we could do - most mzML readers would probably handle it just fine even if its technically
> invalid.
>
> The more I think about it, the more pragmatic the "just emit an empty index with a warning" option
> sounds.
>
> Brian
>
> On Sun, Dec 16, 2012 at 10:42 AM, John Chilton <chi...@um... <mailto:chi...@um...>> wrote:
>
>     Perhaps specification development is different than application
>     development. But in application development if you are processing
>     lists of things, it is better to allow the empty list as input unless
>     you have a good reason not to. So I think it would be more appropriate
>     to ask, "why exclude the possibility for an empty index" rather than
>     "why would you want an empty index"? Do you have an answer for that,
>     what is being harmed by an empty index?
>
>     Though I don't feel it is the right question to ask, I can respond to
>     the question of why allow an empty index. There are definitely good
>     reasons. For instances, there are programs (outside of msconvert) that
>     take in indexed mzML files but not unindexed mzML files (I think one
>     of the quant programs in the TPP for instance). Also, suppose I tell
>     msconvert (or really any program that produces mzML files) that I want
>     to do some filtering and I specify I want the output as an indexed
>     mzML file, it would be a defect in that program if it produces an
>     empty unindexed mzML file.
>
>     If I were an analyst, manually running desktop software and inspecting
>     the outputs are each step, I would never do anything with empty mzML
>     files, I would get to that point and stop. But that is not what I do,
>     I develop pipelines on top of applications. If I have to write special
>     logic to account for handling empty mzML files differently, and so do
>     the application programmers who write the applications I wrap, that
>     ends up being a lot of logic added by many different people with no
>     gain. To me this is a sign of a defect in specification.
>
>     Thanks again for your time,
>     -John
>
>      > Hello John,
>      >
>      > Thanks for your input. I might be missing something, but I don't see
>      > much point in writing out the index if it doesn't contain any entries.
>      > Wouldn't it be better to just write the mzML file without index in cases
>      > where there are no spectra? Maybe you can ask the msconvert developers
>      > to allow for that option, if it is not already possible.
>      >
>      > Best Regards
>      >
>      > Fredrik

Re: [Psidev-ms-dev] Empty mzML files

[Oliver K. <oli...@un...>]

On 17.12.2012, at 20:09, Matt Chambers <mat...@gm...> wrote:

> On the other hand, AFAICT, even in a "pipeline", ending up with an empty file (indexed or not) is 
> pretty much an error condition that should abort the pipeline. The abort need not be an ugly one: it 
I would disagree. You could have a pipeline extracting spectra matching certain criteria and in
some cases that might be a desired outcome to obtain no spectra (e.g., in quality control pipelines
if you want to pull out all spectra matching certain contaminants).

> I think we should move toward consistency in allowing emptiness. Either both indexed and non-indexed 
> mzMLs should be allowed to be empty, or neither. That way applications can handle them uniformly. 
> And I prefer changing the index schema instead of the core schema (even though I'm actually 
> indifferent as to whether emptiness is valid or not).
I totally agree with this.

Just my two cents,
  Oliver




---
Oliver Kohlbacher (oli...@un...)
Professor, Applied Bioinformatics, Center for Bioinformatics Tuebingen
 & Dept. of Computer Science, University of Tuebingen
Director, Quantitative Biology Center
phone: +49-7071-29-70457   http://KohlbacherLab.org 

vCard at: http://abi.inf.uni-tuebingen.de/People/kohlbach/vCard

Re: [Psidev-ms-dev] Empty mzML files

[Matt C. <mat...@gm...>]

On 12/17/2012 1:13 PM, Oliver Kohlbacher wrote:
> On 17.12.2012, at 20:09, Matt Chambers<mat...@gm...>  wrote:
>
>> >On the other hand, AFAICT, even in a "pipeline", ending up with an empty file (indexed or not) is
>> >pretty much an error condition that should abort the pipeline. The abort need not be an ugly one: it
> I would disagree. You could have a pipeline extracting spectra matching certain criteria and in
> some cases that might be a desired outcome to obtain no spectra (e.g., in quality control pipelines
> if you want to pull out all spectra matching certain contaminants).

I just meant that you can't pass the resulting file to any other program (except one that can ignore 
empty files or simply counts spectra). So either take the error code to mean "it was empty" or check 
the file yourself to detect the emptiness. Checking an error code is usually easier than checking an 
mzML file for emptiness, but I agree the extra logic in the pipeline is undesirable.

-Matt

Re: [Psidev-ms-dev] Empty mzML files

[Brian P. <bri...@in...>]

Nearer-term, though, I think it's useful for proteowizard to emit a
(technically invalid) empty index with a warning.  The file wouldn't
validate, but then most readers don't bother validating anyway for
performance reasons.  We could include a comment in the file explaining the
situation, I suppose.

It's true that this pushes the handling of the empty file error condition
downstream, but the point is that not everybody agrees that this *is* an
error  condition, and would actually view emptiness as a useful bit of
information downstream.  I don't get the sense that this is going to break
anybody's pipeline any worse than the current behavior does, but would in
fact be an improvement.

Brian

On Mon, Dec 17, 2012 at 11:18 AM, Matt Chambers
<mat...@gm...>wrote:

>
> On 12/17/2012 1:13 PM, Oliver Kohlbacher wrote:
> > On 17.12.2012, at 20:09, Matt Chambers<mat...@gm...>
>  wrote:
> >
> >> >On the other hand, AFAICT, even in a "pipeline", ending up with an
> empty file (indexed or not) is
> >> >pretty much an error condition that should abort the pipeline. The
> abort need not be an ugly one: it
>


> > I would disagree. You could have a pipeline extracting spectra matching
> certain criteria and in
> > some cases that might be a desired outcome to obtain no spectra (e.g.,
> in quality control pipelines
> > if you want to pull out all spectra matching certain contaminants).
>
> I just meant that you can't pass the resulting file to any other program
> (except one that can ignore
> empty files or simply counts spectra). So either take the error code to
> mean "it was empty" or check
> the file yourself to detect the emptiness. Checking an error code is
> usually easier than checking an
> mzML file for emptiness, but I agree the extra logic in the pipeline is
> undesirable.
>
> -Matt
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>

Re: [Psidev-ms-dev] Empty mzML files

[Fredrik L. <fre...@im...>]

Hi,

I agree there is certainly good use cases for mzML files without 
spectra, and yes, it makes sense to allow for empty index as well. 
However, schema updates are likely to cause problems and should be 
avoided if possible.  I am a bit concerned that updating the index 
schema will cause validating tools to fail if they are not updated to 
support index version 1.1.2. It would maybe cause less problems if 
Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2 
for empty ones, but that is maybe not such a great solution either. But 
on a an index schema update, all Mascot servers will need an update to 
support new indexed mzML files (also those with spectra). So, actually 
it might even be better from a practical point of view to allow pwiz to 
produce invalid 1.1.1 files with empty index, as Brian suggests, than 
updating to schema version 1.1.2 only. A validating tool would protest 
on receiving the file, but it could probably not use it either, so that 
could still be OK if the error message tells what's wrong.

Fredrik

Matt Chambers skrev 2012-12-17 20:09:
> Hi Brian,
>
> I'm moving this back to psidev-ms-dev because I want to wait and see how the specification
> discussion works out. I'm inclined to agree with John that there isn't a strong reason to disallow
> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed mzML file submitted to a
> 1.1.1 validator would fail. But it's only going to happen on an empty file, so it's not a big deal.
>

Re: [Psidev-ms-dev] Empty mzML files

[Matt C. <mat...@gm...>]

Can a 1.1.1 validating parser accept a non-empty 1.1.2 file? Then it would only fail on the empty 
files (which will actually be valid, but won't appear so until the parser is updated). I definitely 
prefer a long-term solution of consistent schema emptiness requirements.

-Matt


On 12/17/2012 2:21 PM, Fredrik Levander wrote:
> Hi,
>
> I agree there is certainly good use cases for mzML files without
> spectra, and yes, it makes sense to allow for empty index as well.
> However, schema updates are likely to cause problems and should be
> avoided if possible.  I am a bit concerned that updating the index
> schema will cause validating tools to fail if they are not updated to
> support index version 1.1.2. It would maybe cause less problems if
> Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2
> for empty ones, but that is maybe not such a great solution either. But
> on a an index schema update, all Mascot servers will need an update to
> support new indexed mzML files (also those with spectra). So, actually
> it might even be better from a practical point of view to allow pwiz to
> produce invalid 1.1.1 files with empty index, as Brian suggests, than
> updating to schema version 1.1.2 only. A validating tool would protest
> on receiving the file, but it could probably not use it either, so that
> could still be OK if the error message tells what's wrong.
>
> Fredrik
>
> Matt Chambers skrev 2012-12-17 20:09:
>> Hi Brian,
>>
>> I'm moving this back to psidev-ms-dev because I want to wait and see how the specification
>> discussion works out. I'm inclined to agree with John that there isn't a strong reason to disallow
>> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed mzML file submitted to a
>> 1.1.1 validator would fail. But it's only going to happen on an empty file, so it's not a big deal.
>>

Re: [Psidev-ms-dev] Empty mzML files

[Fredrik L. <fre...@im...>]

I just tried, and it actually worked fine to send an indexed file with 
mzML1.1.1_idx changed to mzML1.1.2_idx in the <indexedzmML> element 
through Mascot. There is no version attribute in the index, only in the 
mzML itself. So my worries were exaggerated, and it should probably be 
smooth to update the index schema to 1.1.2 and use it for all files.

Fredrik

Matt Chambers skrev 2012-12-17 21:28:
> Can a 1.1.1 validating parser accept a non-empty 1.1.2 file? Then it would only fail on the empty
> files (which will actually be valid, but won't appear so until the parser is updated). I definitely
> prefer a long-term solution of consistent schema emptiness requirements.
>
> -Matt
>
>
> On 12/17/2012 2:21 PM, Fredrik Levander wrote:
>> Hi,
>>
>> I agree there is certainly good use cases for mzML files without
>> spectra, and yes, it makes sense to allow for empty index as well.
>> However, schema updates are likely to cause problems and should be
>> avoided if possible.  I am a bit concerned that updating the index
>> schema will cause validating tools to fail if they are not updated to
>> support index version 1.1.2. It would maybe cause less problems if
>> Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2
>> for empty ones, but that is maybe not such a great solution either. But
>> on a an index schema update, all Mascot servers will need an update to
>> support new indexed mzML files (also those with spectra). So, actually
>> it might even be better from a practical point of view to allow pwiz to
>> produce invalid 1.1.1 files with empty index, as Brian suggests, than
>> updating to schema version 1.1.2 only. A validating tool would protest
>> on receiving the file, but it could probably not use it either, so that
>> could still be OK if the error message tells what's wrong.
>>
>> Fredrik
>>
>> Matt Chambers skrev 2012-12-17 20:09:
>>> Hi Brian,
>>>
>>> I'm moving this back to psidev-ms-dev because I want to wait and see how the specification
>>> discussion works out. I'm inclined to agree with John that there isn't a strong reason to disallow
>>> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed mzML file submitted to a
>>> 1.1.1 validator would fail. But it's only going to happen on an empty file, so it's not a big deal.
>>>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev

Re: [Psidev-ms-dev] Empty mzML files

[Brian P. <bri...@in...>]

So the right thing to do is write an empty index with warning for schema <
1.1.2, and without warning thereafter.

On Mon, Dec 17, 2012 at 12:59 PM, Fredrik Levander <
fre...@im...> wrote:

> I just tried, and it actually worked fine to send an indexed file with
> mzML1.1.1_idx changed to mzML1.1.2_idx in the <indexedzmML> element
> through Mascot. There is no version attribute in the index, only in the
> mzML itself. So my worries were exaggerated, and it should probably be
> smooth to update the index schema to 1.1.2 and use it for all files.
>
> Fredrik
>
> Matt Chambers skrev 2012-12-17 21:28:
> > Can a 1.1.1 validating parser accept a non-empty 1.1.2 file? Then it
> would only fail on the empty
> > files (which will actually be valid, but won't appear so until the
> parser is updated). I definitely
> > prefer a long-term solution of consistent schema emptiness requirements.
> >
> > -Matt
> >
> >
> > On 12/17/2012 2:21 PM, Fredrik Levander wrote:
> >> Hi,
> >>
> >> I agree there is certainly good use cases for mzML files without
> >> spectra, and yes, it makes sense to allow for empty index as well.
> >> However, schema updates are likely to cause problems and should be
> >> avoided if possible.  I am a bit concerned that updating the index
> >> schema will cause validating tools to fail if they are not updated to
> >> support index version 1.1.2. It would maybe cause less problems if
> >> Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2
> >> for empty ones, but that is maybe not such a great solution either. But
> >> on a an index schema update, all Mascot servers will need an update to
> >> support new indexed mzML files (also those with spectra). So, actually
> >> it might even be better from a practical point of view to allow pwiz to
> >> produce invalid 1.1.1 files with empty index, as Brian suggests, than
> >> updating to schema version 1.1.2 only. A validating tool would protest
> >> on receiving the file, but it could probably not use it either, so that
> >> could still be OK if the error message tells what's wrong.
> >>
> >> Fredrik
> >>
> >> Matt Chambers skrev 2012-12-17 20:09:
> >>> Hi Brian,
> >>>
> >>> I'm moving this back to psidev-ms-dev because I want to wait and see
> how the specification
> >>> discussion works out. I'm inclined to agree with John that there isn't
> a strong reason to disallow
> >>> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed
> mzML file submitted to a
> >>> 1.1.1 validator would fail. But it's only going to happen on an empty
> file, so it's not a big deal.
> >>>
> >
> ------------------------------------------------------------------------------
> > LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> > Remotely access PCs and mobile devices and provide instant support
> > Improve your efficiency, and focus on delivering more value-add services
> > Discover what IT Professionals Know. Rescue delivers
> > http://p.sf.net/sfu/logmein_12329d2d
> > _______________________________________________
> > Psidev-ms-dev mailing list
> > Psi...@li...
> > https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>

Re: [Psidev-ms-dev] Empty mzML files

[Eric D. <ede...@sy...>]

So it sounds like there is consensus that we update the index wrapper
schema to allow an empty index at version 1.1.2. I agree that this is the
best solution. Are there any objections to this?



If no objections emerge, I would propose to update the mzML wrapper schema
to 1.1.2 tomorrow (Tuesday). The primary mzML schema will remain unchanged
at 1.1. I will also update the documentation to reflect this update as well
as another pending documentation update.



Objections?  Comments?



Thanks,

Eric





*From:* Brian Pratt [mailto:bri...@in...]
*Sent:* Monday, December 17, 2012 1:03 PM
*To:* Mass spectrometry standard development;
pro...@li...
*Subject:* Re: [Psidev-ms-dev] Empty mzML files



So the right thing to do is write an empty index with warning for schema <
1.1.2, and without warning thereafter.

On Mon, Dec 17, 2012 at 12:59 PM, Fredrik Levander <
fre...@im...> wrote:

I just tried, and it actually worked fine to send an indexed file with
mzML1.1.1_idx changed to mzML1.1.2_idx in the <indexedzmML> element
through Mascot. There is no version attribute in the index, only in the
mzML itself. So my worries were exaggerated, and it should probably be
smooth to update the index schema to 1.1.2 and use it for all files.

Fredrik

Matt Chambers skrev 2012-12-17 21:28:

> Can a 1.1.1 validating parser accept a non-empty 1.1.2 file? Then it
would only fail on the empty
> files (which will actually be valid, but won't appear so until the parser
is updated). I definitely
> prefer a long-term solution of consistent schema emptiness requirements.
>
> -Matt
>
>
> On 12/17/2012 2:21 PM, Fredrik Levander wrote:
>> Hi,
>>
>> I agree there is certainly good use cases for mzML files without
>> spectra, and yes, it makes sense to allow for empty index as well.
>> However, schema updates are likely to cause problems and should be
>> avoided if possible.  I am a bit concerned that updating the index
>> schema will cause validating tools to fail if they are not updated to
>> support index version 1.1.2. It would maybe cause less problems if
>> Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2
>> for empty ones, but that is maybe not such a great solution either. But
>> on a an index schema update, all Mascot servers will need an update to
>> support new indexed mzML files (also those with spectra). So, actually
>> it might even be better from a practical point of view to allow pwiz to
>> produce invalid 1.1.1 files with empty index, as Brian suggests, than
>> updating to schema version 1.1.2 only. A validating tool would protest
>> on receiving the file, but it could probably not use it either, so that
>> could still be OK if the error message tells what's wrong.
>>
>> Fredrik
>>
>> Matt Chambers skrev 2012-12-17 20:09:
>>> Hi Brian,
>>>
>>> I'm moving this back to psidev-ms-dev because I want to wait and see
how the specification
>>> discussion works out. I'm inclined to agree with John that there isn't
a strong reason to disallow
>>> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed
mzML file submitted to a
>>> 1.1.1 validator would fail. But it's only going to happen on an empty
file, so it's not a big deal.
>>>
>
------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev


------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Psidev-ms-dev mailing list
Psi...@li...
https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev

Re: [Psidev-ms-dev] [proteowizard-developer] Empty mzML files

[Brian P. <bri...@in...>]

So is this what the tail end of such a file would look like?
   ....
    </run>
  </mzML>
  <indexList count="2">
    <index name="spectrum">
    </index>
    <index name="chromatogram">
    </index>
  </indexList>
  <indexListOffset>3960</indexListOffset>
  <fileChecksum>03a9be216becb05f58b9bd967ff521f5b9883154</fileChecksum>
</indexedmzML>

or would it be
   ....
    </run>
  </mzML>
  <indexList count="0">
  </indexList>
  <indexListOffset>3902</indexListOffset>
  <fileChecksum>e216bec03a9bb67ff521f5b98805f58b9bd93154</fileChecksum>
</indexedmzML>

?

I lean toward the former, but what would the revised schema dictate?.

Brian

On Mon, Dec 17, 2012 at 4:14 PM, Eric Deutsch
<ede...@sy...>wrote:

> So it sounds like there is consensus that we update the index wrapper
> schema to allow an empty index at version 1.1.2. I agree that this is the
> best solution. Are there any objections to this?
>
>
>
> If no objections emerge, I would propose to update the mzML wrapper schema
> to 1.1.2 tomorrow (Tuesday). The primary mzML schema will remain unchanged
> at 1.1. I will also update the documentation to reflect this update as well
> as another pending documentation update.
>
>
>
> Objections?  Comments?
>
>
>
> Thanks,
>
> Eric
>
>
>
>
>
> *From:* Brian Pratt [mailto:bri...@in...]
> *Sent:* Monday, December 17, 2012 1:03 PM
> *To:* Mass spectrometry standard development;
> pro...@li...
> *Subject:* Re: [Psidev-ms-dev] Empty mzML files
>
>
>
> So the right thing to do is write an empty index with warning for schema <
> 1.1.2, and without warning thereafter.
>
> On Mon, Dec 17, 2012 at 12:59 PM, Fredrik Levander <
> fre...@im...> wrote:
>
> I just tried, and it actually worked fine to send an indexed file with
> mzML1.1.1_idx changed to mzML1.1.2_idx in the <indexedzmML> element
> through Mascot. There is no version attribute in the index, only in the
> mzML itself. So my worries were exaggerated, and it should probably be
> smooth to update the index schema to 1.1.2 and use it for all files.
>
> Fredrik
>
> Matt Chambers skrev 2012-12-17 21:28:
>
> > Can a 1.1.1 validating parser accept a non-empty 1.1.2 file? Then it
> would only fail on the empty
> > files (which will actually be valid, but won't appear so until the
> parser is updated). I definitely
> > prefer a long-term solution of consistent schema emptiness requirements.
> >
> > -Matt
> >
> >
> > On 12/17/2012 2:21 PM, Fredrik Levander wrote:
> >> Hi,
> >>
> >> I agree there is certainly good use cases for mzML files without
> >> spectra, and yes, it makes sense to allow for empty index as well.
> >> However, schema updates are likely to cause problems and should be
> >> avoided if possible.  I am a bit concerned that updating the index
> >> schema will cause validating tools to fail if they are not updated to
> >> support index version 1.1.2. It would maybe cause less problems if
> >> Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2
> >> for empty ones, but that is maybe not such a great solution either. But
> >> on a an index schema update, all Mascot servers will need an update to
> >> support new indexed mzML files (also those with spectra). So, actually
> >> it might even be better from a practical point of view to allow pwiz to
> >> produce invalid 1.1.1 files with empty index, as Brian suggests, than
> >> updating to schema version 1.1.2 only. A validating tool would protest
> >> on receiving the file, but it could probably not use it either, so that
> >> could still be OK if the error message tells what's wrong.
> >>
> >> Fredrik
> >>
> >> Matt Chambers skrev 2012-12-17 20:09:
> >>> Hi Brian,
> >>>
> >>> I'm moving this back to psidev-ms-dev because I want to wait and see
> how the specification
> >>> discussion works out. I'm inclined to agree with John that there isn't
> a strong reason to disallow
> >>> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed
> mzML file submitted to a
> >>> 1.1.1 validator would fail. But it's only going to happen on an empty
> file, so it's not a big deal.
> >>>
> >
> ------------------------------------------------------------------------------
> > LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> > Remotely access PCs and mobile devices and provide instant support
> > Improve your efficiency, and focus on delivering more value-add services
> > Discover what IT Professionals Know. Rescue delivers
> > http://p.sf.net/sfu/logmein_12329d2d
> > _______________________________________________
> > Psidev-ms-dev mailing list
> > Psi...@li...
> > https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> proteowizard-developer mailing list
> pro...@li...
> https://lists.sourceforge.net/lists/listinfo/proteowizard-developer
>
>

Re: [Psidev-ms-dev] [proteowizard-developer] Empty mzML files

[Eric D. <ede...@sy...>]

I haven’t gotten to make the schema changes yet, got derailed on other
things, but still hope to get to it tonight. I would also advocate the
first option. I think we have a general policy for PSI formats “xxxxxList
elements shall not be empty”. I don’t know if it’s engraved anywhere, but
was a policy we tried to carry through. If there’s a xxxxxList element,
there better be something in it, or else, leave it out. So I would rather
implement it with an empty <index>. Clearly not an obvious conclusion, but
at least a precedent to guide the decision.



Additional comments?



Thanks,

Eric





*From:* Brian Pratt [mailto:bri...@in...]
*Sent:* Tuesday, December 18, 2012 10:11 AM
*To:* pro...@li...
*Cc:* Mass spectrometry standard development
*Subject:* Re: [Psidev-ms-dev] [proteowizard-developer] Empty mzML files



So is this what the tail end of such a file would look like?

   ....

    </run>

  </mzML>

  <indexList count="2">

    <index name="spectrum">

    </index>

    <index name="chromatogram">

    </index>

  </indexList>

  <indexListOffset>3960</indexListOffset>

  <fileChecksum>03a9be216becb05f58b9bd967ff521f5b9883154</fileChecksum>

</indexedmzML>



or would it be

   ....

    </run>

  </mzML>

  <indexList count="0">

  </indexList>

  <indexListOffset>3902</indexListOffset>

  <fileChecksum>e216bec03a9bb67ff521f5b98805f58b9bd93154</fileChecksum>

</indexedmzML>



?



I lean toward the former, but what would the revised schema dictate?.



Brian



On Mon, Dec 17, 2012 at 4:14 PM, Eric Deutsch <ede...@sy...>
wrote:

So it sounds like there is consensus that we update the index wrapper
schema to allow an empty index at version 1.1.2. I agree that this is the
best solution. Are there any objections to this?



If no objections emerge, I would propose to update the mzML wrapper schema
to 1.1.2 tomorrow (Tuesday). The primary mzML schema will remain unchanged
at 1.1. I will also update the documentation to reflect this update as well
as another pending documentation update.



Objections?  Comments?



Thanks,

Eric





*From:* Brian Pratt [mailto:bri...@in...]
*Sent:* Monday, December 17, 2012 1:03 PM
*To:* Mass spectrometry standard development;
pro...@li...
*Subject:* Re: [Psidev-ms-dev] Empty mzML files



So the right thing to do is write an empty index with warning for schema <
1.1.2, and without warning thereafter.

On Mon, Dec 17, 2012 at 12:59 PM, Fredrik Levander <
fre...@im...> wrote:

I just tried, and it actually worked fine to send an indexed file with
mzML1.1.1_idx changed to mzML1.1.2_idx in the <indexedzmML> element
through Mascot. There is no version attribute in the index, only in the
mzML itself. So my worries were exaggerated, and it should probably be
smooth to update the index schema to 1.1.2 and use it for all files.

Fredrik

Matt Chambers skrev 2012-12-17 21:28:

> Can a 1.1.1 validating parser accept a non-empty 1.1.2 file? Then it
would only fail on the empty
> files (which will actually be valid, but won't appear so until the parser
is updated). I definitely
> prefer a long-term solution of consistent schema emptiness requirements.
>
> -Matt
>
>
> On 12/17/2012 2:21 PM, Fredrik Levander wrote:
>> Hi,
>>
>> I agree there is certainly good use cases for mzML files without
>> spectra, and yes, it makes sense to allow for empty index as well.
>> However, schema updates are likely to cause problems and should be
>> avoided if possible.  I am a bit concerned that updating the index
>> schema will cause validating tools to fail if they are not updated to
>> support index version 1.1.2. It would maybe cause less problems if
>> Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2
>> for empty ones, but that is maybe not such a great solution either. But
>> on a an index schema update, all Mascot servers will need an update to
>> support new indexed mzML files (also those with spectra). So, actually
>> it might even be better from a practical point of view to allow pwiz to
>> produce invalid 1.1.1 files with empty index, as Brian suggests, than
>> updating to schema version 1.1.2 only. A validating tool would protest
>> on receiving the file, but it could probably not use it either, so that
>> could still be OK if the error message tells what's wrong.
>>
>> Fredrik
>>
>> Matt Chambers skrev 2012-12-17 20:09:
>>> Hi Brian,
>>>
>>> I'm moving this back to psidev-ms-dev because I want to wait and see
how the specification
>>> discussion works out. I'm inclined to agree with John that there isn't
a strong reason to disallow
>>> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed
mzML file submitted to a
>>> 1.1.1 validator would fail. But it's only going to happen on an empty
file, so it's not a big deal.
>>>
>
------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev


------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Psidev-ms-dev mailing list
Psi...@li...
https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev




------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
proteowizard-developer mailing list
pro...@li...
https://lists.sourceforge.net/lists/listinfo/proteowizard-developer

Re: [Psidev-ms-dev] [proteowizard-developer] Empty mzML files

[Brian P. <bri...@in...>]

The first option is already very nearly legal, except that the lower limit
for element count in <index> is unspecified and so defaults to 1.  So that
would be a pretty small change to the schema.

On Tue, Dec 18, 2012 at 5:30 PM, Eric Deutsch
<ede...@sy...>wrote:

> I haven’t gotten to make the schema changes yet, got derailed on other
> things, but still hope to get to it tonight. I would also advocate the
> first option. I think we have a general policy for PSI formats “xxxxxList
> elements shall not be empty”. I don’t know if it’s engraved anywhere, but
> was a policy we tried to carry through. If there’s a xxxxxList element,
> there better be something in it, or else, leave it out. So I would rather
> implement it with an empty <index>. Clearly not an obvious conclusion, but
> at least a precedent to guide the decision.
>
>
>
> Additional comments?
>
>
>
> Thanks,
>
> Eric
>
>
>
>
>
> *From:* Brian Pratt [mailto:bri...@in...]
> *Sent:* Tuesday, December 18, 2012 10:11 AM
> *To:* pro...@li...
> *Cc:* Mass spectrometry standard development
> *Subject:* Re: [Psidev-ms-dev] [proteowizard-developer] Empty mzML files
>
>
>
> So is this what the tail end of such a file would look like?
>
>    ....
>
>     </run>
>
>   </mzML>
>
>   <indexList count="2">
>
>     <index name="spectrum">
>
>     </index>
>
>     <index name="chromatogram">
>
>     </index>
>
>   </indexList>
>
>   <indexListOffset>3960</indexListOffset>
>
>   <fileChecksum>03a9be216becb05f58b9bd967ff521f5b9883154</fileChecksum>
>
> </indexedmzML>
>
>
>
> or would it be
>
>    ....
>
>     </run>
>
>   </mzML>
>
>   <indexList count="0">
>
>   </indexList>
>
>   <indexListOffset>3902</indexListOffset>
>
>   <fileChecksum>e216bec03a9bb67ff521f5b98805f58b9bd93154</fileChecksum>
>
> </indexedmzML>
>
>
>
> ?
>
>
>
> I lean toward the former, but what would the revised schema dictate?.
>
>
>
> Brian
>
>
>
> On Mon, Dec 17, 2012 at 4:14 PM, Eric Deutsch <ede...@sy...>
> wrote:
>
> So it sounds like there is consensus that we update the index wrapper
> schema to allow an empty index at version 1.1.2. I agree that this is the
> best solution. Are there any objections to this?
>
>
>
> If no objections emerge, I would propose to update the mzML wrapper schema
> to 1.1.2 tomorrow (Tuesday). The primary mzML schema will remain unchanged
> at 1.1. I will also update the documentation to reflect this update as well
> as another pending documentation update.
>
>
>
> Objections?  Comments?
>
>
>
> Thanks,
>
> Eric
>
>
>
>
>
> *From:* Brian Pratt [mailto:bri...@in...]
> *Sent:* Monday, December 17, 2012 1:03 PM
> *To:* Mass spectrometry standard development;
> pro...@li...
> *Subject:* Re: [Psidev-ms-dev] Empty mzML files
>
>
>
> So the right thing to do is write an empty index with warning for schema <
> 1.1.2, and without warning thereafter.
>
> On Mon, Dec 17, 2012 at 12:59 PM, Fredrik Levander <
> fre...@im...> wrote:
>
> I just tried, and it actually worked fine to send an indexed file with
> mzML1.1.1_idx changed to mzML1.1.2_idx in the <indexedzmML> element
> through Mascot. There is no version attribute in the index, only in the
> mzML itself. So my worries were exaggerated, and it should probably be
> smooth to update the index schema to 1.1.2 and use it for all files.
>
> Fredrik
>
> Matt Chambers skrev 2012-12-17 21:28:
>
> > Can a 1.1.1 validating parser accept a non-empty 1.1.2 file? Then it
> would only fail on the empty
> > files (which will actually be valid, but won't appear so until the
> parser is updated). I definitely
> > prefer a long-term solution of consistent schema emptiness requirements.
> >
> > -Matt
> >
> >
> > On 12/17/2012 2:21 PM, Fredrik Levander wrote:
> >> Hi,
> >>
> >> I agree there is certainly good use cases for mzML files without
> >> spectra, and yes, it makes sense to allow for empty index as well.
> >> However, schema updates are likely to cause problems and should be
> >> avoided if possible.  I am a bit concerned that updating the index
> >> schema will cause validating tools to fail if they are not updated to
> >> support index version 1.1.2. It would maybe cause less problems if
> >> Proteowizard continues to produce 1.1.1 for files with spectra and 1.1.2
> >> for empty ones, but that is maybe not such a great solution either. But
> >> on a an index schema update, all Mascot servers will need an update to
> >> support new indexed mzML files (also those with spectra). So, actually
> >> it might even be better from a practical point of view to allow pwiz to
> >> produce invalid 1.1.1 files with empty index, as Brian suggests, than
> >> updating to schema version 1.1.2 only. A validating tool would protest
> >> on receiving the file, but it could probably not use it either, so that
> >> could still be OK if the error message tells what's wrong.
> >>
> >> Fredrik
> >>
> >> Matt Chambers skrev 2012-12-17 20:09:
> >>> Hi Brian,
> >>>
> >>> I'm moving this back to psidev-ms-dev because I want to wait and see
> how the specification
> >>> discussion works out. I'm inclined to agree with John that there isn't
> a strong reason to disallow
> >>> empty indexed mzML files. After a schema change, a 1.1.2 empty indexed
> mzML file submitted to a
> >>> 1.1.1 validator would fail. But it's only going to happen on an empty
> file, so it's not a big deal.
> >>>
> >
> ------------------------------------------------------------------------------
> > LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> > Remotely access PCs and mobile devices and provide instant support
> > Improve your efficiency, and focus on delivering more value-add services
> > Discover what IT Professionals Know. Rescue delivers
> > http://p.sf.net/sfu/logmein_12329d2d
> > _______________________________________________
> > Psidev-ms-dev mailing list
> > Psi...@li...
> > https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Psidev-ms-dev mailing list
> Psi...@li...
> https://lists.sourceforge.net/lists/listinfo/psidev-ms-dev
>
>
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> proteowizard-developer mailing list
> pro...@li...
> https://lists.sourceforge.net/lists/listinfo/proteowizard-developer
>
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> proteowizard-developer mailing list
> pro...@li...
> https://lists.sourceforge.net/lists/listinfo/proteowizard-developer
>
>