#358 Clarify attribute namespace rules for packages

[Lucian Smith]

Recently, I was trying to update libsbml to do a better job validating attributes in SBML documents, particularly the ones for packages. As part of that process, we realized that the validation rules in the spec kind of contradicted many examples in the same specs, where people assumed that if the element name was in a 'default' namespace, that its attributes would be in that same default namespace. So, you could have either:

<groups:listOfGroups>
  <groups:group groups:id="g1"/>
</groups:listOfGroups>

that uses 'groups:id', but also:

<listOfGroups xmlns="[groups_uri]">
  <group id="g1"/>
</listOfGroups>

that uses just 'id'.

However! As it turns out, XML is weird in that attributes with no prefix are in no namespace, not the default namespace. And in fact, the default namespace has no bearing whatsoever on the legality or illegality of any attribute.

This means that if we want to allow both 'groups:id' as well as 'id', we have to allow them both in all situations. Sarah's investigations indicate that this rule would be relatively straightforward to implement in the schema, and my own investigations indicate that this rule would be relatively straightforward to implement in libsbml, too.

This means that we would allow the above two situations, but also allow:

<groups:listOfGroups>
  <groups:group id="g1"/>
</groups:listOfGroups>

and

<listOfGroups xmlns="[groups_uri]">
  <group groups:id="g1"/>
</listOfGroups>

At the same time, we would not allow prefixed 'core' attributes, so the following would be illegal:

 <core:species core:id="S1"/>

And the same would be true for the 'metaid' and 'SBO' attributes that everything inherits from SBase. (And, in l3v2, the 'id' and 'name' attributes.)

So! The upshot is that Sarah and I propose to add the following:

• A paragraph in the SBML core spec(s) summarizing the above, in the section where we talk about packages (in section 1.3)
• A paragraph template to add to package specifications, repeating that summary.
• A paragraph to add to package specifications in the 'validation rules' section, to clarify that when a validation rule talks about 'package:attname', it is also legal to use just 'attname'.

I'll draft some text and post in the comments.

(NOTE: package attributes that are added to non-package elements (i.e. 'fbc:charge') would still be required to be labelled with the appropriate prefix: just 'charge' would not be allowed.)

[Lucian Smith]

For way too much information on the above, see https://docs.google.com/document/d/1E5T2_E0tIlQ3JP-1jOtm3UIb2on0VVqvZoSeDy-wQoo/edit?ts=5c1818c7#

(If you want to trace our thoughts leading to the above conclusion.)

[Lucian Smith]

Here's the draft paragraph for the SBML core specs (l3v1 and l3v2). I ended up putting it at the end of section 1.4.3, right after the 'Additional notations for XML purposes' section:

\paragraph{XML Namespace}

For element names, XML has clear rules about how to declare their namespace. In typical SBML documents, the SBML namespace will be the default namespace for the document, and elements will therefore not need to be prefixed with the SBML namespace. However, it is perfectly legal to declare an SBML namespace and use it to explicitly place SBML elements in that namespace. (There may even be no default namespace in a given document, in which case the elements \emph{must} be prefixed.)

In contrast to element names, XML \emph{attribute} names are completely defined by the element in which they appear, and never have a 'default' namespace defined: the element itself declares whether any attributes should be defined with a namespace prefix or not.

As such, any attribute appearing in any SBML UML diagram in this document \emph{must} appear with \emph{no} namespace prefix, even if one is defined for SBML in the document.

The convention for SBML packages is to allow attributes that appear in their UML diagrams to \emph{either} be defined with no namespace prefix, \emph{or} to be defined with that package's namespace as a prefix, for any new element defined by that package. When a package extends an existing SBML element to have a new attribute, the convention is to \emph{require} that this attribute be prefixed with that package's namespace. Previously-released SBML packages did not make this explicit, but are assumed to follow this convention. As these packages undergo updates in the future, these rules will be made explicit.

[Lucian Smith]

Here's what I've drafted for the 'distrib' spec, at the end of the 'Namespace URI and other declarations necessary for using this package' section (3.2):

\paragraph{XML Namespace use}

For element names, XML has clear rules about how to declare and use namespaces. In typical SBML documents, the \distrib namespace will be defined as above, and elements will therefore need to be prefixed with \val{distrib:}.

In contrast to element names, XML \emph{attribute} names are completely defined by the element in which they appear, and never have a 'default' namespace defined: the element itself declares whether any attributes should be defined with a namespace prefix or not.

As is convention for SBML packages, any attribute that appears in a UML diagram in this specification may \emph{either} be defined with no namespace prefix, \emph{or} be defined with the \emph{distrib} namespace as a prefix. (No attributes are defined here as extentions of existing core SBML elements, and thus none of them are required to have the \emph{distrib} namespace as a prefix.)

[Lucian Smith]

And, finally, this is all I think we need to add to the validation rules appendix for packages, right after we talk about how the package prefix doesn't have to literally be 'distrib:', but can be any legal text:

\changed{Attributes from this package are listed in these rules as having the \val{distrib:} prefix, but as is convention for SBML packages, this prefix is optional.}

[Lucian Smith]

OK, I wasn't actually done: here's a bit to add to the l3v2 spec for its validation rules, too, just to be explicit about it:

\item \changed{Attributes listed in these rules have no namespace prefix, and must appear in SBML documents without any explicit namespace, even if the namespace is otherwise correct for \sbmlthree.}

(EDIT: I claimed that l3v2 was a package. It is not! Text now fixed.)

[Lucian Smith]

And, for an example from a specification that does extend core elements, here's what I have for multi. First, for the beginning, section 3.1:

[EDITED following Fengkai's suggestions:]

\paragraph{XML Namespace use}

For element names, XML has clear rules about how to declare and use namespaces. In typical SBML documents, the \multi namespace will be defined as above, and elements will therefore need to be prefixed with \val{multi:}.

In contrast to element names, XML \emph{attribute} names are completely defined by the element in which they appear, and never have a 'default' namespace defined: the element itself declares whether any attributes should be defined with a namespace prefix or not.

Any attribute that appears in a UML diagram in this specification may \emph{either} be defined with no namespace prefix, \emph{or} be defined with the \emph{multi} namespace as a prefix. For attributes added to SBML core elements (\Compartment, \Species, \SimpleSpeciesReference, and within \textbf{\class{MathML}}\xspace), the \emph{multi} namespace prefix is required.)

If \multi is used in an SBML Level~3 Version~2 document, \emph{any} element defined here may have either an \token{id} or a \token{name} with no prefix, as \token{id} and \token{name} were added to \SBase in that level/version.

And then in the Validation appendix:

\changed{Attributes from this package are listed in these rules as having the \val{multi:} prefix, but as is convention for SBML packages, this prefix is optional for elements that this specification defines, and is required for elements from SBML Core that this specification extends (\Compartment, \Species, \SimpleSpeciesReference, and within \textbf{\class{MathML}}\xspace).}

[Sarah Keating]

Just to be on record I agree with the above.

[Fengkai Zhang]

I agree with the above in principle. Suggested some changes and added some comments about the paragraph for multi in a google doc to make it flexible to edit: https://docs.google.com/document/d/1ZOXw1RQfJF6EcarQegwrJcZhBc-hqKl4l87pY_aEIpI

[Lucian Smith]

Edited the above to match your edits and some new edits of my own. Thanks!

[Matthias König]

I agree with the proposed changes. The clarifications are very good.

It would be great if libsbml is only generating one version of the XML (preferable the prefixed <package:xmlnode ....="">) version when writing SBML. In addition the version libsbml is generating should be in agreement with the examples in the spec. As far as I know this is the case.</package:xmlnode>

[Lucian Smith]

libsbml does indeed just generate a single version of the XML. The examples are sometimes a little mixed (as noted), but usually follow either the 'everything has a prefix' mode or 'nothing has a prefix' mode, which should be, I think, clear when you're reading it.

[Lucian Smith]

Having been approved by a majority of SBML editors, I am now marking this 'pending'. It is in SVN, has been added to http://sbml.org/Documents/Specifications/SBML_Level_3/Version_2/Core/Confirmed_issues_in_the_Level_3_Version_2_Core_Specification, and will be part of the forthcoming specification revision.

Further revisions are still welcomed, if Claudine or Sheriff (or another editor) wishes to make more suggestions, or has any objections.

[Lucian Smith]

Released!