[eXist-commits] [eXist-db/exist] 4e949b: [feature] Validate native XSDs against W3C XSD 1.1...

[Juri L. <no...@gi...>]

  Branch: refs/heads/develop
  Home:   https://github.com/eXist-db/exist
  Commit: 4e949b427719ab9ad8d8a87f5746904ae40493d5
      https://github.com/eXist-db/exist/commit/4e949b427719ab9ad8d8a87f5746904ae40493d5
  Author: duncdrum <d.p...@me...>
  Date:   2026-07-13 (Mon, 13 Jul 2026)

  Changed paths:
    M exist-core/pom.xml
    M exist-core/src/main/java/org/exist/util/SchemaVersion.java
    A exist-core/src/test/java/org/exist/util/SchemaVersionFixtureAuditTest.java
    M exist-core/src/test/java/org/exist/util/SchemaVersionTest.java
    A exist-core/src/test/resources/org/exist/validation/entities/XMLSchema.dtd
    M exist-core/src/test/resources/org/exist/validation/entities/XMLSchema.xsd
    A exist-core/src/test/resources/org/exist/validation/entities/datatypes.dtd
    M exist-core/src/test/resources/org/exist/validation/entities/xml.xsd
    M exist-distribution/src/main/config/collection.xconf.init
    A exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/XMLSchema.dtd
    M exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/XMLSchema.xsd
    A exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/datatypes.dtd
    M exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/xml.xsd
    M pom.xml
    A schema/GeneratedSchemaVersions.xml
    M schema/README.md
    M schema/collection.xconf.xsd
    M schema/expath-pkg-extensions/cxan.xsd
    M schema/expath-pkg-extensions/exist.xsd
    M schema/expath-pkg-extensions/repo.xsd
    M schema/expath-pkg.xsd
    A schema/generate-schema-version.xsl

  Log Message:
  -----------
  [feature] Validate native XSDs against W3C XSD 1.1 meta-schema; auto-generate SchemaVersion constants

Extend the Maven validate phase to check every schema/*.xsd against the
W3C XSD 1.1 meta-schema (upgrading the bundled XMLSchema.xsd/dtd to the
2009 XSD 1.1 revision); caught and removed five xsi:type="dcterms:W3CDTF"
appinfo annotations with no backing schema, bumping the affected schema
@version values. Wire a generate-sources execution that reads each
governed XSD's xs:schema/@version and emits SchemaVersion.java constants
so they can never drift from the schemas.

Closes #5541


  Commit: e50a34c23480bb3064de995403edbbac54887fe9
      https://github.com/eXist-db/exist/commit/e50a34c23480bb3064de995403edbbac54887fe9
  Author: duncdrum <d.p...@me...>
  Date:   2026-07-13 (Mon, 13 Jul 2026)

  Changed paths:
    M exist-core/src/main/java/org/exist/collections/IndexInfo.java
    M exist-core/src/main/java/org/exist/collections/MutableCollection.java
    M exist-core/src/main/java/org/exist/resolver/ResolverFactory.java
    M exist-core/src/main/java/org/exist/util/SaxonConfiguration.java
    M exist-core/src/main/java/org/exist/validation/Validator.java
    A exist-core/src/main/java/org/exist/validation/Xsd11SchemaDetection.java
    M exist-core/src/main/java/org/exist/xmlrpc/RpcConnection.java
    M exist-core/src/main/java/org/exist/xquery/functions/fn/transform/Transform.java
    M exist-core/src/main/java/org/exist/xquery/functions/fn/transform/URIResolution.java
    M exist-core/src/main/java/org/exist/xquery/functions/validation/GrammarTooling.java
    M exist-core/src/main/java/org/exist/xquery/functions/validation/Jaxp.java
    M exist-core/src/main/java/org/exist/xslt/XsltURIResolverHelper.java
    A exist-core/src/test/java/org/exist/validation/ValidatorXsd11Test.java
    A exist-core/src/test/java/org/exist/validation/Xsd11StoreTimeValidationTest.java
    M exist-core/src/test/java/xquery/xquery3/XQuery3Tests.java
    M exist-core/src/test/resources/org/exist/validation/catalog.xml
    A exist-core/src/test/resources/org/exist/validation/entities/transform-catalog-test-lib.xsl
    A exist-core/src/test/xquery/transform/catalog.xql

  Log Message:
  -----------
  [feature] Saxon catalog resolver + XSD 1.1 store-time validation pipeline

Wire the XML Catalog API into Saxon's XsltCompiler so xsl:import,
xsl:include, and document() URIs resolve via the eXist-db entity catalog
(Closes #350). Generalize MutableCollection's at-store-time validation to
route through a JAXP 1.1 SchemaFactory when the schema or its catalog-
resolved meta-schema requires XSD 1.1; extract the detection logic into
Xsd11SchemaDetection so validation:jaxp() and org.exist.validation.Validator
share the same probe-compile path. Suppress the spurious xmlresolver WARNING
for the non-existent "default" catalog entry.

Closes #5541


  Commit: 0fe1766a9665aee6a7bf94b7698f2a0b421a623e
      https://github.com/eXist-db/exist/commit/0fe1766a9665aee6a7bf94b7698f2a0b421a623e
  Author: duncdrum <d.p...@me...>
  Date:   2026-07-13 (Mon, 13 Jul 2026)

  Changed paths:
    M exist-core/src/main/java/org/exist/Namespaces.java
    M exist-core/src/main/java/org/exist/collections/MutableCollection.java

  Log Message:
  -----------
  [refactor] Move XSD 1.1 namespace constant to Namespaces; drop unused XML_SCHEMA_NS

XML_SCHEMA_NS was never read — callers already used XMLConstants directly.
XSD_1_1_NS is promoted to Namespaces.XSD_1_1_NS so it is available
codebase-wide without duplication.

Co-Authored-By: Claude Sonnet 4.6 <no...@an...>


  Commit: 2ffaea7136719ca511f3c0a7dfb3b1b2c2be43b5
      https://github.com/eXist-db/exist/commit/2ffaea7136719ca511f3c0a7dfb3b1b2c2be43b5
  Author: duncdrum <d.p...@me...>
  Date:   2026-07-13 (Mon, 13 Jul 2026)

  Changed paths:
    M exist-core/src/main/java/org/exist/collections/MutableCollection.java

  Log Message:
  -----------
  [ignore] Convert Xsd11LexicalHandlerForwarder to a record

Co-Authored-By: Claude Sonnet 4.6 <no...@an...>


  Commit: af2333f396651c7cb7ea2f957cb1eb6fb3d64dba
      https://github.com/eXist-db/exist/commit/af2333f396651c7cb7ea2f957cb1eb6fb3d64dba
  Author: duncdrum <d.p...@me...>
  Date:   2026-07-26 (Sun, 26 Jul 2026)

  Changed paths:
    M exist-core/src/main/java/org/exist/collections/MutableCollection.java
    A exist-core/src/main/java/org/exist/collections/Xsd11SchemaCache.java
    A exist-core/src/main/java/org/exist/collections/Xsd11ValidationHelper.java

  Log Message:
  -----------
  [refactor] Extract XSD 1.1 store-time validation infrastructure out of MutableCollection

MutableCollection carried a schema cache (plain ConcurrentHashMap) plus the
namespace-resolution/validation logic that uses it -- neither is Collection
state. Splits this into two package-private classes:

- Xsd11SchemaCache: the per-namespace "needs XSD 1.1?" cache, now backed by
  Caffeine for consistency with the codebase's other caches (no eviction
  policy, since the namespace set is finite and admin-controlled -- this was
  never a correctness concern, just a style/testability one).
- Xsd11ValidationHelper: resolveXsd11SchemaForNamespace, the dynamic-discovery
  schema holder, parseOrValidateXmlSource, validateWithXsd11Schema, and the
  Xsd11LexicalHandlerForwarder record -- store-time XSD 1.1 validation, not
  Collection bookkeeping.

MutableCollection keeps only its public clearXsd11SchemaByNamespaceCache()
facade (GrammarTooling.java's external call site) and a single delegating
call from storeXmlDocument(); 9 imports that only the moved code needed are
dropped. Behavior-preserving: all 63 tests across the validation/XSD11 suite
pass unchanged.

Addresses dizzzz's review comments on #6530/#6551.


  Commit: 38d2dd127e01322d34454abce9d45d45a0fa08ff
      https://github.com/eXist-db/exist/commit/38d2dd127e01322d34454abce9d45d45a0fa08ff
  Author: duncdrum <d.p...@me...>
  Date:   2026-07-26 (Sun, 26 Jul 2026)

  Changed paths:
    M schema/expath-pkg.xsd

  Log Message:
  -----------
  [bugfix] Bump schema/expath-pkg.xsd @version for the missed dcterms:created edit

4e949b4277 dropped xsi:type="dcterms:W3CDTF" from the dcterms:created
annotation in 5 sibling schema files and correctly bumped @version on 4 of
them (collection.xconf.xsd, cxan.xsd, exist.xsd, repo.xsd) -- this one was
missed, leaving expath-pkg.xsd's content changed but @version still 1.1.0.
CI's schema governance check (ci-schema-checks.yml, added in PR-A) catches
exactly this. Bumps to 1.1.1, a documentation/non-semantic patch per
schema/README.md's policy, matching the sibling files' bumps.

Verified locally against the real merge-base with upstream/develop:
mvn -N xml:transform@schema-governance passes, SchemaVersionSyncTest passes
unaffected (expath-pkg.xsd isn't one of the 5 SchemaVersion.java-tracked
schemas).


  Commit: 12f73861b1fcde7a6e70b6b0d645952498f528e4
      https://github.com/eXist-db/exist/commit/12f73861b1fcde7a6e70b6b0d645952498f528e4
  Author: duncdrum <d.p...@me...>
  Date:   2026-08-17 (Mon, 17 Aug 2026)

  Changed paths:
    M exist-core/src/main/java/org/exist/collections/Xsd11ValidationHelper.java

  Log Message:
  -----------
  [refactor] Drop redundant Optional wrapping in
`resolveXsd11SchemaForNamespace`

use a plain Schema local and wrap once, at the
`Xsd11SchemaCache.put()` call site where the cache's Optional-typed
value (needed because Caffeine can't cache null) actually matters.


  Commit: d6c45545e9014e16ec35412b1748321a6b8f93e6
      https://github.com/eXist-db/exist/commit/d6c45545e9014e16ec35412b1748321a6b8f93e6
  Author: Juri Leino <gi...@li...>
  Date:   2026-08-17 (Mon, 17 Aug 2026)

  Changed paths:
    M exist-core/pom.xml
    M exist-core/src/main/java/org/exist/Namespaces.java
    M exist-core/src/main/java/org/exist/collections/IndexInfo.java
    M exist-core/src/main/java/org/exist/collections/MutableCollection.java
    A exist-core/src/main/java/org/exist/collections/Xsd11SchemaCache.java
    A exist-core/src/main/java/org/exist/collections/Xsd11ValidationHelper.java
    M exist-core/src/main/java/org/exist/resolver/ResolverFactory.java
    M exist-core/src/main/java/org/exist/util/SaxonConfiguration.java
    M exist-core/src/main/java/org/exist/util/SchemaVersion.java
    M exist-core/src/main/java/org/exist/validation/Validator.java
    A exist-core/src/main/java/org/exist/validation/Xsd11SchemaDetection.java
    M exist-core/src/main/java/org/exist/xmlrpc/RpcConnection.java
    M exist-core/src/main/java/org/exist/xquery/functions/fn/transform/Transform.java
    M exist-core/src/main/java/org/exist/xquery/functions/fn/transform/URIResolution.java
    M exist-core/src/main/java/org/exist/xquery/functions/validation/GrammarTooling.java
    M exist-core/src/main/java/org/exist/xquery/functions/validation/Jaxp.java
    M exist-core/src/main/java/org/exist/xslt/XsltURIResolverHelper.java
    A exist-core/src/test/java/org/exist/util/SchemaVersionFixtureAuditTest.java
    M exist-core/src/test/java/org/exist/util/SchemaVersionTest.java
    A exist-core/src/test/java/org/exist/validation/ValidatorXsd11Test.java
    A exist-core/src/test/java/org/exist/validation/Xsd11StoreTimeValidationTest.java
    M exist-core/src/test/java/xquery/xquery3/XQuery3Tests.java
    M exist-core/src/test/resources/org/exist/validation/catalog.xml
    A exist-core/src/test/resources/org/exist/validation/entities/XMLSchema.dtd
    M exist-core/src/test/resources/org/exist/validation/entities/XMLSchema.xsd
    A exist-core/src/test/resources/org/exist/validation/entities/datatypes.dtd
    A exist-core/src/test/resources/org/exist/validation/entities/transform-catalog-test-lib.xsl
    M exist-core/src/test/resources/org/exist/validation/entities/xml.xsd
    A exist-core/src/test/xquery/transform/catalog.xql
    M exist-distribution/src/main/config/collection.xconf.init
    A exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/XMLSchema.dtd
    M exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/XMLSchema.xsd
    A exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/datatypes.dtd
    M exist-jetty-config/src/main/resources/webapp/WEB-INF/entities/xml.xsd
    M pom.xml
    A schema/GeneratedSchemaVersions.xml
    M schema/README.md
    M schema/collection.xconf.xsd
    M schema/expath-pkg-extensions/cxan.xsd
    M schema/expath-pkg-extensions/exist.xsd
    M schema/expath-pkg-extensions/repo.xsd
    M schema/expath-pkg.xsd
    A schema/generate-schema-version.xsl

  Log Message:
  -----------
  Merge pull request #6530 from duncdrum/dp-xsd11-validation

XSD 1.1 store-time document validation + Saxon catalog resolver


Compare: https://github.com/eXist-db/exist/compare/225680c2fb77...d6c45545e901

To unsubscribe from these emails, change your notification settings at https://github.com/eXist-db/exist/settings/notifications