[Exist-open] Unwanted escaping of XIncludes in eXist-1.0rc

[<sju...@ko...>]

Hello all,

It seems Mr. Sourceforge is playing again - I sent the following e-=20
mail in the morning, and nothing has arrived on the list yet. So I =20
retry - I just received another e-mail from the list, so I hope Mr. =20
SF is working again. Apologies if you receive this e-mail twice.

While we were waiting for Mr. SF to do his job, we investigated the =20
issue further (i.e. we tried to understand what happened in the Java =20
code to find the cause), and the following item in the changelog =20
caught our attention:

- Added important safeguard in XmldbURI to ensure that non-ASCII =20
chars are always encoded

If this is relevant to the bug, it seems that the safeguarding was a =20
little overdone: it should not happen to XIncludes in documents =20
residing in eXist, when the expansion is done during serialisation =20
time of the including document. At that point, there's no way we can =20
catch the sudenly mal-transformed URI/xpointer expression, and back-=20
transform it to what it should have been.

Any feedback is much welcome.

Best regards,
Sjur

Videresendt melding:

> Fra: Sjur N=C3=B8rsteb=C3=B8 Moshagen <sju...@ko...>
> Dato: 28. juli 2006 10.06.31 GMT+03:00
> Til: eXist ml <exi...@li...>
> Emne: Unwanted escaping of XIncludes in eXist-1.0rc
>
> Hello all,
>
> I upgraded from the 20060313 snapshot to the recent 1.0rc =20
> yesterday. Most things seem to work fine, but the XInclude =20
> expansion isn't working like it used to. More precisely, in my =20
> case, fragment identifiers are %-escaped, making the references =20
> invalid, and thus killing the XIncludes. Example:
>
> <entry id=3D"33012">
>     <topicClass top=3D"fuolahas"/>
>     <entryref xml:lang=3D"sme">
>         <xi:include href=3D"terms-sme.xml#xpointer(//entry=20
> [@id=3D'=C4=8Dielgesah=C3=A1'])"/>
>     </entryref>
> </entry>
>
> This XInclude shows up in the exist.log as follows:
>
> 2006-07-28 09:35:09,262 [P1-9] DEBUG (XIncludeFilter.java =20
> [startElement]:169) - processing
> include ...
> 2006-07-28 09:35:09,264 [P1-9] DEBUG (XIncludeFilter.java =20
> [processXInclude]:213) - found href=3D"terms-sme.xml#xpointer(//entry=20=

> [@id=3D'=C4=8Dielgesah=C3=A1'])"
> 2006-07-28 09:35:09,282 [P1-9] DEBUG (XQuery.java [compile]:154) - =20
> Query diagnostics:
> [root-node]/descendant-or-self::entry[attribute::id =3D "%C4%=20
> 8Dielgesah%C3%A1"]
> 2006-07-28 09:35:09,283 [P1-9] DEBUG (XQuery.java [compile]:156) - =20
> Compilation took 16
> 2006-07-28 09:35:09,284 [P1-9] INFO  (XIncludeFilter.java =20
> [processXInclude]:305) - xpointe
> r query: [root-node]/descendant-or-self::entry[attribute::id =3D "%C4%=20=

> 8Dielgesah%C3%A1"]
> 2006-07-28 09:35:09,303 [P1-9] DEBUG (HTTPUtils.java =20
> [addLastModifiedHeader]:61) - mostRec
> entDocumentTime: 0
>
> This bug effectively shuts down our application - we rely heavily =20
> on XIncludes to avoid duplication of info.
>
> Nothing relevant has changed in our code as part of the upgrade, =20
> and as the log shows, the first look-up uses the correct UTF-8 =20
> string (without escaping), whereas the other two log entries =20
> display the escaped variant that causes the include to fail. In =20
> cases where the identifier consists of pure ASCII, the XInclude is =20
> resolved properly.
>
> I couldn't find any mentioning of this problem in the recent =20
> discussions about XIncludes, nor in the changelog - my apologies if =20=

> I overlooked something.
>
> Best regards,
> Sjur
>