Re: [Exist-open] Filenames with awkward characters in eXist-db?

[David B. <dj...@gm...>]

Dear exist-open (cc Christian, Peter),

Thank you, Christian, for this information, which I had not known about
previously. If eXist-db installs apps by first unzipping them onto the
server filesystem in a expathrepo directory (giving the OS a chance to
normalize the filenames), does this mean that whether those who install my
app get a composed or decomposed representation of a composite character in
a filename may depend on their OS? If that is the case, it may not matter
where the confusion of the two representations happens, since I need it to
be possible for users on MacOS and other OSs to install the app, and to be
able to address the files by name over REST using the same HTTP URIs.

In response to Peter's observation, although I can set the locale on my own
machine, those who install my app may not know how to set a locale, or be
(reasonably) unable or unwilling to change their setting. My locale is set
to en_US.UTF-8, and response headers on a REST return show
"application/xml; charset=UTF-8" as the content type, so I thought it
should have been able to handle either the composed or decomposed
representation of Cyrillic ё. "Should have been able" has been a leitmotif
in this thread, though, and the additional observations from Christian and
Peter seem to suggest that even should the eXist-db resource management
interfaces be updated to handle filenames with non-ASCII characters
robustly (as the WebDAV interface already seems to do), interacting with
the files by filename using the REST interface may face challenges that
originate outside eXist-db.

Best,

David

On Wed, Nov 25, 2020 at 11:39 PM Christian Wittern <cwi...@gm...>
wrote:

> Dear David,
>
> Just chiming in on this very specific point:
>
> On 26/11/2020 03.05, David Birnbaum wrote:
> > But for those who are curious: on my Mac, with the data directory left
> > with the default value /Users/djb/Library/Application
> > Support/org.exist, packages are unpacked
> > under /Users/djb/Library/Application Support/org.exist/expathrepo/, so
> > trying to install the xar file let me see whether ё was represented by
> > a single composite character or by a base followed by a combining
> > diacritic.
>
> The macOS file system will normalize characters to use decomposed forms
> of characters that are encoded as pre-composed characters in Unicode as
> a rule, so in this case the observed fact does not allow any conclusions
> regarding the handling within eXist or its I/O pipelines.  For that
> purpose, you would need to use a different OS.
>
> All the best,
>
> Christian
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>