exist-open — Open discussion

Re: [Exist-open] Problem accessing /apps/eXide/modules/search.xql.

[Nick S. <nsi...@nu...>]

 From what I can tell, all search terms are throwing the error.

On 11/25/20 11:02 AM, Joe Wicentowski wrote:
> Does any search term produce the error, or only some? Ideally, we’d 
> have a term that will produce the error on a stock system.
>
> On Wed, Nov 25, 2020 at 11:54 AM Nick Sincaglia 
> <nsi...@nu... <mailto:nsi...@nu...>> wrote:
>
>     I am using eXist-db version = eXist Version:     4.7.1
>
>     The steps to produce it are:
>
>     Once in eXide, I navigate to the "Edit" menu. Click the pull down
>     menu. Select "Find in file". Enter my search term in the "Search"
>     bar in the pop-up window. Select "XQuery" in the 'Type' pull down.
>     Click the "All" radio button. Then click the "Search" button in
>     the lower left hand corner of the the Pop-up window.
>
>     Nick
>
>     On 11/25/20 10:46 AM, Joe Wicentowski wrote:
>>     Hi Nick,
>>
>>     Which version of eXist and eXide?
>>
>>     Can you provide steps to reproduce the error on a stock
>>     installation of eXist?
>>
>>     Joe
>>
>>     On Wed, Nov 25, 2020 at 11:30 AM Nick Sincaglia
>>     <nsi...@nu... <mailto:nsi...@nu...>> wrote:
>>
>>         Occasionally, I come across this issue and I was wondering if
>>         anyone
>>         might have a suggestion on how to address it.
>>
>>         When I am using eXide, I try to use the "Find in files" pull
>>         down and I
>>         get the below error. I tried re-indexing my database. That
>>         does not
>>         work. I found /apps/eXide/modules/search.xql and added a
>>         white space to
>>         the bottom of the file and re-saved it. I have this helps
>>         sometimes when
>>         eXist-db can't locate a file.
>>
>>         Any suggestions on how to resolve this issue?
>>
>>
>>         HTTP ERROR 500
>>
>>         Problem accessing /apps/eXide/modules/search.xql. Reason:
>>
>>              Server Error
>>
>>         Caused by:
>>
>>         javax.servlet.ServletException:
>>         javax.servlet.ServletException: An error
>>         occurred while processing request to
>>         /apps/eXide/modules/search.xql: An
>>         error occurred: null
>>              at
>>         org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
>>              at
>>         org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
>>              at
>>         org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>>              at org.eclipse.jetty.server.Server.handle(Server.java:502)
>>              at
>>         org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
>>              at
>>         org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
>>              at
>>         org.eclipse.jetty.io
>>         <http://org.eclipse.jetty.io>.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
>>              at org.eclipse.jetty.io
>>         <http://org.eclipse.jetty.io>.FillInterest.fillable(FillInterest.java:103)
>>              at org.eclipse.jetty.io
>>         <http://org.eclipse.jetty.io>.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
>>              at
>>         org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
>>              at
>>         org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
>>              at
>>         org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
>>              at
>>         org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
>>              at
>>         org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
>>              at
>>         org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
>>              at
>>         org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
>>              at java.lang.Thread.run(Thread.java:748)
>>         Caused by: javax.servlet.ServletException: An error occurred
>>         while
>>         processing request to /apps/eXide/modules/search.xql: An
>>         error occurred:
>>         null
>>              at
>>         org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:370)
>>              at
>>         javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>              at
>>         org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
>>              at
>>         org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
>>              at
>>         de.betterform.agent.web.filter.XFormsFilter.doFilter(XFormsFilter.java:171)
>>              at
>>         org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
>>              at
>>         org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>>              at
>>         org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
>>              at
>>         org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
>>              at
>>         org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>>              at
>>         org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
>>              at
>>         org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
>>              at
>>         org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>>              at
>>         org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
>>              at
>>         org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>>              at
>>         org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>>              at
>>         org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
>>              at
>>         org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>>              at
>>         org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
>>              at
>>         org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>>              at
>>         org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
>>              at
>>         org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
>>              ... 16 more
>>
>>         -- 
>>         Nick Sincaglia
>>         President/Founder
>>         NueMeta, LLC
>>         Digital Media & Technology
>>         Phone: +1-630-303-7035
>>         nsi...@nu... <mailto:nsi...@nu...>
>>         http://www.nuemeta.com <http://www.nuemeta.com>
>>         Skype: nsincaglia
>>
>>
>>
>>         _______________________________________________
>>         Exist-open mailing list
>>         Exi...@li...
>>         <mailto:Exi...@li...>
>>         https://lists.sourceforge.net/lists/listinfo/exist-open
>>         <https://lists.sourceforge.net/lists/listinfo/exist-open>
>>
>
>     -- 
>     Nick Sincaglia
>     President/Founder
>     NueMeta, LLC
>     Digital Media & Technology
>     Phone: +1-630-303-7035
>     nsi...@nu...  <mailto:nsi...@nu...>  
>     http://www.nuemeta.com  <http://www.nuemeta.com>
>     Skype: nsincaglia
>
> -- 
> Sent from my iPhone

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[David B. <dj...@gm...>]

Dear exist-open (cc Peter, Eduard, Michael),

By way of context, for those just joining the thread, I've been installing
my files with unfriendly filenames (names that include percent signs and
square brackets, which would not be valid URI components unless they were
precent-escaped first) through the <oXygen/> WebDAV interface, and today I
tried installing a xar file of the app with the package manager. The
installation broke with the following error message (shown in the package
manager interface): "Invalid URI: Malformed escape pair at index 7:
verb-1a%[x]%-forms.xml". This is, indeed, an invalid URI, but it isn't an
invalid filename, and since package manager apparently uses URIs for
installation, I would have expected it to percent-escape the percent signs
and square brackets as part of the installation routine, instead of
assuming that the filename can be repurposed as URI without modification.

I tried installing the xar file in order to follow up on Peter's
suggestion, below, that I examine how ё is represented in a filename on the
file system after installation, and it revealed a relevant issue earlier in
my pipeline that I had overlooked. This paragraph is a digression in the
context of my exploration of eXist-db file management, so readers should
feel free to skip to the next paragraph. But for those who are curious: on
my Mac, with the data directory left with the default
value /Users/djb/Library/Application Support/org.exist, packages are
unpacked under /Users/djb/Library/Application
Support/org.exist/expathrepo/, so trying to install the xar file let me see
whether ё was represented by a single composite character or by a base
followed by a combining diacritic. Although my original data, which I used
earlier in my development pipeline to create the output filenames, has the
single composite character, the filename that gets unpacked during
installation has the two-character base+diacritic representation. I don't
think this discovery fully explains the REST issue that I reported, since I
had tried accessing the file with both the one-character and the
two-character representations in REST and received an error in both cases.
But this experiment did lead to the discovery that the two-character
representation in the filename seems to be created early in the pipeline
(that is, it gets into the xar file because it was in the xml file when I
added it to the eXist-db app), so that I read data with the one-character
representation at the beginning of my project, long before eXist-db comes
into the picture; use it to construct a filename; and wind up with a
filename that has the two-character representation, which then gets passed
along to my eXist-db app. That isn't an eXist-db issue, obviously, and I
would want to track it down and deal with it in order to run a more
controlled and orderly test of how the eXist-db REST interface deals with
the file.

Meanwhile, though, that the package manager makes the (apparently
undocumented, or am I reading carelessly?) assumption that filenames can be
used to compose URIs without percent-encoding them first is a deal-breaker.
Specifically, if package manager were not a requirement, I could grumble a
bit about the limitations of the Java admin client and eXide file manager
and then work around them by using the <oXygen/> WebDAV interface. But
that's a developer's perspective, and users of my app need to be able to
install it with package manager. This discovery about filename restrictions
within package manager, then, seems to take me back to the beginning, that
is, to the unwelcome conclusion that this is a losing battle, and I need to
give up on my original hope of using self-documenting filenames and instead
distort my filenames by avoiding the characters that pose challenges for
the eXist-db file management interfaces.

I understand why those who developed the Java admin client, eXide file
manager, package manager, and perhaps also the REST interface might
reasonably not have anticipated that eXist-db users would have a need for
such unusual filenames, and might therefore either 1) not have been aware
of the issue (and accidentally not created tests for it) or 2) knew about
it but concluded that nobody would have this need, and that the problem was
therefore only theoretical (that is, they may have thought it reasonable to
decide not to fix it). Over the past couple of weeks I've learned a lot
that I didn't know previously about eXist-db file management, and alongside
this knowledge gained I think the takeaway, given that package-manager
functionality is a strict requirement, is that I should give up and change
my filenames. I would like to see better filename ~ URI round-tripping
within the various eXist-db file management interfaces, but I do not have
the programming skills to write that code myself, and I appreciate that my
filename needs 1) are an edge case that most users won't care about, and 2)
have a work-around in that I can change my filenames, which means that
fixing this behavior is, for understandable reasons, unlikely to be a high
priority. I am grateful for the help and advice that I've received on this
list during my explorations, and especially the generous contributions by
Peter, Eduard, and Michael.

Sincerely,

David

On Tue, Nov 24, 2020 at 5:21 PM Hungerburg 13 <pc...@my...> wrote:

> Am 24.11.20 um 22:41 schrieb David Birnbaum:
> > Dear exist-open (cc Peter, Eduard),
> >
> > Thank you, Peter, for the follow-up. I managed to upload a file called
> > "verb-1a%[x]%-forms.xml" into my eXist-db instance by using the
> > <oXygen/> interface, and when I tried:
> >
> >
> http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/verb-1a%25%5Bx%5D%25-forms.xml
> >
> > in Chrome, Firefox, and curl it returned the resource.
>
> Cool, that would say, that webdav and rest interfaces are compatible, at
> least in regard to URI special characters within the ASCII range.
>
> > But the error I
> > reported (in both browsers and curl) with "verb-1a, ё.xml" is still
> > there. When I try to retrieve "verb-1a, ё.xml" with:q >
>
> http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/verb-1a%2C%20%D1%91-forms.xml
>
> unicode letter ё is UTF-8 two bytes: 0xD1 0x91 - looks legit - When I
> modify the little java prog from the previous mail, it will be echoed as
> "ё", not escaped.
>
> I cant help out any further, perhaps to debug further, would be good to
> upload "ё.txt" file, and then look how it will show in the local
> filsystem on the server?
>
> Peter
>
> PS: Do not do any of this on a live system!
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] Problem accessing /apps/eXide/modules/search.xql.

[Joe W. <jo...@gm...>]

Does any search term produce the error, or only some? Ideally, we’d have a
term that will produce the error on a stock system.

On Wed, Nov 25, 2020 at 11:54 AM Nick Sincaglia <nsi...@nu...>
wrote:

> I am using eXist-db version = eXist Version:     4.7.1
>
> The steps to produce it are:
>
> Once in eXide, I navigate to the "Edit" menu. Click the pull down menu.
> Select "Find in file". Enter my search term in the "Search" bar in the
> pop-up window. Select "XQuery" in the 'Type' pull down. Click the "All"
> radio button. Then click the "Search" button in the lower left hand corner
> of the the Pop-up window.
>
> Nick
>
> On 11/25/20 10:46 AM, Joe Wicentowski wrote:
>
> Hi Nick,
>
> Which version of eXist and eXide?
>
> Can you provide steps to reproduce the error on a stock installation of
> eXist?
>
> Joe
>
> On Wed, Nov 25, 2020 at 11:30 AM Nick Sincaglia <nsi...@nu...>
> wrote:
>
>> Occasionally, I come across this issue and I was wondering if anyone
>> might have a suggestion on how to address it.
>>
>> When I am using eXide, I try to use the "Find in files" pull down and I
>> get the below error. I tried re-indexing my database. That does not
>> work. I found /apps/eXide/modules/search.xql and added a white space to
>> the bottom of the file and re-saved it. I have this helps sometimes when
>> eXist-db can't locate a file.
>>
>> Any suggestions on how to resolve this issue?
>>
>>
>> HTTP ERROR 500
>>
>> Problem accessing /apps/eXide/modules/search.xql. Reason:
>>
>>      Server Error
>>
>> Caused by:
>>
>> javax.servlet.ServletException: javax.servlet.ServletException: An error
>> occurred while processing request to /apps/eXide/modules/search.xql: An
>> error occurred: null
>>      at
>>
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
>>      at
>>
>> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
>>      at
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>>      at org.eclipse.jetty.server.Server.handle(Server.java:502)
>>      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
>>      at
>>
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
>>      at
>> org.eclipse.jetty.io
>> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
>>      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
>>      at org.eclipse.jetty.io
>> .ChannelEndPoint$2.run(ChannelEndPoint.java:118)
>>      at
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
>>      at
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
>>      at
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
>>      at
>>
>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
>>      at
>>
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
>>      at
>>
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
>>      at
>>
>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
>>      at java.lang.Thread.run(Thread.java:748)
>> Caused by: javax.servlet.ServletException: An error occurred while
>> processing request to /apps/eXide/modules/search.xql: An error occurred:
>> null
>>      at
>>
>> org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:370)
>>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>      at
>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
>>      at
>>
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
>>      at
>>
>> de.betterform.agent.web.filter.XFormsFilter.doFilter(XFormsFilter.java:171)
>>      at
>>
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
>>      at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>>      at
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
>>      at
>>
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
>>      at
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>>      at
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
>>      at
>>
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
>>      at
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>>      at
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
>>      at
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>>      at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>>      at
>>
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
>>      at
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>>      at
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
>>      at
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>>      at
>>
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
>>      at
>>
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
>>      ... 16 more
>>
>> --
>> Nick Sincaglia
>> President/Founder
>> NueMeta, LLC
>> Digital Media & Technology
>> Phone: +1-630-303-7035
>> nsi...@nu...
>> http://www.nuemeta.com
>> Skype: nsincaglia
>>
>>
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>
> --
> Nick Sincaglia
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-...@nu... http://www.nuemeta.com
> Skype: nsincaglia
>
> --
Sent from my iPhone

Re: [Exist-open] Problem accessing /apps/eXide/modules/search.xql.

[Nick S. <nsi...@nu...>]

I am using eXist-db version = eXist Version:     4.7.1

The steps to produce it are:

Once in eXide, I navigate to the "Edit" menu. Click the pull down menu. 
Select "Find in file". Enter my search term in the "Search" bar in the 
pop-up window. Select "XQuery" in the 'Type' pull down. Click the "All" 
radio button. Then click the "Search" button in the lower left hand 
corner of the the Pop-up window.

Nick

On 11/25/20 10:46 AM, Joe Wicentowski wrote:
> Hi Nick,
>
> Which version of eXist and eXide?
>
> Can you provide steps to reproduce the error on a stock installation 
> of eXist?
>
> Joe
>
> On Wed, Nov 25, 2020 at 11:30 AM Nick Sincaglia 
> <nsi...@nu... <mailto:nsi...@nu...>> wrote:
>
>     Occasionally, I come across this issue and I was wondering if anyone
>     might have a suggestion on how to address it.
>
>     When I am using eXide, I try to use the "Find in files" pull down
>     and I
>     get the below error. I tried re-indexing my database. That does not
>     work. I found /apps/eXide/modules/search.xql and added a white
>     space to
>     the bottom of the file and re-saved it. I have this helps
>     sometimes when
>     eXist-db can't locate a file.
>
>     Any suggestions on how to resolve this issue?
>
>
>     HTTP ERROR 500
>
>     Problem accessing /apps/eXide/modules/search.xql. Reason:
>
>          Server Error
>
>     Caused by:
>
>     javax.servlet.ServletException: javax.servlet.ServletException: An
>     error
>     occurred while processing request to
>     /apps/eXide/modules/search.xql: An
>     error occurred: null
>          at
>     org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
>          at
>     org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
>          at
>     org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>          at org.eclipse.jetty.server.Server.handle(Server.java:502)
>          at
>     org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
>          at
>     org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
>          at
>     org.eclipse.jetty.io
>     <http://org.eclipse.jetty.io>.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
>          at org.eclipse.jetty.io
>     <http://org.eclipse.jetty.io>.FillInterest.fillable(FillInterest.java:103)
>          at org.eclipse.jetty.io
>     <http://org.eclipse.jetty.io>.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
>          at
>     org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
>          at
>     org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
>          at
>     org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
>          at
>     org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
>          at
>     org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
>          at
>     org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
>          at
>     org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
>          at java.lang.Thread.run(Thread.java:748)
>     Caused by: javax.servlet.ServletException: An error occurred while
>     processing request to /apps/eXide/modules/search.xql: An error
>     occurred:
>     null
>          at
>     org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:370)
>          at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>          at
>     org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
>          at
>     org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
>          at
>     de.betterform.agent.web.filter.XFormsFilter.doFilter(XFormsFilter.java:171)
>          at
>     org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
>          at
>     org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>          at
>     org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
>          at
>     org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
>          at
>     org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>          at
>     org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
>          at
>     org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
>          at
>     org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>          at
>     org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
>          at
>     org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>          at
>     org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>          at
>     org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
>          at
>     org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>          at
>     org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
>          at
>     org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>          at
>     org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
>          at
>     org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
>          ... 16 more
>
>     -- 
>     Nick Sincaglia
>     President/Founder
>     NueMeta, LLC
>     Digital Media & Technology
>     Phone: +1-630-303-7035
>     nsi...@nu... <mailto:nsi...@nu...>
>     http://www.nuemeta.com <http://www.nuemeta.com>
>     Skype: nsincaglia
>
>
>
>     _______________________________________________
>     Exist-open mailing list
>     Exi...@li...
>     <mailto:Exi...@li...>
>     https://lists.sourceforge.net/lists/listinfo/exist-open
>     <https://lists.sourceforge.net/lists/listinfo/exist-open>
>

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] Problem accessing /apps/eXide/modules/search.xql.

[Joe W. <jo...@gm...>]

Hi Nick,

Which version of eXist and eXide?

Can you provide steps to reproduce the error on a stock installation of
eXist?

Joe

On Wed, Nov 25, 2020 at 11:30 AM Nick Sincaglia <nsi...@nu...>
wrote:

> Occasionally, I come across this issue and I was wondering if anyone
> might have a suggestion on how to address it.
>
> When I am using eXide, I try to use the "Find in files" pull down and I
> get the below error. I tried re-indexing my database. That does not
> work. I found /apps/eXide/modules/search.xql and added a white space to
> the bottom of the file and re-saved it. I have this helps sometimes when
> eXist-db can't locate a file.
>
> Any suggestions on how to resolve this issue?
>
>
> HTTP ERROR 500
>
> Problem accessing /apps/eXide/modules/search.xql. Reason:
>
>      Server Error
>
> Caused by:
>
> javax.servlet.ServletException: javax.servlet.ServletException: An error
> occurred while processing request to /apps/eXide/modules/search.xql: An
> error occurred: null
>      at
>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
>      at
>
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
>      at
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>      at org.eclipse.jetty.server.Server.handle(Server.java:502)
>      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
>      at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
>      at
> org.eclipse.jetty.io
> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
>      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
>      at org.eclipse.jetty.io
> .ChannelEndPoint$2.run(ChannelEndPoint.java:118)
>      at
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
>      at
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
>      at
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
>      at
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
>      at
>
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
>      at
>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
>      at
>
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
>      at java.lang.Thread.run(Thread.java:748)
> Caused by: javax.servlet.ServletException: An error occurred while
> processing request to /apps/eXide/modules/search.xql: An error occurred:
> null
>      at
>
> org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:370)
>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>      at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
>      at
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
>      at
> de.betterform.agent.web.filter.XFormsFilter.doFilter(XFormsFilter.java:171)
>      at
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
>      at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
>      at
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
>      at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
>      at
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>      at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
>      at
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
>      at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>      at
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
>      at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>      at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
>      at
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
>      at
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>      at
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
>      at
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
>      at
>
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
>      at
>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
>      ... 16 more
>
> --
> Nick Sincaglia
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-630-303-7035
> nsi...@nu...
> http://www.nuemeta.com
> Skype: nsincaglia
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

[Exist-open] Problem accessing /apps/eXide/modules/search.xql.

[Nick S. <nsi...@nu...>]

Occasionally, I come across this issue and I was wondering if anyone 
might have a suggestion on how to address it.

When I am using eXide, I try to use the "Find in files" pull down and I 
get the below error. I tried re-indexing my database. That does not 
work. I found /apps/eXide/modules/search.xql and added a white space to 
the bottom of the file and re-saved it. I have this helps sometimes when 
eXist-db can't locate a file.

Any suggestions on how to resolve this issue?


HTTP ERROR 500

Problem accessing /apps/eXide/modules/search.xql. Reason:

     Server Error

Caused by:

javax.servlet.ServletException: javax.servlet.ServletException: An error 
occurred while processing request to /apps/eXide/modules/search.xql: An 
error occurred: null
     at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
     at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
     at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
     at org.eclipse.jetty.server.Server.handle(Server.java:502)
     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
     at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
     at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
     at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
     at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
     at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
     at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
     at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
     at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
     at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
     at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
     at java.lang.Thread.run(Thread.java:748)
Caused by: javax.servlet.ServletException: An error occurred while 
processing request to /apps/eXide/modules/search.xql: An error occurred: 
null
     at 
org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:370)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
     at 
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
     at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
     at 
de.betterform.agent.web.filter.XFormsFilter.doFilter(XFormsFilter.java:171)
     at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
     at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
     at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
     at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
     at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
     at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
     at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
     at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
     at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
     at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
     at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
     at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
     at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
     at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
     at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
     at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
     at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
     ... 16 more

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] Programmatically call the view.xql module

[Michael W. <wes...@ja...>]

Hi Alisdair,

I would do everything in XQuery. No need for XSLT. If all of the
recipes are in a uniform HTML structure, such as the recipe wrapped in a
<div id="recipe">...</div>, that makes it very easy to grab with an XPath.
xmldb:store(...) then puts it into your eXist database. Depending on how
well the recipe pages are structured, you can extract ingredients and other
metadata for storing in metadata of your own definition. Pretty much
anything that can be done with XSLT can be done with XQuery/XPath.

Take care.

2020年11月25日(水) 16:26 Jean-Paul Rehr <re...@gm...>:

> Dear Alisdair, is there a reason not to use eXist's triggers?
>
> https://exist-db.org/exist/apps/doc/triggers
>
> ie. store the exterior source in a specific collection -> triggers XSLT
> transformation and stores new result (it seems to me any templates you are
> using can be rewritten into the transformation Xquery itself and avoid
> going through view at all)?
>
> This might allow finer control of each step, particularly when things go
> sideways?
>
> Best,
> JPR
>
>
>
> On Wed, Nov 25, 2020 at 7:28 AM Alasdair Dougall <
> ala...@gm...> wrote:
>
>> Hi All,
>>
>> My wife's recipe website has recipes that are mainly static with some
>> dynamic sections.
>>
>> In effect, I can crawl the site to get the pages, but I would rather
>> trigger a xquery on saving a document within given directories to generate
>> html files and to store them in the appropriate directory.  So, is it
>> possible to run a query that would produce similar output to:
>>
>> <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
>>     <forward url="{$exist:controller}/xquery/{$pagetype}.xql">
>>         <set-attribute name="indexkey" value="{$exist:resource}"/>
>>          <set-attribute name="xslt.input" value="model"/>
>>          <set-attribute name="xslt.stylesheet"
>> value="{concat($exist:root,$exist:controller,"/templates/",$pagetype,".xslt")}"/>
>>          <set-attribute name="title"
>> value="{request:get-attribute('title')}"/>
>>          <set-attribute name="keywords"
>> value="{request:get-attribute('keywords')}"/>
>>      </forward>
>>   <view>
>>     <forward servlet="XSLTServlet">
>>        <set-attribute name="xslt.uri" value="{request:get-uri()}"/>
>>        <clear-attribute name="xslt.input"/>
>>     </forward>
>>     <forward url="{$exist:controller}/modules/view.xql"/>
>>   </view>
>>   <error-handler>
>> .....
>>   </error-handler>
>> </dispatch>
>>
>> So the sequence of calls are:
>>
>> 1.  Call pagetype.xql as in recipe.xql
>> 2.  Transform withXSLTServlet using a forward to pagetype.xslt, i.e.
>> recipe.xslt
>> 3.  Call view to expand templating entities
>>
>> Thanks in advance,
>>
>> Alasdair
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] Programmatically call the view.xql module

[Jean-Paul R. <re...@gm...>]

Dear Alisdair, is there a reason not to use eXist's triggers?

https://exist-db.org/exist/apps/doc/triggers

ie. store the exterior source in a specific collection -> triggers XSLT
transformation and stores new result (it seems to me any templates you are
using can be rewritten into the transformation Xquery itself and avoid
going through view at all)?

This might allow finer control of each step, particularly when things go
sideways?

Best,
JPR



On Wed, Nov 25, 2020 at 7:28 AM Alasdair Dougall <ala...@gm...>
wrote:

> Hi All,
>
> My wife's recipe website has recipes that are mainly static with some
> dynamic sections.
>
> In effect, I can crawl the site to get the pages, but I would rather
> trigger a xquery on saving a document within given directories to generate
> html files and to store them in the appropriate directory.  So, is it
> possible to run a query that would produce similar output to:
>
> <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
>     <forward url="{$exist:controller}/xquery/{$pagetype}.xql">
>         <set-attribute name="indexkey" value="{$exist:resource}"/>
>          <set-attribute name="xslt.input" value="model"/>
>          <set-attribute name="xslt.stylesheet"
> value="{concat($exist:root,$exist:controller,"/templates/",$pagetype,".xslt")}"/>
>          <set-attribute name="title"
> value="{request:get-attribute('title')}"/>
>          <set-attribute name="keywords"
> value="{request:get-attribute('keywords')}"/>
>      </forward>
>   <view>
>     <forward servlet="XSLTServlet">
>        <set-attribute name="xslt.uri" value="{request:get-uri()}"/>
>        <clear-attribute name="xslt.input"/>
>     </forward>
>     <forward url="{$exist:controller}/modules/view.xql"/>
>   </view>
>   <error-handler>
> .....
>   </error-handler>
> </dispatch>
>
> So the sequence of calls are:
>
> 1.  Call pagetype.xql as in recipe.xql
> 2.  Transform withXSLTServlet using a forward to pagetype.xslt, i.e.
> recipe.xslt
> 3.  Call view to expand templating entities
>
> Thanks in advance,
>
> Alasdair
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

[Exist-open] Programmatically call the view.xql module

[Alasdair D. <ala...@gm...>]

Hi All,

My wife's recipe website has recipes that are mainly static with some
dynamic sections.

In effect, I can crawl the site to get the pages, but I would rather
trigger a xquery on saving a document within given directories to generate
html files and to store them in the appropriate directory.  So, is it
possible to run a query that would produce similar output to:

<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
    <forward url="{$exist:controller}/xquery/{$pagetype}.xql">
        <set-attribute name="indexkey" value="{$exist:resource}"/>
         <set-attribute name="xslt.input" value="model"/>
         <set-attribute name="xslt.stylesheet"
value="{concat($exist:root,$exist:controller,"/templates/",$pagetype,".xslt")}"/>
         <set-attribute name="title"
value="{request:get-attribute('title')}"/>
         <set-attribute name="keywords"
value="{request:get-attribute('keywords')}"/>
     </forward>
  <view>
    <forward servlet="XSLTServlet">
       <set-attribute name="xslt.uri" value="{request:get-uri()}"/>
       <clear-attribute name="xslt.input"/>
    </forward>
    <forward url="{$exist:controller}/modules/view.xql"/>
  </view>
  <error-handler>
.....
  </error-handler>
</dispatch>

So the sequence of calls are:

1.  Call pagetype.xql as in recipe.xql
2.  Transform withXSLTServlet using a forward to pagetype.xslt, i.e.
recipe.xslt
3.  Call view to expand templating entities

Thanks in advance,

Alasdair

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Hungerburg 13 <pc...@my...>]

Am 24.11.20 um 22:41 schrieb David Birnbaum:
> Dear exist-open (cc Peter, Eduard),
> 
> Thank you, Peter, for the follow-up. I managed to upload a file called 
> "verb-1a%[x]%-forms.xml" into my eXist-db instance by using the 
> <oXygen/> interface, and when I tried:
> 
>     http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/verb-1a%25%5Bx%5D%25-forms.xml
> 
> in Chrome, Firefox, and curl it returned the resource.

Cool, that would say, that webdav and rest interfaces are compatible, at 
least in regard to URI special characters within the ASCII range.

> But the error I 
> reported (in both browsers and curl) with "verb-1a, ё.xml" is still 
> there. When I try to retrieve "verb-1a, ё.xml" with:q > 
http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/verb-1a%2C%20%D1%91-forms.xml

unicode letter ё is UTF-8 two bytes: 0xD1 0x91 - looks legit - When I 
modify the little java prog from the previous mail, it will be echoed as 
"ё", not escaped.

I cant help out any further, perhaps to debug further, would be good to 
upload "ё.txt" file, and then look how it will show in the local 
filsystem on the server?

Peter

PS: Do not do any of this on a live system!

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[David B. <dj...@gm...>]

Dear exist-open (cc Peter, Eduard),

Thank you, Peter, for the follow-up. I managed to upload a file called
"verb-1a%[x]%-forms.xml" into my eXist-db instance by using the <oXygen/>
interface, and when I tried:

http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/verb-1a%25%5Bx%5D%25-forms.xml


in Chrome, Firefox, and curl it returned the resource. But the error I
reported (in both browsers and curl) with "verb-1a, ё.xml" is still there.
When I try to retrieve "verb-1a, ё.xml" with:

http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/verb-1a%2C%20%D1%91-forms.xml


I get the error I reported earlier:

HTTP ERROR 404 Document /db/apps/cz/xml/by-form/verb-1a,%20?-forms.xml not
found
URI: /exist/rest/db/apps/cz/xml/by-form/verb-1a%2C%20%D1%91-forms.xml
STATUS: 404
MESSAGE: Document /db/apps/cz/xml/by-form/verb-1a,%20?-forms.xml not found
SERVLET: EXistServlet

The filename shows up when I ask for a collection listing with:

http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/


which returns (see the highlighted resource name):

<exist:result xmlns:exist="http://exist.sourceforge.net/NS/exist">

<exist:collection name="/db/apps/cz/xml/by-form"
created="2020-10-07T22:59:35.777-04:00" owner="admin" group="dba"
permissions="rwxr-xr-x">

<exist:resource name="prep-forms.xml"
created="2020-10-10T15:32:59.576-04:00"
last-modified="2020-10-23T12:10:57.875-04:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="pron-adj-forms.xml"
created="2020-10-15T23:13:02.842-04:00"
last-modified="2020-11-24T11:42:56.2-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="num-adj-forms.xml"
created="2020-10-25T10:36:07.497-04:00"
last-modified="2020-10-25T10:36:07.497-04:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="num-forms.xml"
created="2020-10-18T16:15:38.569-04:00"
last-modified="2020-10-23T11:58:07.018-04:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="pron-forms.xml"
created="2020-10-11T21:10:36.175-04:00"
last-modified="2020-10-23T12:10:57.825-04:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="adj-split-forms.xml"
created="2020-10-07T22:59:35.778-04:00"
last-modified="2020-11-24T11:38:55.904-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="adv-forms.xml"
created="2020-10-12T13:21:31.984-04:00"
last-modified="2020-11-24T11:42:56.414-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="paren-forms.xml"
created="2020-10-13T14:17:20.883-04:00"
last-modified="2020-11-24T11:42:56.05-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="part-forms.xml"
created="2020-10-13T14:18:25.164-04:00"
last-modified="2020-11-24T11:42:56.025-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="interj-forms.xml"
created="2020-10-13T14:29:52.801-04:00"
last-modified="2020-11-24T11:42:56.119-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="pred-forms.xml"
created="2020-10-13T15:59:57.391-04:00"
last-modified="2020-11-24T11:42:56.084-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="conj-forms.xml"
created="2020-10-09T21:29:26.751-04:00"
last-modified="2020-10-23T12:10:58.329-04:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="verb-1a-forms.xml"
created="2020-11-24T11:10:10.567-05:00"
last-modified="2020-11-24T11:10:10.567-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="*verb-1a%2C%20%D1%91-forms.xml*"
created="2020-11-24T11:11:25.374-05:00"
last-modified="2020-11-24T11:11:25.374-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="verb-1a%25%5Bx%5D%25-forms.xml"
created="2020-11-24T11:11:26.474-05:00"
last-modified="2020-11-24T11:11:26.473-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="verb-1a%257%25-forms.xml"
created="2020-11-24T11:11:27.729-05:00"
last-modified="2020-11-24T11:11:27.729-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="verb-1a%25x%25-forms.xml"
created="2020-11-24T11:11:27.803-05:00"
last-modified="2020-11-24T11:11:27.803-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

<exist:resource name="verb-2a-forms.xml"
created="2020-11-24T11:11:28.903-05:00"
last-modified="2020-11-24T11:11:28.903-05:00" owner="admin" group="dba"
permissions="rw-r--r--"/>

</exist:collection>

</exist:result>


I understand from the PS, below, that perhaps the issue is that the
browsers and curl manage escaping differently than the eXist-db REST
interface, but the browser interface error message, above, returns the
correct percent-encoded filename. If I try the same thing without doing my
own encoding, that is, use the literal filename "verb-1a, ё.xml" in the
browser address bar, I get the same error message, including the same
(percent-encoded) representation of the URI. No matter which spelling I
enter in the address bar (with ё or with %D1%91), the browser address bar
displays that part as ё when it returns the error message, but in the HTTP
ERROR and MESSAGE lines it shows a question mark instead of the Cyrillic
character.

I wondered whether the fact that Cyrillic ё (U+0451) could be encoded as
composed of a base character е (U+0435) followed by a combining diaeresis
(U+0308) could be the problem, so I tried decomposing it myself in my
browser address bar, but it still failed:

HTTP ERROR 404 Document /db/apps/cz/xml/by-form/verb-1a,%20??.xml not found
URI: /exist/rest/db/apps/cz/xml/by-form/verb-1a,%20%D0%B5%CC%88.xml
STATUS: 404
MESSAGE: Document /db/apps/cz/xml/by-form/verb-1a,%20??.xml not found
SERVLET: EXistServlet

There are two question marks this time because %D0%B5 represents the base
letter е and %CC%88 represents the combining diaeresis. I expected this one
to fail, since when I created the file I used the single combined ё
character (U+0451), and when it shows up in the collection listing, above,
it shows the single character..

Best,

David

On Tue, Nov 24, 2020 at 3:56 PM Peter <pc...@my...> wrote:

> Hello David, Eduard, looks like today I replied to a private message,
> here the gist of it:
>
> > What I am saying is that exist-db uses this constrructor
>
> Not everywhere, see
>
> https://github.com/eXist-db/exist/blob/10809646b5fc48195185b3399e084fb11e1d7e37/exist-core/src/main/java/org/exist/http/servlets/EXistServlet.java#L205
>
> I wrote that "hack" five years ago, credit got lost sometime, so it
> seems, but it is still there.
>
> The REST interface should therefore cope with them awkward characters?
>
> @David, would you make certain? Beware, what you get displayed in a
> directory listing might not be what has to be sent to the server - web
> browsers do escaping for transport for you, you might end up double
> escaping, curl does not, you might up not escaping!
>
> Peter
>
> PS: @David - It is not that easy to shift the blame around. The problem
> is in fact partly due to "home grown" conversions, but is very much also
> in the Java 2EE spec, in how eXist-db gets passed its arguments from the
> jetty container, which really is an abomination.
>
>
>
> Am 24.11.20 um 19:20 schrieb David Birnbaum:
> > Dear exist-open (cc Eduard, Peter),
> >
> > Have I understood correctly from the conversation below that eXist-db
> > uses a broken Java constructor to convert filenames to URIs, that is, a
> > constructor that incorrectly raises an error when operating on filenames
> > that contain percent signs and square brackets, even though those
> > characters can be percent-escaped and the percent-escaped versions can
> > be used in URIs? If that is the case, this would appear to be a bug in
> > eXist-db—perhaps a bug inherited from a faulty Java resource, but one
> > that could be fixed within eXist-db resource management interfaces by
> > shunning the faulty resource in favor of a (if necessary home-grown)
> > compliant filename-to-URI conversion routine. Or have I misunderstood
> > something about how filename-to-URI mapping is supposed to work? I did
> > consult https://tools.ietf.org/html/rfc3986#section-2, which seems to
> > tell me that filenames that contain percent signs and square brackets
> > can be used as URIs by percent-encoding those (and other reserved)
> > characters.
> >
> > Best,
> >
> > David
> >
> >
> > On Tue, Nov 24, 2020 at 1:16 AM Eduard Drenth <ed...@fr...
> > <mailto:ed...@fr...>> wrote:
> >
> >     What I am saying is that exist-db uses this constrructor
> >
> >     -----Original Message-----
> >     *From*: Hungerburg 13 <pc...@my...
> >     <mailto:Hungerburg%2013%20%3c...@my...%3e>>
> >     *To*: exi...@li...
> >     <mailto:exi...@li...>
> >     *Subject*: Re: [Exist-open] Filenames with awkward characters in
> >     eXist-db?
> >     *Date*: Mon, 23 Nov 2020 23:27:04 +0100
> >
> >     Am 23.11.20 um 22:08 schrieb Eduard Drenth:
> >
> >>     Another followup on this one:
> >>
> >>     Under the hood java.net.URI is used, a URI with either % or brackets
> >>     isn't valid, try:
> >>
> >>     new URI("test%percent.xml");
> >>     new URI("test[with]brackets.xml");
> >>
> >
> >     Not so quick! You should use another constructor, to work around such
> >
> >     awkward characters, see attached java source - beware the path string
> >
> >     then must start with a slash.
> >
> >
> >     The error message is somehow misleading though. Perhaps the four
> >
> >     argument constructor just can do a better guess at what is wanted,
> URIs
> >
> >     are quite a complicated beast.
> >
> >
> >     Peter
>

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Hungerburg 13 <pc...@my...>]

Hello David, Eduard, looks like today I replied to a private message, 
here the gist of it:

> What I am saying is that exist-db uses this constrructor

Not everywhere, see 
https://github.com/eXist-db/exist/blob/10809646b5fc48195185b3399e084fb11e1d7e37/exist-core/src/main/java/org/exist/http/servlets/EXistServlet.java#L205

I wrote that "hack" five years ago, credit got lost sometime, so it 
seems, but it is still there.

The REST interface should therefore cope with them awkward characters?

@David, would you make certain? Beware, what you get displayed in a 
directory listing might not be what has to be sent to the server - web 
browsers do escaping for transport for you, you might end up double 
escaping, curl does not, you might up not escaping!

Peter

PS: @David - It is not that easy to shift the blame around. The problem 
is in fact partly due to "home grown" conversions, but is very much also 
in the Java 2EE spec, in how eXist-db gets passed its arguments from the 
jetty container, which really is an abomination.



Am 24.11.20 um 19:20 schrieb David Birnbaum:
> Dear exist-open (cc Eduard, Peter),
> 
> Have I understood correctly from the conversation below that eXist-db 
> uses a broken Java constructor to convert filenames to URIs, that is, a 
> constructor that incorrectly raises an error when operating on filenames 
> that contain percent signs and square brackets, even though those 
> characters can be percent-escaped and the percent-escaped versions can 
> be used in URIs? If that is the case, this would appear to be a bug in 
> eXist-db—perhaps a bug inherited from a faulty Java resource, but one 
> that could be fixed within eXist-db resource management interfaces by 
> shunning the faulty resource in favor of a (if necessary home-grown) 
> compliant filename-to-URI conversion routine. Or have I misunderstood 
> something about how filename-to-URI mapping is supposed to work? I did 
> consult https://tools.ietf.org/html/rfc3986#section-2, which seems to 
> tell me that filenames that contain percent signs and square brackets 
> can be used as URIs by percent-encoding those (and other reserved) 
> characters.
> 
> Best,
> 
> David
> 
> 
> On Tue, Nov 24, 2020 at 1:16 AM Eduard Drenth <ed...@fr... 
> <mailto:ed...@fr...>> wrote:
> 
>     What I am saying is that exist-db uses this constrructor
> 
>     -----Original Message-----
>     *From*: Hungerburg 13 <pc...@my...
>     <mailto:Hungerburg%2013%20%3c...@my...%3e>>
>     *To*: exi...@li...
>     <mailto:exi...@li...>
>     *Subject*: Re: [Exist-open] Filenames with awkward characters in
>     eXist-db?
>     *Date*: Mon, 23 Nov 2020 23:27:04 +0100
> 
>     Am 23.11.20 um 22:08 schrieb Eduard Drenth:
> 
>>     Another followup on this one:
>>
>>     Under the hood java.net.URI is used, a URI with either % or brackets
>>     isn't valid, try:
>>
>>     new URI("test%percent.xml");
>>     new URI("test[with]brackets.xml");
>>
> 
>     Not so quick! You should use another constructor, to work around such
> 
>     awkward characters, see attached java source - beware the path string
> 
>     then must start with a slash.
> 
> 
>     The error message is somehow misleading though. Perhaps the four
> 
>     argument constructor just can do a better guess at what is wanted, URIs
> 
>     are quite a complicated beast.
> 
> 
>     Peter

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[David B. <dj...@gm...>]

Dear exist-open (cc Eduard, Peter),

Have I understood correctly from the conversation below that eXist-db uses
a broken Java constructor to convert filenames to URIs, that is, a
constructor that incorrectly raises an error when operating on filenames
that contain percent signs and square brackets, even though those
characters can be percent-escaped and the percent-escaped versions can be
used in URIs? If that is the case, this would appear to be a bug in
eXist-db—perhaps a bug inherited from a faulty Java resource, but one that
could be fixed within eXist-db resource management interfaces by shunning
the faulty resource in favor of a (if necessary home-grown) compliant
filename-to-URI conversion routine. Or have I misunderstood something about
how filename-to-URI mapping is supposed to work? I did consult
https://tools.ietf.org/html/rfc3986#section-2, which seems to tell me that
filenames that contain percent signs and square brackets can be used as
URIs by percent-encoding those (and other reserved) characters.

Best,

David


On Tue, Nov 24, 2020 at 1:16 AM Eduard Drenth <ed...@fr...>
wrote:

> What I am saying is that exist-db uses this constrructor
>
> -----Original Message-----
> *From*: Hungerburg 13 <pc...@my...
> <Hungerburg%2013%20%3c...@my...%3e>>
> *To*: exi...@li...
> *Subject*: Re: [Exist-open] Filenames with awkward characters in eXist-db?
> *Date*: Mon, 23 Nov 2020 23:27:04 +0100
>
> Am 23.11.20 um 22:08 schrieb Eduard Drenth:
>
> Another followup on this one:
>
>
> Under the hood java.net.URI is used, a URI with either % or brackets
>
> isn't valid, try:
>
>
> new URI("test%percent.xml");
>
> new URI("test[with]brackets.xml");
>
>
>
> Not so quick! You should use another constructor, to work around such
>
> awkward characters, see attached java source - beware the path string
>
> then must start with a slash.
>
>
> The error message is somehow misleading though. Perhaps the four
>
> argument constructor just can do a better guess at what is wanted, URIs
>
> are quite a complicated beast.
>
>
> Peter
>
> _______________________________________________
>
> Exist-open mailing list
>
> Exi...@li...
>
>
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> --
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Eduard D. <ed...@fr...>]

What I am saying is that exist-db uses this constrructor

-----Original Message-----
From: Hungerburg 13 <pc...@my...<mailto:Hungerburg%2013%20%3c...@my...%3e>>
To: exi...@li...<mailto:exi...@li...>
Subject: Re: [Exist-open] Filenames with awkward characters in eXist-db?
Date: Mon, 23 Nov 2020 23:27:04 +0100


Am 23.11.20 um 22:08 schrieb Eduard Drenth:

Another followup on this one:


Under the hood java.net.URI is used, a URI with either % or brackets

isn't valid, try:


new URI("test%percent.xml");

new URI("test[with]brackets.xml");



Not so quick! You should use another constructor, to work around such

awkward characters, see attached java source - beware the path string

then must start with a slash.


The error message is somehow misleading though. Perhaps the four

argument constructor just can do a better guess at what is wanted, URIs

are quite a complicated beast.


Peter

_______________________________________________

Exist-open mailing list

<mailto:Exi...@li...>

Exi...@li...


<https://lists.sourceforge.net/lists/listinfo/exist-open>

https://lists.sourceforge.net/lists/listinfo/exist-open


--

Eduard Drenth, Software Architekt

ed...@fr...<mailto:ed...@fr...>

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Hungerburg 13 <pc...@my...>]

Am 23.11.20 um 22:08 schrieb Eduard Drenth:
> Another followup on this one:
> 
> Under the hood java.net.URI is used, a URI with either % or brackets 
> isn't valid, try:
> 
> new URI("test%percent.xml");
> new URI("test[with]brackets.xml");
> 

Not so quick! You should use another constructor, to work around such 
awkward characters, see attached java source - beware the path string 
then must start with a slash.

The error message is somehow misleading though. Perhaps the four 
argument constructor just can do a better guess at what is wanted, URIs 
are quite a complicated beast.

Peter

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Eduard D. <ed...@fr...>]

Another followup on this one:
Under the hood java.net.URI is used, a URI with either % or brackets
isn't valid, try:
new URI("test%percent.xml");
new URI("test[with]brackets.xml");
-----Original Message-----From: Eduard Drenth <
ed...@fr...>To: David Birnbaum <dj...@gm...>, 
exi...@li... <exi...@li...>Cc:
Michael Westbay <wes...@ja...>Subject: Re: [Exist-
open] Filenames with awkward characters in eXist-db?Date: Mon, 23 Nov
2020 19:07:38 +0100
Of some help perhaps here (have yet to write docs for exist-db):
https://search.maven.org/search?q=a:exist-db-addons containing a
DataSyncTask for exist-db that enables you to separate editing workflow
on a filesystem from publishing worklfow in exist.
Example config:
<job class="org.fryske_akademy.exist.jobs.DataSyncTask" type="system"
period="10" repeat="0" >
    <parameter name="collection"
value="xmldb:exist:///db/apps/teidictjson/data"/>
    <parameter name="datadir" value="/data"/>
</job>
<job class="org.fryske_akademy.exist.jobs.DataSyncTaskCron"
type="system" cron-trigger="0 0 2 ? * *" >
    <parameter name="collection"
value="xmldb:exist:///db/apps/teidictjson/data"/>
    <parameter name="datadir" value="/data"/>
</job>

Bye, Eduard
-----Original Message-----From: David Birnbaum <dj...@gm...>To: 
exi...@li... <exi...@li...>Cc:
Eduard Drenth <ed...@fr...>, Michael Westbay <
wes...@ja...>Subject: Re: [Exist-open] Filenames
with awkward characters in eXist-db?Date: Mon, 23 Nov 2020 12:05:14
-0500
Dear All,
Thanks again to those who have responded quickly and helpfully. To
answer Peter's question, the <oXygen/> interface is through WebDAV, and
it works for me, as it does for Michael. While I'm unhappy to hear that
the problems I reported with the Java admin client and eXide were bugs
(user error, although embarrassing, is easier to fix), that information
tells me that I should stop trying to get those interfaces to work for
my purposes, and fall back on a work-around.

Michael's most recent posting segues into a more general question about
workflow, that is, the three methods he mentions invite me to consider
why I've been using the Java admin client to upload files instead of an
alternative. In case that larger context is helpful, whether for
filename management or just for considering eXist-db development in
general: I've tried a lot of different approaches to developing for
eXist-db, including eXide, <oXygen/>, Atom, and VS Code, and I've
settled recently on VS Code (with the eXist-db module). My reason is
that I find it most natural to work in a Git repo that lives on my file
system with automatic sync to my local eXist-db installation. When the
parts are all working at their best, I find this approach approach
simpler than working in a local Git repo and having to upload manually
to my local eXist-db installation to test changes as I implement them,
and I find it simpler than working directly inside eXist-db (whether
with eXide or with <oXygen/>) because the synchronization with VS Code
is automatic, so I can avoid having to sync the app within eXist-db
with the local repo explicitly. In this way the files on the file
system and the resources inside my local eXist-db are automatically
kept in sync. Atom has similar support, and I switched from that to VS
Code because Wolfgang mentioned in the eXist-db Slack channel that he
had made that change in his own workflow, and that inspired me to try
out VS Code.

The workflow above does not do everything I need. Specifically, it has
the same problem with filenames that have percent signs and square
brackets as the Java admin client, so that when I create a file with a
name that contains those characters in my file system, the automatic
synchronization with my running eXist-db instance fails. Additionally,
the automatic synchronization seems to have a file-size limit, which
the Java admin client doesn't have, so in the case of files that 1)
have acceptable filenames and 2) are too large for automatic sync, I
have been using the Java admiin client to upload them. Some of those
files are created in a different repo (they are part of a different
project that I am reusing in my eXist-db app), and if I have understood
correctly, I can't use xmldb:store() to store files from the file
system into eXist-db. If my understanding of this detail is correct,
that would preclude Michael's third approach. I'm working with
individual files during development, rather than a restore operation,
so Michael's second approach also doesn't match my situation closely.
What I can do, though, is consistent with Michael's first approach: use
<oXygen/> to manage the uploads of files that the Java admin client (or
automatic VS Code sync) refuses to accept. This is more awkward than
having the automatic sync work with those files, and also more awkward
than using the Java admin client because it means opening the eXist-db
instance inside <oXygen/> not to work with the files, as one usually
does in <oXygen/>, but just to manage uploads. But it will get the job
done, and I prefer it to dumbing down my self-documenting filenames,
which was the only other potential workaround I had identified.

Best,

David



On Sun, Nov 22, 2020 at 9:50 PM Michael Westbay <
wes...@ja...> wrote:
> Hi David,
> All files with UTF-8 names are put into my eXist instances in one of
> three ways:
> WebDAV -- oXygen and other WebDAV clients, dragging and dropping from
> the MacOS Finder to the client.
> Note: I don't use the MacOS built in WebDAV client as it inserts all
> kinds of junk into the database when I do.
> 
> Restore -- I use the following restore script from the command line
> in the eXist backup directory I want to restore:
> 
> export JAVA_OPTIONS="-Xms256M -Xmx512M -Dfile.encoding=UTF-8"
> export EXIST_HOME=~/exist
> ${JAVA_HOME}/bin/java ${JAVA_OPTIONS} -Dexist.home=$EXIST_HOME -jar
> $EXIST_HOME/start.jar backup -r `pwd`/__contents__.xml -u admin
> 
> XQuery -- The origin of most of the XML files in my database are
> placed there through some sort of XQuery process, concluding in a
> call to xmldb:store($col,$name,$page,"text/xml"). $name in this case
> is the UTF-8 encoded name (such as 松田-宣浩.xml). I do not URI encode
> it, so the kanji are not converted to %E6%9D%BE%E7%94%B0-
> %E5%AE%A3%E6%B5%A9.xml before the call. That leads me to believe that
> xmldb:store does the URI encoding internally.I know these work, so
> have stuck with these methods. Do you think that any of these methods
> could fit into your work flow?
> 
> 
> 
> 2020年11月23日(月) 6:09 David Birnbaum <dj...@gm...>:
> > Dear exist-open,
> > Thank you, Eduard and Michael, for the quick responses. I verified
> > that when I launch the eXist-db Java admin client it is launched
> > with -Dfile.encoding=UTF-8 among the JAVA_OPTS settings. That seems
> > to be the default; it's part of the client launch script that was
> > installed with eXist-db. Because I am not able to write my own Java
> > code, my upload methods are limited to the interfaces that eXist-db 
> > provides, such as the Java admin client and eXide, as well as the
> > access supported through <oXygen/>. The short version is that only
> > <oXygen/> is able to manage my files in a natural way, that is, it
> > can upload and access them. Here are the details about the Java
> > admin client, eXide, <oXygen/>, and the eXist-db REST interface:
> > 
> > 1. The Java admin client refuses to upload files with filenames
> > that include percent signs and square brackets, that is, certain
> > ASCII characters that have special meaning in URIs. The error
> > message in the Java admin client is
> > "/Users/djb/repos/cz/pos/verb/test-1a%7%.xml could not be encoded
> > as a URI". I think this is incorrect because this filename can be
> > encoded as a URI; for example, I can encode it as a URI with either
> > encode-for-uri() or, in XProc, p:urify(). When I try to upload the
> > file by using the file manager component of eXide, there is no
> > error (or other) message, but the file is not uploaded.
> > 
> > 2. The Java admin client does upload files with non-ASCII
> > characters in their filenames, such as "test-1, ё.xml", which
> > contains a Cyrillic character (also a space, which is ASCII, but
> > requires special handling in URIs), as long as they do not also
> > contain percent signs or square brackets. These show up in the Java
> > admin client file listing in their percent-encoded form (test-
> > 1,%20%D1%91.xml) and in the "Manage files" option in eXide in their
> > string form (test-1, ё.xml). However the eXide Manage files
> > interface cannot upload these files; as with the preceding example,
> > there is no error, but the file is not uploaded. Once I get the
> > file onto the system, if I double-click on it in the Java admin
> > client listing, it opens. If I try to open it in eXide by clicking
> > inside the File manager interface, I get an error message: "Failed
> > to load document /db/apps/cz/xml//by-form/test-1, ё.xml: 400 Bad
> > Request". But if I address the file inside a doc() function in
> > eXide (e.g., doc("/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml"))
> > in eXide, it is accessible.
> > 
> > 3. <oXygen> can upload and open both types of files.
> > 
> > 4. When I use the REST API in a browser or with curl at the command
> > line to show a directory listing (
> > http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/), the
> > filename is listed in its percent-encoded form (test-
> > 1,%20%D1%91.xml). But if I add this filename to the end of the URI
> > in the browser or on the command line with curl (
> > http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
> > ), I get an error report: 
> > 
> > > HTTP ERROR 404 Document /db/apps/cz/xml/by-form/test-1,%20?.xml
> > > not found
> > > URI:	/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
> > > STATUS:	404
> > > MESSAGE:	Document /db/apps/cz/xml/by-form/test-1,%20?.xml not
> > > found
> > > SERVLET:	EXistServlet
> > 
> > I can work around the limitations in a variety of ways, from using
> > a different file-naming strategy to managing the files only through
> > <oXygen/>, but neither of those is consistent with my normal
> > workflow, and I would also like to understand whether my
> > expectations with respect to access through the Java admin client,
> > the eXide File manager interface, and the REST API are realistic,
> > that is, whether the problems reflect misunderstanding on my part
> > or inconsistent or incorrect behavior from the eXist-db file
> > management resources. What I expect is that any interface should be
> > able to manage any file with any legal filename that can be
> > converted to a URI using encode-for-uri(). As far as I can tell,
> > only <oXygen/> is able to do this, and the Java admin client, the
> > eXide file manager, and the eXist-db REST API run into trouble (of
> > various types) with one or both of these types of filenames. When I
> > look at Eduard's and Michael's examples in this thread, their
> > filenames contain non-ASCII characters, but they do not appear to
> > contain percent signs or square brackets, and I wonder whether
> > perhaps that explains why my filenames are problematic for me while
> > theirs are not for them. Michael reports no problems with
> > <oXygen/>, and that was my experience, as well (including with
> > filenames that contain percent signs and square brackets), so my
> > issues are with the Java admin client, eXide, and the REST API.
> > 
> > Best,
> > 
> > David
> > 
> > 
> > 
> > On Wed, Nov 18, 2020 at 7:26 AM Michael Westbay <
> > wes...@ja...> wrote:
> > > 2020年11月18日(水) 19:59 Eduard Drenth <ed...@fr...>:
> > > > 
> > > > 
> > > > 
> > > > If you standardize everything in the whole chain of
> > > > components/programs/processes on utf-8 you won't have problems
> > > > is my experience. Don't forget JAVA_OPTS=-Dfile.encoding=UTF-8.
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > I use XmldbURI.xmldbUriFor(path.toFile().getName()); to
> > > >  store documents in a collection.
> > > > 
> > > > I use URLDecoder.decode(u,
> > > > "UTF-8"); to get the original filename.
> > > 
> > > I second this. I use Japanese and Korean for filenames with no
> > > problem having everything everything set to UTF-8. It works fine
> > > with oXygen and Nova editors' file trees this way.
> > > 
> > > Take care.
> > > 
> > > 
> > > -- 
> > > Michael Westbay
> > > Writer/System Administrator
> > > http://www.japanesebaseball.com/
> 
> 
-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder





-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Eduard D. <ed...@fr...>]

Of some help perhaps here (have yet to write docs for exist-db):

https://search.maven.org/search?q=a:exist-db-addons containing a
DataSyncTask for exist-db that enables you to separate editing workflow
on a filesystem from publishing worklfow in exist.

Example config:

<job class="org.fryske_akademy.exist.jobs.DataSyncTask" type="system"
period="10" repeat="0" >
    <parameter name="collection"
value="xmldb:exist:///db/apps/teidictjson/data"/>
    <parameter name="datadir" value="/data"/>
</job>
<job class="org.fryske_akademy.exist.jobs.DataSyncTaskCron"
type="system" cron-trigger="0 0 2 ? * *" >
    <parameter name="collection"
value="xmldb:exist:///db/apps/teidictjson/data"/>
    <parameter name="datadir" value="/data"/>
</job>

Bye, Eduard

-----Original Message-----
From: David Birnbaum <dj...@gm...>
To: exi...@li... <exi...@li...>
Cc: Eduard Drenth <ed...@fr...>, Michael Westbay <
wes...@ja...>
Subject: Re: [Exist-open] Filenames with awkward characters in eXist-
db?
Date: Mon, 23 Nov 2020 12:05:14 -0500

Dear All,
Thanks again to those who have responded quickly and helpfully. To
answer Peter's question, the <oXygen/> interface is through WebDAV, and
it works for me, as it does for Michael. While I'm unhappy to hear that
the problems I reported with the Java admin client and eXide were bugs
(user error, although embarrassing, is easier to fix), that information
tells me that I should stop trying to get those interfaces to work for
my purposes, and fall back on a work-around.

Michael's most recent posting segues into a more general question about
workflow, that is, the three methods he mentions invite me to consider
why I've been using the Java admin client to upload files instead of an
alternative. In case that larger context is helpful, whether for
filename management or just for considering eXist-db development in
general: I've tried a lot of different approaches to developing for
eXist-db, including eXide, <oXygen/>, Atom, and VS Code, and I've
settled recently on VS Code (with the eXist-db module). My reason is
that I find it most natural to work in a Git repo that lives on my file
system with automatic sync to my local eXist-db installation. When the
parts are all working at their best, I find this approach approach
simpler than working in a local Git repo and having to upload manually
to my local eXist-db installation to test changes as I implement them,
and I find it simpler than working directly inside eXist-db (whether
with eXide or with <oXygen/>) because the synchronization with VS Code
is automatic, so I can avoid having to sync the app within eXist-db
with the local repo explicitly. In this way the files on the file
system and the resources inside my local eXist-db are automatically
kept in sync. Atom has similar support, and I switched from that to VS
Code because Wolfgang mentioned in the eXist-db Slack channel that he
had made that change in his own workflow, and that inspired me to try
out VS Code.

The workflow above does not do everything I need. Specifically, it has
the same problem with filenames that have percent signs and square
brackets as the Java admin client, so that when I create a file with a
name that contains those characters in my file system, the automatic
synchronization with my running eXist-db instance fails. Additionally,
the automatic synchronization seems to have a file-size limit, which
the Java admin client doesn't have, so in the case of files that 1)
have acceptable filenames and 2) are too large for automatic sync, I
have been using the Java admiin client to upload them. Some of those
files are created in a different repo (they are part of a different
project that I am reusing in my eXist-db app), and if I have understood
correctly, I can't use xmldb:store() to store files from the file
system into eXist-db. If my understanding of this detail is correct,
that would preclude Michael's third approach. I'm working with
individual files during development, rather than a restore operation,
so Michael's second approach also doesn't match my situation closely.
What I can do, though, is consistent with Michael's first approach: use
<oXygen/> to manage the uploads of files that the Java admin client (or
automatic VS Code sync) refuses to accept. This is more awkward than
having the automatic sync work with those files, and also more awkward
than using the Java admin client because it means opening the eXist-db
instance inside <oXygen/> not to work with the files, as one usually
does in <oXygen/>, but just to manage uploads. But it will get the job
done, and I prefer it to dumbing down my self-documenting filenames,
which was the only other potential workaround I had identified.

Best,

David




On Sun, Nov 22, 2020 at 9:50 PM Michael Westbay <
wes...@ja...> wrote:
> Hi David,
> All files with UTF-8 names are put into my eXist instances in one of
> three ways:
> WebDAV -- oXygen and other WebDAV clients, dragging and dropping from
> the MacOS Finder to the client.
> Note: I don't use the MacOS built in WebDAV client as it inserts all
> kinds of junk into the database when I do.
> 
> Restore -- I use the following restore script from the command line
> in the eXist backup directory I want to restore:
> 
> export JAVA_OPTIONS="-Xms256M -Xmx512M -Dfile.encoding=UTF-8"
> export EXIST_HOME=~/exist
> ${JAVA_HOME}/bin/java ${JAVA_OPTIONS} -Dexist.home=$EXIST_HOME -jar
> $EXIST_HOME/start.jar backup -r `pwd`/__contents__.xml -u admin
> 
> XQuery -- The origin of most of the XML files in my database are
> placed there through some sort of XQuery process, concluding in a
> call to xmldb:store($col,$name,$page,"text/xml"). $name in this case
> is the UTF-8 encoded name (such as 松田-宣浩.xml). I do not URI encode
> it, so the kanji are not converted to %E6%9D%BE%E7%94%B0-
> %E5%AE%A3%E6%B5%A9.xml before the call. That leads me to believe that
> xmldb:store does the URI encoding internally.I know these work, so
> have stuck with these methods. Do you think that any of these methods
> could fit into your work flow?
> 
> 
> 
> 2020年11月23日(月) 6:09 David Birnbaum <dj...@gm...>:
> > Dear exist-open,
> > Thank you, Eduard and Michael, for the quick responses. I verified
> > that when I launch the eXist-db Java admin client it is launched
> > with -Dfile.encoding=UTF-8 among the JAVA_OPTS settings. That seems
> > to be the default; it's part of the client launch script that was
> > installed with eXist-db. Because I am not able to write my own Java
> > code, my upload methods are limited to the interfaces that eXist-db 
> > provides, such as the Java admin client and eXide, as well as the
> > access supported through <oXygen/>. The short version is that only
> > <oXygen/> is able to manage my files in a natural way, that is, it
> > can upload and access them. Here are the details about the Java
> > admin client, eXide, <oXygen/>, and the eXist-db REST interface:
> > 
> > 1. The Java admin client refuses to upload files with filenames
> > that include percent signs and square brackets, that is, certain
> > ASCII characters that have special meaning in URIs. The error
> > message in the Java admin client is
> > "/Users/djb/repos/cz/pos/verb/test-1a%7%.xml could not be encoded
> > as a URI". I think this is incorrect because this filename can be
> > encoded as a URI; for example, I can encode it as a URI with either
> > encode-for-uri() or, in XProc, p:urify(). When I try to upload the
> > file by using the file manager component of eXide, there is no
> > error (or other) message, but the file is not uploaded.
> > 
> > 2. The Java admin client does upload files with non-ASCII
> > characters in their filenames, such as "test-1, ё.xml", which
> > contains a Cyrillic character (also a space, which is ASCII, but
> > requires special handling in URIs), as long as they do not also
> > contain percent signs or square brackets. These show up in the Java
> > admin client file listing in their percent-encoded form (test-
> > 1,%20%D1%91.xml) and in the "Manage files" option in eXide in their
> > string form (test-1, ё.xml). However the eXide Manage files
> > interface cannot upload these files; as with the preceding example,
> > there is no error, but the file is not uploaded. Once I get the
> > file onto the system, if I double-click on it in the Java admin
> > client listing, it opens. If I try to open it in eXide by clicking
> > inside the File manager interface, I get an error message: "Failed
> > to load document /db/apps/cz/xml//by-form/test-1, ё.xml: 400 Bad
> > Request". But if I address the file inside a doc() function in
> > eXide (e.g., doc("/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml"))
> > in eXide, it is accessible.
> > 
> > 3. <oXygen> can upload and open both types of files.
> > 
> > 4. When I use the REST API in a browser or with curl at the command
> > line to show a directory listing (
> > http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/), the
> > filename is listed in its percent-encoded form (test-
> > 1,%20%D1%91.xml). But if I add this filename to the end of the URI
> > in the browser or on the command line with curl (
> > http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
> > ), I get an error report: 
> > 
> > > HTTP ERROR 404 Document /db/apps/cz/xml/by-form/test-1,%20?.xml
> > > not found
> > > URI:	/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
> > > STATUS:	404
> > > MESSAGE:	Document /db/apps/cz/xml/by-form/test-1,%20?.xml not
> > > found
> > > SERVLET:	EXistServlet
> > 
> > I can work around the limitations in a variety of ways, from using
> > a different file-naming strategy to managing the files only through
> > <oXygen/>, but neither of those is consistent with my normal
> > workflow, and I would also like to understand whether my
> > expectations with respect to access through the Java admin client,
> > the eXide File manager interface, and the REST API are realistic,
> > that is, whether the problems reflect misunderstanding on my part
> > or inconsistent or incorrect behavior from the eXist-db file
> > management resources. What I expect is that any interface should be
> > able to manage any file with any legal filename that can be
> > converted to a URI using encode-for-uri(). As far as I can tell,
> > only <oXygen/> is able to do this, and the Java admin client, the
> > eXide file manager, and the eXist-db REST API run into trouble (of
> > various types) with one or both of these types of filenames. When I
> > look at Eduard's and Michael's examples in this thread, their
> > filenames contain non-ASCII characters, but they do not appear to
> > contain percent signs or square brackets, and I wonder whether
> > perhaps that explains why my filenames are problematic for me while
> > theirs are not for them. Michael reports no problems with
> > <oXygen/>, and that was my experience, as well (including with
> > filenames that contain percent signs and square brackets), so my
> > issues are with the Java admin client, eXide, and the REST API.
> > 
> > Best,
> > 
> > David
> > 
> > 
> > 
> > On Wed, Nov 18, 2020 at 7:26 AM Michael Westbay <
> > wes...@ja...> wrote:
> > > 2020年11月18日(水) 19:59 Eduard Drenth <ed...@fr...>:
> > > > 
> > > > 
> > > > 
> > > > If you standardize everything in the whole chain of
> > > > components/programs/processes on utf-8 you won't have problems
> > > > is my experience. Don't forget JAVA_OPTS=-Dfile.encoding=UTF-8.
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > I use XmldbURI.xmldbUriFor(path.toFile().getName()); to
> > > >  store documents in a collection.
> > > > 
> > > > I use URLDecoder.decode(u,
> > > > "UTF-8"); to get the original filename.
> > > 
> > > I second this. I use Japanese and Korean for filenames with no
> > > problem having everything everything set to UTF-8. It works fine
> > > with oXygen and Nova editors' file trees this way.
> > > 
> > > Take care.
> > > 
> > > 
> > > -- 
> > > Michael Westbay
> > > Writer/System Administrator
> > > http://www.japanesebaseball.com/
> 
> 
-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[David B. <dj...@gm...>]

Dear All,

Thanks again to those who have responded quickly and helpfully. To answer
Peter's question, the <oXygen/> interface is through WebDAV, and it works
for me, as it does for Michael. While I'm unhappy to hear that the problems
I reported with the Java admin client and eXide were bugs (user error,
although embarrassing, is easier to fix), that information tells me that I
should stop trying to get those interfaces to work for my purposes, and
fall back on a work-around.

Michael's most recent posting segues into a more general question about
workflow, that is, the three methods he mentions invite me to consider why
I've been using the Java admin client to upload files instead of an
alternative. In case that larger context is helpful, whether for filename
management or just for considering eXist-db development in general: I've
tried a lot of different approaches to developing for eXist-db, including
eXide, <oXygen/>, Atom, and VS Code, and I've settled recently on VS Code
(with the eXist-db module). My reason is that I find it most natural to
work in a Git repo that lives on my file system with automatic sync to my
local eXist-db installation. When the parts are all working at their best,
I find this approach approach simpler than working in a local Git repo and
having to upload manually to my local eXist-db installation to test changes
as I implement them, and I find it simpler than working directly inside
eXist-db (whether with eXide or with <oXygen/>) because the synchronization
with VS Code is automatic, so I can avoid having to sync the app within
eXist-db with the local repo explicitly. In this way the files on the file
system and the resources inside my local eXist-db are automatically kept in
sync. Atom has similar support, and I switched from that to VS Code because
Wolfgang mentioned in the eXist-db Slack channel that he had made that
change in his own workflow, and that inspired me to try out VS Code.

The workflow above does not do everything I need. Specifically, it has the
same problem with filenames that have percent signs and square brackets as
the Java admin client, so that when I create a file with a name that
contains those characters in my file system, the automatic synchronization
with my running eXist-db instance fails. Additionally, the automatic
synchronization seems to have a file-size limit, which the Java admin
client doesn't have, so in the case of files that 1) have acceptable
filenames and 2) are too large for automatic sync, I have been using the
Java admiin client to upload them. Some of those files are created in a
different repo (they are part of a different project that I am reusing in
my eXist-db app), and if I have understood correctly, I can't use
xmldb:store() to store files from the file system into eXist-db. If my
understanding of this detail is correct, that would preclude Michael's
third approach. I'm working with individual files during development,
rather than a restore operation, so Michael's second approach also doesn't
match my situation closely. What I can do, though, is consistent with
Michael's first approach: use <oXygen/> to manage the uploads of files that
the Java admin client (or automatic VS Code sync) refuses to accept. This
is more awkward than having the automatic sync work with those files, and
also more awkward than using the Java admin client because it means opening
the eXist-db instance inside <oXygen/> not to work with the files, as one
usually does in <oXygen/>, but just to manage uploads. But it will get the
job done, and I prefer it to dumbing down my self-documenting filenames,
which was the only other potential workaround I had identified.

Best,

David



On Sun, Nov 22, 2020 at 9:50 PM Michael Westbay <
wes...@ja...> wrote:

> Hi David,
>
> All files with UTF-8 names are put into my eXist instances in one of three
> ways:
>
>    1. WebDAV -- oXygen and other WebDAV clients, dragging and dropping
>    from the MacOS Finder to the client.
>    Note: I don't use the MacOS built in WebDAV client as it inserts all
>    kinds of junk into the database when I do.
>
>    2. Restore -- I use the following restore script from the command line
>    in the eXist backup directory I want to restore:
>
>    export JAVA_OPTIONS="-Xms256M -Xmx512M -Dfile.encoding=UTF-8"
>    export EXIST_HOME=~/exist
>    ${JAVA_HOME}/bin/java ${JAVA_OPTIONS} -Dexist.home=$EXIST_HOME -jar
>    $EXIST_HOME/start.jar backup -r `pwd`/__contents__.xml -u admin
>
>    3. XQuery -- The origin of most of the XML files in my database are
>    placed there through some sort of XQuery process, concluding in a call to
>    xmldb:store($col,$name,$page,"text/xml"). $name in this case is the UTF-8
>    encoded name (such as 松田-宣浩.xml). I *do not* URI encode it, so the
>    kanji are *not* converted to %E6%9D%BE%E7%94%B0-%E5%AE%A3%E6%B5%A9.xml
>    before the call. That leads me to believe that xmldb:store does the URI
>    encoding internally.
>
> I know these work, so have stuck with these methods. Do you think that any
> of these methods could fit into your work flow?
>
>
> 2020年11月23日(月) 6:09 David Birnbaum <dj...@gm...>:
>
>> Dear exist-open,
>>
>> Thank you, Eduard and Michael, for the quick responses. I verified that
>> when I launch the eXist-db Java admin client it is launched
>> with -Dfile.encoding=UTF-8 among the JAVA_OPTS settings. That seems to be
>> the default; it's part of the client launch script that was installed with
>> eXist-db. Because I am not able to write my own Java code, my upload
>> methods are limited to the interfaces that eXist-db provides, such as the
>> Java admin client and eXide, as well as the access supported through
>> <oXygen/>. The short version is that only <oXygen/> is able to manage my
>> files in a natural way, that is, it can upload and access them. Here are
>> the details about the Java admin client, eXide, <oXygen/>, and the eXist-db
>> REST interface:
>>
>> 1. The Java admin client refuses to upload files with filenames that
>> include percent signs and square brackets, that is, certain ASCII
>> characters that have special meaning in URIs. The error message in the Java
>> admin client is "/Users/djb/repos/cz/pos/verb/test-1a%7%.xml could not be
>> encoded as a URI". I think this is incorrect because this filename can be
>> encoded as a URI; for example, I can encode it as a URI with either
>> encode-for-uri() or, in XProc, p:urify(). When I try to upload the file by
>> using the file manager component of eXide, there is no error (or other)
>> message, but the file is not uploaded.
>>
>> 2. The Java admin client does upload files with non-ASCII characters in
>> their filenames, such as "test-1, ё.xml", which contains a Cyrillic
>> character (also a space, which is ASCII, but requires special handling in
>> URIs), as long as they do not also contain percent signs or square
>> brackets. These show up in the Java admin client file listing in their
>> percent-encoded form (test-1,%20%D1%91.xml) and in the "Manage files"
>> option in eXide in their string form (test-1, ё.xml). However the eXide
>> Manage files interface cannot upload these files; as with the preceding
>> example, there is no error, but the file is not uploaded. Once I get the
>> file onto the system, if I double-click on it in the Java admin client
>> listing, it opens. If I try to open it in eXide by clicking inside the File
>> manager interface, I get an error message: "Failed to load document
>> /db/apps/cz/xml//by-form/test-1, ё.xml: 400 Bad Request". But if I address
>> the file inside a doc() function in eXide (e.g.,
>> doc("/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml")) in eXide, it is
>> accessible.
>>
>> 3. <oXygen> can upload and open both types of files.
>>
>> 4. When I use the REST API in a browser or with curl at the command line
>> to show a directory listing (
>> http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/), the filename
>> is listed in its percent-encoded form (test-1,%20%D1%91.xml). But if I add
>> this filename to the end of the URI in the browser or on the command line
>> with curl (
>> http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml),
>> I get an error report:
>>
>> HTTP ERROR 404 Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
>> URI: /exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
>> STATUS: 404
>> MESSAGE: Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
>> SERVLET: EXistServlet
>>
>>
>> I can work around the limitations in a variety of ways, from using a
>> different file-naming strategy to managing the files only through
>> <oXygen/>, but neither of those is consistent with my normal workflow, and
>> I would also like to understand whether my expectations with respect to
>> access through the Java admin client, the eXide File manager interface, and
>> the REST API are realistic, that is, whether the problems reflect
>> misunderstanding on my part or inconsistent or incorrect behavior from the
>> eXist-db file management resources. What I expect is that any interface
>> should be able to manage any file with any legal filename that can be
>> converted to a URI using encode-for-uri(). As far as I can tell, only
>> <oXygen/> is able to do this, and the Java admin client, the eXide file
>> manager, and the eXist-db REST API run into trouble (of various types) with
>> one or both of these types of filenames. When I look at Eduard's and
>> Michael's examples in this thread, their filenames contain non-ASCII
>> characters, but they do not appear to contain percent signs or square
>> brackets, and I wonder whether perhaps that explains why my filenames are
>> problematic for me while theirs are not for them. Michael reports no
>> problems with <oXygen/>, and that was my experience, as well (including
>> with filenames that contain percent signs and square brackets), so my
>> issues are with the Java admin client, eXide, and the REST API.
>>
>> Best,
>>
>> David
>>
>>
>> On Wed, Nov 18, 2020 at 7:26 AM Michael Westbay <
>> wes...@ja...> wrote:
>>
>>> 2020年11月18日(水) 19:59 Eduard Drenth <ed...@fr...>:
>>>
>>>> If you standardize everything in the whole chain of
>>>> components/programs/processes on utf-8 you won't have problems is my
>>>> experience. Don't forget JAVA_OPTS=-Dfile.encoding=UTF-8.
>>>>
>>>> I use XmldbURI.xmldbUriFor(path.toFile().getName()); to store
>>>> documents in a collection.
>>>> I use URLDecoder.decode(u, "UTF-8"); to get the original filename.
>>>>
>>>
>>> I second this. I use Japanese and Korean for filenames with no problem
>>> having everything everything set to UTF-8. It works fine with oXygen and
>>> Nova editors' file trees this way.
>>>
>>> Take care.
>>>
>>> --
>>> Michael Westbay
>>> Writer/System Administrator
>>> http://www.japanesebaseball.com/
>>>
>>
>
> --
> Michael Westbay
> Writer/System Administrator
> http://www.japanesebaseball.com/
>

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Michael W. <wes...@ja...>]

Hi David,

All files with UTF-8 names are put into my eXist instances in one of three
ways:

   1. WebDAV -- oXygen and other WebDAV clients, dragging and dropping from
   the MacOS Finder to the client.
   Note: I don't use the MacOS built in WebDAV client as it inserts all
   kinds of junk into the database when I do.

   2. Restore -- I use the following restore script from the command line
   in the eXist backup directory I want to restore:

   export JAVA_OPTIONS="-Xms256M -Xmx512M -Dfile.encoding=UTF-8"
   export EXIST_HOME=~/exist
   ${JAVA_HOME}/bin/java ${JAVA_OPTIONS} -Dexist.home=$EXIST_HOME -jar
   $EXIST_HOME/start.jar backup -r `pwd`/__contents__.xml -u admin

   3. XQuery -- The origin of most of the XML files in my database are
   placed there through some sort of XQuery process, concluding in a call to
   xmldb:store($col,$name,$page,"text/xml"). $name in this case is the UTF-8
   encoded name (such as 松田-宣浩.xml). I *do not* URI encode it, so the kanji
   are *not* converted to %E6%9D%BE%E7%94%B0-%E5%AE%A3%E6%B5%A9.xml before
   the call. That leads me to believe that xmldb:store does the URI encoding
   internally.

I know these work, so have stuck with these methods. Do you think that any
of these methods could fit into your work flow?


2020年11月23日(月) 6:09 David Birnbaum <dj...@gm...>:

> Dear exist-open,
>
> Thank you, Eduard and Michael, for the quick responses. I verified that
> when I launch the eXist-db Java admin client it is launched
> with -Dfile.encoding=UTF-8 among the JAVA_OPTS settings. That seems to be
> the default; it's part of the client launch script that was installed with
> eXist-db. Because I am not able to write my own Java code, my upload
> methods are limited to the interfaces that eXist-db provides, such as the
> Java admin client and eXide, as well as the access supported through
> <oXygen/>. The short version is that only <oXygen/> is able to manage my
> files in a natural way, that is, it can upload and access them. Here are
> the details about the Java admin client, eXide, <oXygen/>, and the eXist-db
> REST interface:
>
> 1. The Java admin client refuses to upload files with filenames that
> include percent signs and square brackets, that is, certain ASCII
> characters that have special meaning in URIs. The error message in the Java
> admin client is "/Users/djb/repos/cz/pos/verb/test-1a%7%.xml could not be
> encoded as a URI". I think this is incorrect because this filename can be
> encoded as a URI; for example, I can encode it as a URI with either
> encode-for-uri() or, in XProc, p:urify(). When I try to upload the file by
> using the file manager component of eXide, there is no error (or other)
> message, but the file is not uploaded.
>
> 2. The Java admin client does upload files with non-ASCII characters in
> their filenames, such as "test-1, ё.xml", which contains a Cyrillic
> character (also a space, which is ASCII, but requires special handling in
> URIs), as long as they do not also contain percent signs or square
> brackets. These show up in the Java admin client file listing in their
> percent-encoded form (test-1,%20%D1%91.xml) and in the "Manage files"
> option in eXide in their string form (test-1, ё.xml). However the eXide
> Manage files interface cannot upload these files; as with the preceding
> example, there is no error, but the file is not uploaded. Once I get the
> file onto the system, if I double-click on it in the Java admin client
> listing, it opens. If I try to open it in eXide by clicking inside the File
> manager interface, I get an error message: "Failed to load document
> /db/apps/cz/xml//by-form/test-1, ё.xml: 400 Bad Request". But if I address
> the file inside a doc() function in eXide (e.g.,
> doc("/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml")) in eXide, it is
> accessible.
>
> 3. <oXygen> can upload and open both types of files.
>
> 4. When I use the REST API in a browser or with curl at the command line
> to show a directory listing (
> http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/), the filename
> is listed in its percent-encoded form (test-1,%20%D1%91.xml). But if I add
> this filename to the end of the URI in the browser or on the command line
> with curl (
> http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml),
> I get an error report:
>
> HTTP ERROR 404 Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
> URI: /exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
> STATUS: 404
> MESSAGE: Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
> SERVLET: EXistServlet
>
>
> I can work around the limitations in a variety of ways, from using a
> different file-naming strategy to managing the files only through
> <oXygen/>, but neither of those is consistent with my normal workflow, and
> I would also like to understand whether my expectations with respect to
> access through the Java admin client, the eXide File manager interface, and
> the REST API are realistic, that is, whether the problems reflect
> misunderstanding on my part or inconsistent or incorrect behavior from the
> eXist-db file management resources. What I expect is that any interface
> should be able to manage any file with any legal filename that can be
> converted to a URI using encode-for-uri(). As far as I can tell, only
> <oXygen/> is able to do this, and the Java admin client, the eXide file
> manager, and the eXist-db REST API run into trouble (of various types) with
> one or both of these types of filenames. When I look at Eduard's and
> Michael's examples in this thread, their filenames contain non-ASCII
> characters, but they do not appear to contain percent signs or square
> brackets, and I wonder whether perhaps that explains why my filenames are
> problematic for me while theirs are not for them. Michael reports no
> problems with <oXygen/>, and that was my experience, as well (including
> with filenames that contain percent signs and square brackets), so my
> issues are with the Java admin client, eXide, and the REST API.
>
> Best,
>
> David
>
>
> On Wed, Nov 18, 2020 at 7:26 AM Michael Westbay <
> wes...@ja...> wrote:
>
>> 2020年11月18日(水) 19:59 Eduard Drenth <ed...@fr...>:
>>
>>> If you standardize everything in the whole chain of
>>> components/programs/processes on utf-8 you won't have problems is my
>>> experience. Don't forget JAVA_OPTS=-Dfile.encoding=UTF-8.
>>>
>>> I use XmldbURI.xmldbUriFor(path.toFile().getName()); to store documents
>>> in a collection.
>>> I use URLDecoder.decode(u, "UTF-8"); to get the original filename.
>>>
>>
>> I second this. I use Japanese and Korean for filenames with no problem
>> having everything everything set to UTF-8. It works fine with oXygen and
>> Nova editors' file trees this way.
>>
>> Take care.
>>
>> --
>> Michael Westbay
>> Writer/System Administrator
>> http://www.japanesebaseball.com/
>>
>

-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Hungerburg 13 <pc...@my...>]

Am 22.11.20 um 23:22 schrieb Eduard Drenth:
> Dear all,
> 
> I tested with a file called za%ak.xml, when trying to upload it via 
> eXide db manager nothing happens, no errors show. This confirms one of 
> the things David sees. One for a github issue I think.
> 
> The java admin client I did not test but I suspect this is also one for 
> an issue.
> 
> Feels like a close look at how file/document(name) handling is done in 
> the various components of exist would be wise.


The eXide db manager is a /third party/ addon module, however useful. 
You should try the REST and the WEBDAV interfaces. Perhaps there are 
other on-board methods, which one is oXygen using?

Peter, just another user lurking

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[Eduard D. <ed...@fr...>]

Dear all,

I tested with a file called za%ak.xml, when trying to upload it via eXide db manager nothing happens, no errors show. This confirms one of the things David sees. One for a github issue I think.

The java admin client I did not test but I suspect this is also one for an issue.

Feels like a close look at how file/document(name) handling is done in the various components of exist would be wise.

Regards

-----Original Message-----
From: David Birnbaum <dj...@gm...<mailto:David%20Birnbaum%20%3cd...@gm...%3e>>
To: Michael Westbay <wes...@ja...<mailto:Michael%20Westbay%20%3cw...@ja...%3e>>
Cc: Eduard Drenth <ed...@fr...<mailto:Eduard%20Drenth%20%3ce...@fr...%3e>>, exi...@li... <exi...@li...<mailto:%22e...@li...%22%20%3ce...@li...%3e>>
Subject: Re: [Exist-open] Filenames with awkward characters in eXist-db?
Date: Sun, 22 Nov 2020 16:09:39 -0500

Dear exist-open,

Thank you, Eduard and Michael, for the quick responses. I verified that when I launch the eXist-db Java admin client it is launched with -Dfile.encoding=UTF-8 among the JAVA_OPTS settings. That seems to be the default; it's part of the client launch script that was installed with eXist-db. Because I am not able to write my own Java code, my upload methods are limited to the interfaces that eXist-db provides, such as the Java admin client and eXide, as well as the access supported through <oXygen/>. The short version is that only <oXygen/> is able to manage my files in a natural way, that is, it can upload and access them. Here are the details about the Java admin client, eXide, <oXygen/>, and the eXist-db REST interface:

1. The Java admin client refuses to upload files with filenames that include percent signs and square brackets, that is, certain ASCII characters that have special meaning in URIs. The error message in the Java admin client is "/Users/djb/repos/cz/pos/verb/test-1a%7%.xml could not be encoded as a URI". I think this is incorrect because this filename can be encoded as a URI; for example, I can encode it as a URI with either encode-for-uri() or, in XProc, p:urify(). When I try to upload the file by using the file manager component of eXide, there is no error (or other) message, but the file is not uploaded.

2. The Java admin client does upload files with non-ASCII characters in their filenames, such as "test-1, ё.xml", which contains a Cyrillic character (also a space, which is ASCII, but requires special handling in URIs), as long as they do not also contain percent signs or square brackets. These show up in the Java admin client file listing in their percent-encoded form (test-1,%20%D1%91.xml) and in the "Manage files" option in eXide in their string form (test-1, ё.xml). However the eXide Manage files interface cannot upload these files; as with the preceding example, there is no error, but the file is not uploaded. Once I get the file onto the system, if I double-click on it in the Java admin client listing, it opens. If I try to open it in eXide by clicking inside the File manager interface, I get an error message: "Failed to load document /db/apps/cz/xml//by-form/test-1, ё.xml: 400 Bad Request". But if I address the file inside a doc() function in eXide (e.g., doc("/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml")) in eXide, it is accessible.

3. <oXygen> can upload and open both types of files.

4. When I use the REST API in a browser or with curl at the command line to show a directory listing (http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/), the filename is listed in its percent-encoded form (test-1,%20%D1%91.xml). But if I add this filename to the end of the URI in the browser or on the command line with curl (http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml), I get an error report:

HTTP ERROR 404 Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
URI: /exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
STATUS: 404
MESSAGE: Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
SERVLET: EXistServlet

I can work around the limitations in a variety of ways, from using a different file-naming strategy to managing the files only through <oXygen/>, but neither of those is consistent with my normal workflow, and I would also like to understand whether my expectations with respect to access through the Java admin client, the eXide File manager interface, and the REST API are realistic, that is, whether the problems reflect misunderstanding on my part or inconsistent or incorrect behavior from the eXist-db file management resources. What I expect is that any interface should be able to manage any file with any legal filename that can be converted to a URI using encode-for-uri(). As far as I can tell, only <oXygen/> is able to do this, and the Java admin client, the eXide file manager, and the eXist-db REST API run into trouble (of various types) with one or both of these types of filenames. When I look at Eduard's and Michael's examples in this thread, their filenames contain non-ASCII characters, but they do not appear to contain percent signs or square brackets, and I wonder whether perhaps that explains why my filenames are problematic for me while theirs are not for them. Michael reports no problems with <oXygen/>, and that was my experience, as well (including with filenames that contain percent signs and square brackets), so my issues are with the Java admin client, eXide, and the REST API.

Best,

David


On Wed, Nov 18, 2020 at 7:26 AM Michael Westbay <wes...@ja...<mailto:wes...@ja...>> wrote:
2020年11月18日(水) 19:59 Eduard Drenth <ed...@fr...<mailto:ed...@fr...>>:
If you standardize everything in the whole chain of components/programs/processes on utf-8 you won't have problems is my experience. Don't forget JAVA_OPTS=-Dfile.encoding=UTF-8.

I use XmldbURI.xmldbUriFor(path.toFile().getName()); to store documents in a collection.
I use URLDecoder.decode(u, "UTF-8"); to get the original filename.


I second this. I use Japanese and Korean for filenames with no problem having everything everything set to UTF-8. It works fine with oXygen and Nova editors' file trees this way.

Take care.

--
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

--

Eduard Drenth, Software Architekt

ed...@fr...<mailto:ed...@fr...>

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] Filenames with awkward characters in eXist-db?

[David B. <dj...@gm...>]

Dear exist-open,

Thank you, Eduard and Michael, for the quick responses. I verified that
when I launch the eXist-db Java admin client it is launched
with -Dfile.encoding=UTF-8 among the JAVA_OPTS settings. That seems to be
the default; it's part of the client launch script that was installed with
eXist-db. Because I am not able to write my own Java code, my upload
methods are limited to the interfaces that eXist-db provides, such as the
Java admin client and eXide, as well as the access supported through
<oXygen/>. The short version is that only <oXygen/> is able to manage my
files in a natural way, that is, it can upload and access them. Here are
the details about the Java admin client, eXide, <oXygen/>, and the eXist-db
REST interface:

1. The Java admin client refuses to upload files with filenames that
include percent signs and square brackets, that is, certain ASCII
characters that have special meaning in URIs. The error message in the Java
admin client is "/Users/djb/repos/cz/pos/verb/test-1a%7%.xml could not be
encoded as a URI". I think this is incorrect because this filename can be
encoded as a URI; for example, I can encode it as a URI with either
encode-for-uri() or, in XProc, p:urify(). When I try to upload the file by
using the file manager component of eXide, there is no error (or other)
message, but the file is not uploaded.

2. The Java admin client does upload files with non-ASCII characters in
their filenames, such as "test-1, ё.xml", which contains a Cyrillic
character (also a space, which is ASCII, but requires special handling in
URIs), as long as they do not also contain percent signs or square
brackets. These show up in the Java admin client file listing in their
percent-encoded form (test-1,%20%D1%91.xml) and in the "Manage files"
option in eXide in their string form (test-1, ё.xml). However the eXide
Manage files interface cannot upload these files; as with the preceding
example, there is no error, but the file is not uploaded. Once I get the
file onto the system, if I double-click on it in the Java admin client
listing, it opens. If I try to open it in eXide by clicking inside the File
manager interface, I get an error message: "Failed to load document
/db/apps/cz/xml//by-form/test-1, ё.xml: 400 Bad Request". But if I address
the file inside a doc() function in eXide (e.g.,
doc("/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml")) in eXide, it is
accessible.

3. <oXygen> can upload and open both types of files.

4. When I use the REST API in a browser or with curl at the command line to
show a directory listing (
http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/), the filename is
listed in its percent-encoded form (test-1,%20%D1%91.xml). But if I add
this filename to the end of the URI in the browser or on the command line
with curl (
http://localhost:8080/exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml),
I get an error report:

HTTP ERROR 404 Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
URI: /exist/rest/db/apps/cz/xml/by-form/test-1,%20%D1%91.xml
STATUS: 404
MESSAGE: Document /db/apps/cz/xml/by-form/test-1,%20?.xml not found
SERVLET: EXistServlet


I can work around the limitations in a variety of ways, from using a
different file-naming strategy to managing the files only through
<oXygen/>, but neither of those is consistent with my normal workflow, and
I would also like to understand whether my expectations with respect to
access through the Java admin client, the eXide File manager interface, and
the REST API are realistic, that is, whether the problems reflect
misunderstanding on my part or inconsistent or incorrect behavior from the
eXist-db file management resources. What I expect is that any interface
should be able to manage any file with any legal filename that can be
converted to a URI using encode-for-uri(). As far as I can tell, only
<oXygen/> is able to do this, and the Java admin client, the eXide file
manager, and the eXist-db REST API run into trouble (of various types) with
one or both of these types of filenames. When I look at Eduard's and
Michael's examples in this thread, their filenames contain non-ASCII
characters, but they do not appear to contain percent signs or square
brackets, and I wonder whether perhaps that explains why my filenames are
problematic for me while theirs are not for them. Michael reports no
problems with <oXygen/>, and that was my experience, as well (including
with filenames that contain percent signs and square brackets), so my
issues are with the Java admin client, eXide, and the REST API.

Best,

David


On Wed, Nov 18, 2020 at 7:26 AM Michael Westbay <
wes...@ja...> wrote:

> 2020年11月18日(水) 19:59 Eduard Drenth <ed...@fr...>:
>
>> If you standardize everything in the whole chain of
>> components/programs/processes on utf-8 you won't have problems is my
>> experience. Don't forget JAVA_OPTS=-Dfile.encoding=UTF-8.
>>
>> I use XmldbURI.xmldbUriFor(path.toFile().getName()); to store documents
>> in a collection.
>> I use URLDecoder.decode(u, "UTF-8"); to get the original filename.
>>
>
> I second this. I use Japanese and Korean for filenames with no problem
> having everything everything set to UTF-8. It works fine with oXygen and
> Nova editors' file trees this way.
>
> Take care.
>
> --
> Michael Westbay
> Writer/System Administrator
> http://www.japanesebaseball.com/
>

[Exist-open] Community Call time change

[Joe W. <jo...@gm...>]

Hi all,

In last week's Community Call, we agreed to move the afternoon (15:30 CET)
sessions to the evening (19:30 CET), due to low attendance at the afternoon
calls. Thus, starting with tomorrow's call:


*All Community Calls will take place at the same time each week,
19:30-20:30 CET.*

The update agenda for tomorrow's call can be found here
<https://docs.google.com/document/d/1nrNJ1eAMLaesq5HzVaQ3QfK1ovK5mzLmn_yGfo595zo/edit?usp=sharing>.
The public Google Calendar has also been updated; its link is here
<https://calendar.google.com/calendar/u/0?cid=OHVnNmtwcnFnNWNvNmRwZGZxc2FrY283MWtAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ>.
The automated notifications to the eXist-db Slack workspace's #community
channel will reflect the new time.

As always, everyone is welcome to join! If you have items to discuss,
please post them in the agenda document. It's editable by all. The agenda
document (pinned to the #community channel) also contains the link to join
the call.

Best,
Joe

Re: [Exist-open] hello world

[Jean-Paul R. <re...@gm...>]

Dear Nicolas,

I responded to your post on Stack Overflow, but I think you've gone a bit
sideways in learning eXist-db.

Start with the basics:
- create and save your XML documents through eXide
- create and execute your XQueries through eXide

All of this is done using the buttons at the top of eXide (see screenshot
below):

[image: Screen Shot 2020-11-21 at 12.10.11.png]

In this way, eXide can validate your XML documents and your XQueries
*before* you store them in the database. I suspect some of your problems
are stemming from badly-formed XML; in that case, your queries will never
work.

For more faster, more dynamic assistance feel free to join the eXist slack
community at
https://join.slack.com/t/exist-db/shared_invite/zt-89ukckoj-zbbC_V2QDIR5nGMokG9TQA
- it's easier to help troubleshoot there.

Cheers,
JPR




On Sat, Nov 21, 2020 at 9:51 AM Nicholas <sau...@gm...>
wrote:

> Thanks, Michael.  Probably I'm making a silly error:
>
>
> https://stackoverflow.com/q/64941000/4531180
>
>
> with the FLWOR.
>
>
> -Nick
>
>
> On 11/18/20 11:36 PM, Michael Westbay wrote:
>
> Hi Nick,
>
> Yes, that was placed on my local version.
>
> If you only got "<notes/>" then that means that it didn't find any *note* element
> in the collection "/db/temp". The annotated script reads:
>
> xquery version "3.1";           (: XQuery version declaration for the
> XQuery processor :)
>
> let $notes := collection('/db/temp')/note    (: Gather all
> <note>...</note> documents under the /db/temp collection/folder/directory
> in the database :)
>
> return <notes>{$notes}</notes>     (: Output a <notes>...</notes> tag with
> the gathered <note>...</note> documents embedded in it :)
>
>
> Since you had nothing within the <notes/> tag, it found not
> <note>...</note> documents on your eXist instance.
>
> You should create a lot of <note>...</note> documents within a collection
> and see that result. Then output:
>
> return <notes>{$note/body}</notes>
>
>
> and see what happens.
>
> Take care.
>
>
> 2020年11月19日(木) 14:59 Nicholas <sau...@gm...>:
>
>> Hi Michael,
>>
>>
>> Browsing to:
>>
>>
>> http://localhost:8080/exist/rest/db/scripts/notes.xq
>>
>>
>> gives me:
>>
>> <notes/>
>>
>> with an xquery of:
>>
>> xquery version "3.1";
>>
>> let $notes := collection('/db/temp')/note
>>
>> return <notes>{$notes}</notes>
>>
>> and a "note" file:
>>
>> <note>
>> <to>Tove</to>
>> <from>Jani</from>
>> <heading>Reminder</heading>
>> <body>Don't forget me this weekend!</body>
>> </note>
>>
>>
>> anyhow, that's enough output to get me rolling, so thanks.  I think it's
>> just a mix-up there with plural/singular.  Everything is on localhost.
>> Most interesting!
>>
>>
>> thanks,
>>
>>
>> Nick
>>
>>
>>
>> On 11/18/20 7:10 PM, Michael Westbay wrote:
>>
>> Hi Nick,
>>
>> First of all, XML files are NOT stored on the file system. The non-XML
>> files that do get stored there should NEVER be accessed outside of eXist as
>> that will throw the database's internal idea of what is in the database off.
>>
>> Within eXide, go to the "File" menu and select "Manage." this will let
>> you explore the database's internal file system to see where it is you
>> placed the XML file. Please make a note of the full path to where you
>> placed your <note> document.
>>
>> Next, create a new XQuery document. I placed your sample note under the
>> '/db/temp' *collection* (a collection corresponds to a folder in the
>> file hierarchy). If you enter a lot of these <note/> documents into the
>> same collection, you can get a list of them with:
>>
>> xquery version "3.1";
>>
>> let $notes := collection('/db/temp')/note
>>
>> return <notes>{$notes}</notes>
>>
>>
>> I only have the one <note> document, so my output is:
>>
>> [image: image.png]
>>
>>
>> You can then save the XQuery document to somewhere like
>> "/db/scripts/notes.xq" and access it at:
>>
>> localhost:8080/exist/rest/db/scripts.notes.xq
>>
>>
>> Hope this helps get you started.
>>
>> Take care.
>>
>>
>> 2020年11月19日(木) 11:50 Nicholas <sau...@gm...>:
>>
>>> I created a simple document through eXide on localhost:
>>>
>>> https://unix.stackexchange.com/q/620383/101935
>>>
>>> but can't seem to find the actual file.  The content is from a sample:
>>>
>>> https://www.w3schools.com/xml/note.xml
>>>
>>> and I just wanted to store "in" eXist for query.
>>>
>>>
>>> thanks,
>>>
>>> Nick
>>>
>>>
>>> On 11/18/20 5:37 PM, Alasdair Dougall wrote:
>>> > Hi Nicholas,
>>> >
>>> > As Michael said in his reply, a problem to solve is the way to go.
>>> Another source of information is a book called eXist written by Erik Siegel
>>> and Adam Ritter.  It will give you details of eXist.
>>> >
>>> > Alasdair
>>> >
>>> > Sent from my iPad
>>> >
>>> >> On 19 Nov 2020, at 9:56 am, Nicholas <sau...@gm...>
>>> wrote:
>>> >>
>>> >> I'm just looking for a simple starting point.  I don't necessarily
>>> need a GUI, or web app, etc.  Just looking to put an XML document into
>>> eXist and run a few XQuery's against it.  Any pointers?
>>> >>
>>> >>
>>> >> thanks,
>>> >>
>>> >> Nick
>>> >>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Exist-open mailing list
>>> >> Exi...@li...
>>> >> https://lists.sourceforge.net/lists/listinfo/exist-open
>>>
>>>
>>> _______________________________________________
>>> Exist-open mailing list
>>> Exi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>>
>>
>>
>> --
>> Michael Westbay
>> Writer/System Administrator
>> http://www.japanesebaseball.com/
>>
>>
>
> --
> Michael Westbay
> Writer/System Administrator
> http://www.japanesebaseball.com/
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] official docker hub image (Nicholas)

[Duncan P. <du...@ex...>]

Hi Nicholas, 

With the release of exist 5.0.0 we moved to a new build system, and changed the way we build docker images. It lead to smaller docker images, easier customization, and a tighter integration with the testing and release infrastructure of exist. 

Due to a bug in the dockerhub code, the new and updated readme is not displayed along the new images. Hence, I put the warning on the old outdated readme, which contains examples in exist v4.x syntax. 

The updated readme and docker code is in exist’s core repo. The repo containing the old docker stuff has been archived. We cannot delete it yet, as it would break more integrations between docker hub and GitHub, and quickly remove old images we rely on for testing, and which users are depending upon. 

Other then the fact that the readme displayed on dockerhub is stuck with an  outdated version, there are no consequences for users of our docker images whatsoever.   Updates continue to be automagical and under the same stable existdb/existdb:TAG address. No code changes are necessary to interact with the docker registry. 

It will be years before we should consider deleting the old repo, and since very few projects on dockerhub rely on the features that are bugged, a fix anytime soon is very unlikely. If however, the bug is fixed you should immediately see the current readme on dockerhub. 

If you d like to propose changes to the readme in  exist’s core repo or to the warning on the old readme, to make this thing less confusing we would welcome your PR. 

Greetings
Duncan

Sent from my iPad

> Looking at docker hub, the official image documentation says:
> 
> 
> This repository will be archived soon. Images will continue to be 
> available via DockerHub under the old address, but the build will move 
> to eXist-db's core repo.
> 
> 
> what are the implications to this decision?? And, why was it made?