exist-open — Open discussion

Re: [Exist-open] exist multiple cores

[Lars W. <lar...@ex...>]

It’s really weird why the app should be slow on a system with more ghz. There must be some kind of other limitation in the new environment.

With regard to Java 21: Where did you read that eXist-db 6.x would be bound to max Java 17? I know at least two people running eXist-db 6 with Java 21. I’m aware of some discussion about some edge cases that could cause trouble with Java 21 but afaik it’s working fine at least for those two people. Did you ever try it yourself? Nevertheless, if the performance was fine in the old environment but is not in the new one, the Java version will very likely not be the problem because you did not run Java 21 in the old system. 

Sorry I can’t help more but without actually looking in the new and old environment and running a few tests it’s kind of impossible to say, what the issue could be. 

Fingers crossed you can sort it out! 

Best, 

Lars



> On 8. Oct 2025, at 17:34, Eduard Drenth <ed...@fr...> wrote:
> 
> Thanks for the info, good to know about one xquery being bound to one core.
> 
> In the mean time we turned hyperthreading off, that helped, but not enough yet.
> 
> It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
> 
> The queries seem cpu bound as top shows cpu percentage above 200%.
> 
> Increasing memory does not help, IO does not seem to be a problem.
> 
> It is doable for now, but I am happy that new hardware is being ordered, hope that helps.
> 
> Unfortunately I cannot benefit from the many performance improvements in java either, because exist 6 is bound to java 17 max.
> 
> Regards,
> Eduard Drenth, Software Architekt
> 
> ed...@fr...
> 
> Doelestrjitte 8
> 8911 DX  Ljouwert
> (058) 213 14 14
> +31 62 094 34 28
> https://www.fryske-akademy.nl/
> https://frysker.nl/
> https://frisian.eu/<Outlook-d4b4dmlh.svg>
> gpg: https://keyserver.ubuntu.com/pks/lookup?search=eduard+drenth&fingerprint=on&op=index
> From: Lars Windauer <lar...@ex...>
> Sent: Tuesday, October 7, 2025 10:26 PM
> To: Exist-open <Exi...@li...>
> Subject: Re: [Exist-open] exist multiple cores
>  Dear Eduard, 
> 
> eXist-db can benefit from multiple cores without the need to configure anything. Having said this, please be aware that a single XQuery (dbbroker) can always run on only one core! So a single XQuery can’t utilise 20 cores but on a 20 core system you can have 20 parallel queries that each use one core. 
> 
> Can you provide us with information about the old and new environment? Gigahertz of your CPUs, how much RAM does the system have and how much of this is assigned to eXist-db. What about the hard drives, is eXist-db running from a SSD with the same filesystem as in the old environment? 
> 
> Best, 
> 
> Lars
> 
> 
> > On 6. Oct 2025, at 16:42, Eduard Drenth <ed...@fr...> wrote:
> > 
> > Dear all,
> > 
> > We are in the process of migrating exist from our own hardware to a provider.
> > 
> > Performance drops significantly, the main cause is that exist seems to be using one core only.
> > 
> > Can exist benefit from multiple cores? And if yes, how is that configured?
> > 
> > Regards,
> > 
> > Eduard Drenth, Software Architekt
> > 
> > ed...@fr...
> > _______________________________________________
> > Exist-open mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-open
> 
> 
> 
> 
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Pieter L. <pie...@be...>]

Hi Willem,

I would suggest using the latest release or build rather than 6.2.0 and 
checking if the same behavior occurs. My personal experience is that 
lock ups occur when I try to update XQuery resources while they are in 
use. Therefore I do not update the applications while they are running 
production.

Best,
Pieter

On 10/12/2025 1:01 PM, Willem van der Westhuizen wrote:
> Hi
>
> We use existdb extensively to run xqueries on our external databases, 
> couchdb and others. We dont use the existdb datastore for our data.
>
> We used to run on an old 3.3.3 version of existdb, and recently 
> migrated to 6.2. Since then we have found that existdb locks the 
> xquery that is being executed, causing deadlocks that eventually locks 
> up the server. Is there an easy way to tell existdb to revert to the 
> older locking behaviour that does not lock up the xquery being run?
>
>
> Regards
>

-- 
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web: www.benjamins.com

Re: [Exist-open] exist multiple cores

[Adam R. <ad...@ex...>]

Thanks for the info, good to know about one xquery being bound to one core.
>


It's not as simple as that I am afraid. The shared resources within
eXist-db are guarded by locks.

At the dbx file level, eXist-db uses a mutex as a lock, and so can only
ever have 1 thread, that thread can be either reading or writing, but not
both.
At the higher Document and Collection object level, each of those also has
at least one ReadWrite lock, there may be a single writer thread, or
multiple reader threads, but throughput here is limited by the underlying
dbx level when needing to load or save those objects.

If you are not performing store/load operations, then key to performance
can be how you layout your data in documents and collections. Both use a
hierarchical locking scheme along their URI path, so you want to avoid
common parent collections where contention for locks is high.


In the mean time we turned hyperthreading off, that helped, but not enough
> yet.
>

Makes sense. For eXist-db you will want the most performance per-core
possible.


It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
> The queries seem cpu bound as top shows cpu percentage above 200%.
>

It would be interesting to know where that time is spent. Is it lock
contention, or is it meaningful work?



> Increasing memory does not help, IO does not seem to be a problem.
>

If you are CPU bound, changing those worn't help you.


Unfortunately I cannot benefit from the many performance improvements in
> java either, because exist 6 is bound to java 17 max.
>

eXist-db 6 actually requires Java 8.

-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Adam R. <ad...@ex...>]

Hi Willem,

In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is
treated as any other document, and so it is locked for the duration of
operations that happen upon it. Unfortunately this means that whilst a
query is executing the document holding that query will keep a READ lock on
the document.

A document may have many either a single write lock, or one or more read
locks. This means that the query can only be updated by one person/client
at a time, but that multiple clients can run the query in parallel.

I am not sure how a deadlock could occur with this. Can you provide some
further information about your issue, and how to reproduce it please?

Thanks, Adam.

On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen <wi...@kw...>
wrote:

> Hi
>
> We use existdb extensively to run xqueries on our external databases,
> couchdb and others. We dont use the existdb datastore for our data.
>
> We used to run on an old 3.3.3 version of existdb, and recently migrated
> to 6.2. Since then we have found that existdb locks the xquery that is
> being executed, causing deadlocks that eventually locks up the server.
> Is there an easy way to tell existdb to revert to the older locking
> behaviour that does not lock up the xquery being run?
>
>
> Regards
>
> --
> Willem van der Westhuizen
> +27 82 9200718
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] Autocompleting src and href attributes

[Alasdair D. <ala...@gm...>]

Hi Dannes,

You’re in luck, grab yourself a bowl of popcorn and enjoy the movie.

I save using the eXide Save button. You will see the screencast shows the unusual behaviour.

See: https://share.icloud.com/photos/001aTqWO4WgYunQQaVKSTHJdA

Thanks in advance,

Alasdair

Sent from my iPhone

> On 14 Oct 2025, at 6:01 am, Dannes Wessels <di...@ex...> wrote:
> 
> Hi,
> 
>> On 29 Sep 2025, at 03:56, Alasdair Dougall <ala...@gm...> wrote:
>> 
>> The issue is on saving and closing the file, it expand the href to:
> 
> How do you do the save?
> 
> Cheers
> 
> Dannes
> 

Re: [Exist-open] Autocompleting src and href attributes

[Dannes W. <di...@ex...>]

Hi,

> On 29 Sep 2025, at 03:56, Alasdair Dougall <ala...@gm...> wrote:
> 
> The issue is on saving and closing the file, it expand the href to:

How do you do the save?

Cheers

Dannes

[Exist-open] Urgent assistance with locking behaviour of existdb

[Willem v. d. W. <wi...@kw...>]

Hi

We use existdb extensively to run xqueries on our external databases, 
couchdb and others. We dont use the existdb datastore for our data.

We used to run on an old 3.3.3 version of existdb, and recently migrated 
to 6.2. Since then we have found that existdb locks the xquery that is 
being executed, causing deadlocks that eventually locks up the server. 
Is there an easy way to tell existdb to revert to the older locking 
behaviour that does not lock up the xquery being run?


Regards

-- 
Willem van der Westhuizen
+27 82 9200718

Re: [Exist-open] exist multiple cores

[Eduard D. <ed...@fr...>]

Thanks for the info, good to know about one xquery being bound to one core.

In the mean time we turned hyperthreading off, that helped, but not enough yet.

It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.

The queries seem cpu bound as top shows cpu percentage above 200%.

Increasing memory does not help, IO does not seem to be a problem.

It is doable for now, but I am happy that new hardware is being ordered, hope that helps.

Unfortunately I cannot benefit from the many performance improvements in java either, because exist 6 is bound to java 17 max.

Regards,

Eduard Drenth, Software Architekt


ed...@fr...


Doelestrjitte 8

8911 DX  Ljouwert

(058) 213 14 14

+31 62 094 34 28

https://www.fryske-akademy.nl/
https://frysker.nl/
https://frisian.eu/
[cid:1bf8ec89-316a-4d2a-817f-2b6a9ac915db]

gpg: https://keyserver.ubuntu.com/pks/lookup?search=eduard+drenth&fingerprint=on&op=index

________________________________
From: Lars Windauer <lar...@ex...>
Sent: Tuesday, October 7, 2025 10:26 PM
To: Exist-open <Exi...@li...>
Subject: Re: [Exist-open] exist multiple cores

Dear Eduard,

eXist-db can benefit from multiple cores without the need to configure anything. Having said this, please be aware that a single XQuery (dbbroker) can always run on only one core! So a single XQuery can’t utilise 20 cores but on a 20 core system you can have 20 parallel queries that each use one core.

Can you provide us with information about the old and new environment? Gigahertz of your CPUs, how much RAM does the system have and how much of this is assigned to eXist-db. What about the hard drives, is eXist-db running from a SSD with the same filesystem as in the old environment?

Best,

Lars


> On 6. Oct 2025, at 16:42, Eduard Drenth <ed...@fr...> wrote:
>
> Dear all,
>
> We are in the process of migrating exist from our own hardware to a provider.
>
> Performance drops significantly, the main cause is that exist seems to be using one core only.
>
> Can exist benefit from multiple cores? And if yes, how is that configured?
>
> Regards,
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open




_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] exist multiple cores

[Lars W. <lar...@ex...>]

Dear Eduard, 

eXist-db can benefit from multiple cores without the need to configure anything. Having said this, please be aware that a single XQuery (dbbroker) can always run on only one core! So a single XQuery can’t utilise 20 cores but on a 20 core system you can have 20 parallel queries that each use one core. 

Can you provide us with information about the old and new environment? Gigahertz of your CPUs, how much RAM does the system have and how much of this is assigned to eXist-db. What about the hard drives, is eXist-db running from a SSD with the same filesystem as in the old environment? 

Best, 

Lars


> On 6. Oct 2025, at 16:42, Eduard Drenth <ed...@fr...> wrote:
> 
> Dear all,
> 
> We are in the process of migrating exist from our own hardware to a provider.
> 
> Performance drops significantly, the main cause is that exist seems to be using one core only.
> 
> Can exist benefit from multiple cores? And if yes, how is that configured?
> 
> Regards,
> 
> Eduard Drenth, Software Architekt
> 
> ed...@fr...
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

[Exist-open] exist multiple cores

[Eduard D. <ed...@fr...>]

Dear all,

We are in the process of migrating exist from our own hardware to a provider.

Performance drops significantly, the main cause is that exist seems to be using one core only.

Can exist benefit from multiple cores? And if yes, how is that configured?

Regards,


Eduard Drenth, Software Architekt


ed...@fr...

[Exist-open] Reminder Declarative Amsterdam 2025

[Erik S. <er...@xa...>]

Declarative Amsterdam 2025 will be happening at CWI, Amsterdam Science Park on Thursday/Friday November 6 and 7, 2025.

The program has been updated since our previous notification, with additional presentations by Adam Retter and Tomos Hillman.

*	Thursday morning will start with an advanced tutorial on ixml by Steven Pemberton and has a parallel track by Adam Retter about setting up a web API using an XML database.
*	In the afternoon we will have a tutorial on XPath, XSLT and XQuery 4.0 by Juri Leino and John Lumley.
*	We thought it would be appropriate to have Liam Quin's presentation the same day, given that it will be  <https://www.dpconline.org/events/world-digital-preservation-day> World Digital Preservation Day.
*	The symposium will continue on Friday.

The symposium offers a blend of renowned speakers alongside lesser-known experts. To drop a few names (in order of appearance): Bast, Hiep, Agnoletto & Fourny, Hillman, Holman, Middel, Leino, Painter, Couthures, Pemberton, Keith. The presentations cover a wide range of topics, including various techniques, tools, applications, and implementations. Once more an engaging and informative lineup for our attendees, as you can see for yourself at  <https://declarative.amsterdam/program> https://declarative.amsterdam/program

The conference is a hybrid event, live at the Science Park in Amsterdam, and live-streamed.
Registration is open; early bird registration closes October 1st, so hurry along to  <https://declarative.amsterdam/registration> https://declarative.amsterdam/registration. 

All the best,

The Declarative Amsterdam Conference Committee.

 

[Exist-open] Autocompleting src and href attributes

[Alasdair D. <ala...@gm...>]

Hi All,

Recently, I have had eXist eXide do some strange things on saving.  This
was on versions 6.2 and now 6.4  I have my website under
https://www.foodiesite.com, my eXist Dashboard access through
https://admin.foodiesite.com and all works well, until now.

For years I have had relative resource references in app-foodiesite.xqm
module file, In it, I have a few calculated resource locations.  For
example:

 <div class="text-center">
   <a href="/articles/{data($article/dockey)}" class="btn btn-primary
w-75">Click for Article</a>
 </div>

The issue is on saving and closing the file, it expand the href to:

<div class="text-center">
   <a href="*https://admin.foodiesite.com
<https://admin.foodiesite.com>*/articles/{data($article/dockey)}"
class="btn btn-primary w-75">Click for Article</a>
</div>

This is not correct as the nginx mapping for admin is just that admin
dashboard only.

I can work around this by setting the full href path, i.e.;
  ... href="https://www.foodiesite.com/articles/{data($article/dockey)}"

Has anybody else had such fun and games, or am I the lucky 'outlier' of the
group?

Alasdair DougalI

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Claudius T. <cla...@gm...>]

*single space

On Wed, 24 Sep 2025, 13:42 Claudius Teodorescu, <
cla...@gm...> wrote:

> You can also use
>
> /**/
>
> Instead of empty space.
>
> Claudius
>
> On Wed, 24 Sep 2025, 12:45 Henrik Nielsen, <hen...@gm...> wrote:
>
>> Thank you all. I did include the Javascript. To avoid tags being made
>> self-closing when elements are empty I believe I have found a workaround:
>> putting a single space in the otherwise empty element seems to be accepted
>> without subsequent modification by eXist. With regard to the Fore issue
>> itself, I have made a minimal test case that works as it should, so I am
>> building on that instead of what I had previously written.
>>
>> Thanks again,
>>
>> Henrik Thiil NIelsen
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Claudius T. <cla...@gm...>]

You can also use

/**/

Instead of empty space.

Claudius

On Wed, 24 Sep 2025, 12:45 Henrik Nielsen, <hen...@gm...> wrote:

> Thank you all. I did include the Javascript. To avoid tags being made
> self-closing when elements are empty I believe I have found a workaround:
> putting a single space in the otherwise empty element seems to be accepted
> without subsequent modification by eXist. With regard to the Fore issue
> itself, I have made a minimal test case that works as it should, so I am
> building on that instead of what I had previously written.
>
> Thanks again,
>
> Henrik Thiil NIelsen
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Jean-Paul R. <re...@gm...>]

Dear Henrik,

eXist can handle HTML documents in one of two ways:
1. if we want the HTML parsed as nodes and query-able or usable as
templates, it must be XML compliant in format (important: this has nothing
to do with XHTML namespace)
2. otherwise HTML file must be stored as a binary document...in which case
it is no longer nodes and no longer query-able ad it would have to be
streamed to the browser as such

Option 1 means you would have <fx-fore create-nodes=""></fx-fore> in your
document. You can then output it using the following declarations in your
query prolog (i.e. serialization):

declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization
";

declare option output:method "html5";
declare option output:media-type "text/html";

Your html will appear in the browser as <fx-fore create-nodes>.


HTH,
JPR

On Tue, Sep 23, 2025 at 5:23 PM Henrik Nielsen <hen...@gm...>
wrote:

> I have just started using Fore with exist.db and have run into the problem
> that eXide will not accept undefined attributes in Fore's custom elements.
> So "<fx-fore create-nodes>" (without quotation marks in my page) is not
> treated as a valid opening tag since the attribute "create-nodes" is
> expected to be given a value. Does exist.db parse HTML as XML, disregarding
> both the .html extension and the valid doctype declaration ("<!DOCTYPE
> html>")? Or have I made some stupid mistake?
>
> The list archive was offline and I didn't find anything relevant at Stack
> Overflow.
>
> Henrik Thiil Nielsen
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

[Exist-open] Problem with Fore's undefined attributes in eXide

[Henrik N. <hen...@gm...>]

Thank you all. I did include the Javascript. To avoid tags being made
self-closing when elements are empty I believe I have found a workaround:
putting a single space in the otherwise empty element seems to be accepted
without subsequent modification by eXist. With regard to the Fore issue
itself, I have made a minimal test case that works as it should, so I am
building on that instead of what I had previously written.

Thanks again,

Henrik Thiil NIelsen

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[aik01 <ai...@un...>]

Afaik, eXide edits XHTML in your case (see https://www.w3schools.com/html/html_xhtml.asp).
Therefore attribute minimization is not allowed. Beware of selfclosing tags esp in instances because Fore itself will internally parsing your text as html.
The handling of mimetypes within eXistDB is an other thing.  

Peter

On Sunday, September 21, 2025 22:13 CEST, Henrik Nielsen <hen...@gm...> wrote:

> I have just started using Fore with exist.db and have run into the problem
> that eXide will not accept undefined attributes in Fore's custom elements.
> So "<fx-fore create-nodes>" (without quotation marks in my page) is not
> treated as a valid opening tag since the attribute "create-nodes" is
> expected to be given a value. Does exist.db parse HTML as XML, disregarding
> both the .html extension and the valid doctype declaration ("<!DOCTYPE
> html>")? Or have I made some stupid mistake?
> 
> The list archive was offline and I didn't find anything relevant at Stack
> Overflow
> 
> Henrik Thiil Nielsen

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Michael W. <wes...@ja...>]

Hi Henrik,

Are you including the required JavaScript? Based on the documentation on
their GitHub page, the `fx-fore` (and other `fx-` tags) appear to be
processed in the JavaScript on the browser. Any results of these processes
will show up in the browser, not in the eXist database without you
explicitly sending it there.

Hope this helps.

Take care.

2025年9月24日(水) 0:23 Henrik Nielsen <hen...@gm...>:

> I have just started using Fore with exist.db and have run into the problem
> that eXide will not accept undefined attributes in Fore's custom elements.
> So "<fx-fore create-nodes>" (without quotation marks in my page) is not
> treated as a valid opening tag since the attribute "create-nodes" is
> expected to be given a value. Does exist.db parse HTML as XML, disregarding
> both the .html extension and the valid doctype declaration ("<!DOCTYPE
> html>")? Or have I made some stupid mistake?
>
> The list archive was offline and I didn't find anything relevant at Stack
> Overflow.
>
> Henrik Thiil Nielsen
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

Re: [Exist-open] Problem with Fore's undefined attributes in eXide

[Julian J. <jul...@ad...>]

I’m quite sure eXist only parses XML (and therefore, by extension, 
XHTML). A question based on that might be whether eXist should react to 
"<!DOCTYPE html>" by not trying to parse it? But this is off the top of 
my hat.

Best,
Julian

Am 21.09.25 um 22:13 schrieb Henrik Nielsen:
> I have just started using Fore with exist.db and have run into the 
> problem that eXide will not accept undefined attributes in Fore's 
> custom elements. So "<fx-fore create-nodes>" (without quotation marks 
> in my page) is not treated as a valid opening tag since the attribute 
> "create-nodes" is expected to be given a value. Does exist.db parse 
> HTML as XML, disregarding both the .html extension and the valid 
> doctype declaration ("<!DOCTYPE html>")? Or have I made some stupid 
> mistake?
>
> The list archive was offline and I didn't find anything relevant at 
> Stack Overflow.
>
> Henrik Thiil Nielsen
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

[Exist-open] Problem with Fore's undefined attributes in eXide

[Henrik N. <hen...@gm...>]

I have just started using Fore with exist.db and have run into the problem
that eXide will not accept undefined attributes in Fore's custom elements.
So "<fx-fore create-nodes>" (without quotation marks in my page) is not
treated as a valid opening tag since the attribute "create-nodes" is
expected to be given a value. Does exist.db parse HTML as XML, disregarding
both the .html extension and the valid doctype declaration ("<!DOCTYPE
html>")? Or have I made some stupid mistake?

The list archive was offline and I didn't find anything relevant at Stack
Overflow.

Henrik Thiil Nielsen

Re: [Exist-open] Listing documents owned by user

[Alberto S. <am...@zb...>]

Hi, Michael

Thanks, I need to do that programmatically.
The collection has more than 100k documents :-)

Will try your proposal, thanks

Alberto Simões

On Wednesday, August 6th, 2025 at 01:19, Michael Westbay <wes...@ja...> wrote:

> Hi Alberto,
>
> The easiest way is to use the "File" → "Manage" menu item in eXide. That gives you a Finder/File Explorer dialog of all of your collections.
>
> Programmatically, sm:get-permissions(uri) (import module namespace sm="http://exist-db.org/xquery/securitymanager";) will get the owner, group, and mode of a given resource.
>
> Hope this helps.
>
> Take care.
>
> 2025年8月6日(水) 1:00 Alberto Simões <am...@zb...>:
>
>> Hi
>>
>> Is there a way to list all documents in a collection owned by a specific user?
>>
>> Thanks
>>
>> Alberto Simões
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>
> --
>
> Michael Westbay
> Writer/System Administrator
> http://www.japanesebaseball.com/

Re: [Exist-open] Listing documents owned by user

[Juri L. <ju...@ex...>]

On 04.08.25 16:05, Alberto Simões wrote:
> Is there a way to list all documents in a collection owned by a specific user?
>
If I would have to do that:

Search for all resources owned by a specific user in an exist-db instance.


I would use the command line client xst (see 
https://www.npmjs.com/package/@existdb/xst)

With the necessary environment variables set to connect to the database 
you wish to query you can call:

   xst ls /db --long --recursive

This will list all collections and resources, the connecting user has 
access to.

Piping the output into grep allows then to filter by a user name (here 
eXide)

   xst ls /db --long --recursive | grep eXide

Since you want to limit output to a certain collection

   xst ls /db/my/collection --long --recursive | grep eXide



Hope this helps,
Juri Leino

Re: [Exist-open] Listing documents owned by user

[Michael W. <wes...@ja...>]

Hi Alberto,

The easiest way is to use the "File" → "Manage" menu item in eXide. That
gives you a Finder/File Explorer dialog of all of your collections.

Programmatically, sm:get-permissions(uri) (import module namespace sm="
http://exist-db.org/xquery/securitymanager";)  will get the owner, group,
and mode of a given resource.

Hope this helps.

Take care.

2025年8月6日(水) 1:00 Alberto Simões <am...@zb...>:

> Hi
>
> Is there a way to list all documents in a collection owned by a specific
> user?
>
> Thanks
>
>
>
> Alberto Simões
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/

[Exist-open] Listing documents owned by user

[Alberto S. <am...@zb...>]

Hi

Is there a way to list all documents in a collection owned by a specific user?

Thanks



Alberto Simões

Re: [Exist-open] "Collection /db/system/security/exist/accounts cannot be created."

[Michael W. <wes...@ja...>]

Hi Tony,

Okay, the "Failed to open database" message concerns me.

under /usr/local/src/exist3-data do you have the following files?

-rw-rw-r--.  1 exist exist       1451 Jul 25 08:57 00000000d6.log
-rw-rw-r--.  1 exist exist   13991936 Jul 24 22:40 collections.dbx
-rw-r--r--.  1 exist exist       1515 Jul 24 22:38 counters
-rw-rw-r--.  1 exist exist         16 Jul 25 08:57 dbx_dir.lck
-rw-rw-r--.  1 exist exist 6785642496 Jul 24 22:38 dom.dbx
drwxrwxr-x. 15 exist exist       4096 Sep 27  2020 expathrepo
drwxrwxr-x.  3 exist exist        132 May 20 12:14 export
drwxrwxr-x.  3 exist exist         16 Apr  3  2018 fs
drwxrwxr-x.  2 exist exist          6 Jul 21 20:47 fs.journal
-rw-rw-r--.  1 exist exist        140 Apr 16  2018 jmxservlet.token
-rw-rw-r--.  1 exist exist         16 Jul 25 08:57 journal.lck
drwxrwxr-x.  2 exist exist        103 Jul 23  2022 lucene
-rw-rw-r--.  1 exist exist    4636672 May 22 21:46 ngram.dbx
drwxrwxr-x.  2 exist exist       8192 Jul 25 08:44 range
-rw-rw-r--.  1 exist exist        120 Apr  9  2018 README
-rw-rw-r--.  1 exist exist        257 Apr  9  2018 RECOVERY
-rw-------.  1 exist exist         11 Mar  8 10:20 restxq.registry
drwxrwxr-x.  2 exist exist         42 Jul 23  2022 sanity
-rw-rw-r--.  1 exist exist       8192 Jul 23  2022 sort.dbx
-rw-rw-r--.  1 exist exist 6046371840 Jul 24 22:40 structure.dbx
-rw-rw-r--.  1 exist exist      63342 Jul 21 20:35 symbols.dbx
-rw-rw-r--.  1 exist exist   46583808 Jul 24 22:40 values.dbx

The dom.dbx file should be very large. That's where all of your data is at.
If this file is
broken, then the data isn't recoverable.

The RECOVERY file states:

RECOVERY
========
If the index got corrupted, this is how to recover:

- stop database
- remove all ".log" transaction log files
- remove all ".lck" lock files
- remove all ".dbx" files *EXCEPT dom.dbx, collections.dbx and symbols.dbx*
- start database

I really don't like doing it without a backup. So first, make a tarball
of /usr/local/src/exist3-data
as a backup of the raw data then start the recovery process above.

And once the database is up and running again, run:

cd /usr/local/src/exist3
bin/backup.sh --dir ~/tmp/2025-07-25-backup --backup /db

And this is my script modified for your setup that I run daily via crontab:

#!/bin/sh
unset JAVA_OPTS
export JAVA_HOME=/usr
export EXIST_HOME=/usr/local/src/exist3
export TARGET=/tmp
export DATE=`date +"%Y-%m-%d"`
export PATH=$JAVA_HOME/bin:/usr/local/bin:/usr/bin:/bin
cd $EXIST_HOME
bin/backup.sh --backup /db --dir $TARGET/full-$DATE -u "admin" -p "password"

I thought it was supposed to get the username and password from the
$EXIST_HOME/backup.properties
file, but when I tested it by hand just now without the -u/-p parameters it
failed. Not sure why.

Anyway, daily backups going forward would be a very good idea.

Take care.

2025年7月24日(木) 23:21 Tony Graham <tg...@an...>:

> Michael,
>
> Thanks for following through with this.
>
> On 23/07/2025 09:00, Michael Westbay wrote:
> > Okay. Let's first find EXIST_HOME by finding where the conf.xml file is.
> >
> > find /usr/local -name conf.xml
>
> It found multiple, but the ones of interest are:
>
> /usr/local/src/exist3/installer/conf.xml
> /usr/local/src/exist3/conf.xml
>
> > If /usr/local is EXIST_HOME, then run.sh will be in /usr/local/bin along
> > with all of the other eXist executables.
>
> The only things in '/usr/local/bin' are 'tracd' and 'trac-admin' for use
> with Trac. (Still the best issue tracking/wiki system.)
>
> > This conf.xml file will then specify where the data directory is:
>
> '/usr/local/src/exist3/conf.xml' has:
>
>      <db-connection cacheSize="256M" checkMaxCacheSize="true"
> collectionCache="64M" database="native"
>          files="/usr/local/src/exist3-data" pageSize="4096"
> nodesBuffer="1000" cacheShrinkThreshold="10000"
>          doc-ids="default" minDiskSpace="128M">
>
> > You could also search for run.sh the same way:
> >
> > find /usr/local -name run.sh
>
> As before, I don't have a 'run.sh' (unless you count 'run.sh' for
> Antenna House Formatter).
>
> Running the Java installer as a non-root user has never given me a
> 'run.sh'.
>
> I downloaded the 'run.sh' [1] for that eXist-db version from GitHub,
> edited it to use the same Java alternative as used for the database, ran
> it, and got:
>
> ----
> cd "/usr/local/src/exist3/" ; ./bin/run.sh org.exist.backup.ExportMain
> -x -d /mnt/tmp
> ERROR: Failed to open database:
> org.exist.storage.BrokerPoolServiceException: org.exist.EXistException:
> java.io.IOException: Collection /db/system/security/exist/accounts
> cannot be created.
> ----
>
> If this can be solved, the problem then becomes that we don't know how
> many other collections will have similar problems, so I'm not hopeful.
>
> Regards,
>
>
> Tony Graham.
> --
> Senior Architect
> XML Division
> Antenna House, Inc.
> ----
> Skerries, Ireland
> tg...@an...
>
> [1]
>
> https://raw.githubusercontent.com/eXist-db/exist/refs/tags/eXist-3.0.RC2/bin/run.sh
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Michael Westbay
Writer/System Administrator
http://www.japanesebaseball.com/