exist-open — Open discussion

Re: [Exist-open] exist multiple cores

[Craig B. <cra...@ma...>]

> On Oct 16, 2025, at 10:33 AM, Eduard Drenth <ed...@FR...> wrote:
> 
> Have been trying to upload a 386M document into exist 6.4.0 in order to test query performance compared to querying 70000 separate documents. 
> 
> Question before I continue trying different settings/jvm's/etc.:
> 
> Is querying one large document expected to be significantly faster compared to querying many separate small documents?


The answer to that very likely depends on how many nodes you have, what kind of indexes you have, and what kind of queries you're running.  In my experience, accessing nodes later in a large document using the structural index is much slower than accessing nodes earlier in the same document.  I demonstrated that for queries using the preceding and following axes here:

https://github.com/eXist-db/exist/issues/2129
wildcard following and preceding axes are slow · Issue #2129 · eXist-db/exist
github.com



________________________________________
Craig A. Berry

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser

Re: [Exist-open] Urgent assistance with locking behaviour of existdb - or how to change the architecture of a scheduled ETL service

[Willem v. d. W. <wi...@kw...>]

Hi,

The etl engine is an existdb module, that is called by an xquery. One 
hypothesis is that each time that the xquery runs, it loads an instance 
of the module into memory, and thats is where the locking happens. No 
way to tell if that is indeed the case, and what the internal limiations 
are.

The one interesting observation is that even in older version 3.3.3 if 
the xquery is pending up due to the remote service being called is 
stalled, then the number of instances in exists always stop responding 
at around 160 plus or minus, then the whole exist instance locks up. The 
number of database brokers, and memory etc. was never a limit.

In the later versions of exist the deadlock may occur at much lover 
levels. To give an example, we have rewritten most of the critical code 
that locked up in javascript, and there was no issue in the downstream 
services.

The explanation offered of recursive calling of modules might be a clue, 
because that is certainly the case in our code. We would prefer to keep 
the ETL in exist though, hence trying to find a solution.

Regards

Willem

On 2025/10/18 11:38, Claudius Teodorescu wrote:
> Hi!
>
> Would it help to have the ETL engine as an eXist-db module?
>
>
> Claudius Teodorescu
>
> On Sat, 18 Oct 2025 at 11:29, Willem van der Westhuizen 
> <wi...@kw...> wrote:
>
>     Maybe there is a different question to ask. We are using exist and
>     xquery extensively with an etl engine that we wrote to read data from
>     couchdb and place into MySQL for reporting purposes. We are currently
>     using a shell script to launch 20 instances of the the etl with
>     different parameters to pick up a batch and process it. Would
>     there be a
>     better way in exist to write the scheduler to make those calls,
>     and then
>     to wait for each instance to be complete before calling the next,
>     that
>     would not involve calling the same rest service muliple times from
>     curl?
>
>     Regards
>
>     Willem
>
>     On 2025/10/15 11:48, Willem van der Westhuizen wrote:
>     > Definitely not updating. Just running,
>     >
>     > On 2025/10/15 10:30, Juri Leino wrote:
>     >> Hi Willem!
>     >>
>     >> The behaviour you are describing is usually a sign for an
>     attempt to
>     >> replace/update a running XQuery.  If this is the case the easiest
>     >> solution is to make sure that the query you want to replace /
>     change
>     >> is not being run at the moment.
>     >>
>     >> I agree that we should investigate if we even have to lock the
>     XQuery
>     >> in the first place. But that is a separate discussion.
>     >>
>     >> Regards,
>     >> Juri
>     >>
>     >> On 12.10.25 13:01, Willem van der Westhuizen wrote:
>     >>> Hi
>     >>>
>     >>> We use existdb extensively to run xqueries on our external
>     >>> databases, couchdb and others. We dont use the existdb
>     datastore for
>     >>> our data.
>     >>>
>     >>> We used to run on an old 3.3.3 version of existdb, and recently
>     >>> migrated to 6.2. Since then we have found that existdb locks the
>     >>> xquery that is being executed, causing deadlocks that eventually
>     >>> locks up the server. Is there an easy way to tell existdb to
>     revert
>     >>> to the older locking behaviour that does not lock up the xquery
>     >>> being run?
>     >>>
>     >>>
>     >>> Regards
>     >>>
>     >>
>     >>
>     >> _______________________________________________
>     >> Exist-open mailing list
>     >> Exi...@li...
>     >> https://lists.sourceforge.net/lists/listinfo/exist-open
>     >
>     >
>     > _______________________________________________
>     > Exist-open mailing list
>     > Exi...@li...
>     > https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>     _______________________________________________
>     Exist-open mailing list
>     Exi...@li...
>     https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
> -- 
> Cu stimă,
> Claudius Teodorescu

Re: [Exist-open] Urgent assistance with locking behaviour of existdb - or how to change the architecture of a scheduled ETL service

[Claudius T. <cla...@gm...>]

Hi!

Would it help to have the ETL engine as an eXist-db module?


Claudius Teodorescu

On Sat, 18 Oct 2025 at 11:29, Willem van der Westhuizen <wi...@kw...>
wrote:

> Maybe there is a different question to ask. We are using exist and
> xquery extensively with an etl engine that we wrote to read data from
> couchdb and place into MySQL for reporting purposes. We are currently
> using a shell script to launch 20 instances of the the etl with
> different parameters to pick up a batch and process it. Would there be a
> better way in exist to write the scheduler to make those calls, and then
> to wait for each instance to be complete before calling the next, that
> would not involve calling the same rest service muliple times from curl?
>
> Regards
>
> Willem
>
> On 2025/10/15 11:48, Willem van der Westhuizen wrote:
> > Definitely not updating. Just running,
> >
> > On 2025/10/15 10:30, Juri Leino wrote:
> >> Hi Willem!
> >>
> >> The behaviour you are describing is usually a sign for an attempt to
> >> replace/update a running XQuery.  If this is the case the easiest
> >> solution is to make sure that the query you want to replace / change
> >> is not being run at the moment.
> >>
> >> I agree that we should investigate if we even have to lock the XQuery
> >> in the first place. But that is a separate discussion.
> >>
> >> Regards,
> >> Juri
> >>
> >> On 12.10.25 13:01, Willem van der Westhuizen wrote:
> >>> Hi
> >>>
> >>> We use existdb extensively to run xqueries on our external
> >>> databases, couchdb and others. We dont use the existdb datastore for
> >>> our data.
> >>>
> >>> We used to run on an old 3.3.3 version of existdb, and recently
> >>> migrated to 6.2. Since then we have found that existdb locks the
> >>> xquery that is being executed, causing deadlocks that eventually
> >>> locks up the server. Is there an easy way to tell existdb to revert
> >>> to the older locking behaviour that does not lock up the xquery
> >>> being run?
> >>>
> >>>
> >>> Regards
> >>>
> >>
> >>
> >> _______________________________________________
> >> Exist-open mailing list
> >> Exi...@li...
> >> https://lists.sourceforge.net/lists/listinfo/exist-open
> >
> >
> > _______________________________________________
> > Exist-open mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Cu stimă,
Claudius Teodorescu

Re: [Exist-open] exist multiple cores

[Eduard D. <ed...@fr...>]

Dear all,

Luckily I found settings that boost performance for our situation. I share them so they can be of use.

(My question about one large versus many small remains, many small will I think make collection() a lot faster.)

The most important is caching collection (in our case only changing with a new docker build):

declare variable $teidictjson:data :=
     if ($teidictjson:collcache) then
        if (fn:not(fn:empty(cache:get("teidictjson","coll")))) then
            cache:get("teidictjson","coll")
        else (let $r := cache:put("teidictjson","coll",collection($config:data-root))
                return cache:get("teidictjson","coll")
              )
     else collection($config:data-root);

docker-compose.yml, I suspect less memory would do as well:

   deploy:
      replicas: 1
      resources:
        limits:
          memory: "8GB"
      restart_policy:
        condition: any
    environment:
      # NOTE This overrides the maintained JAVA_TOOL_OPTIONS in the Dockerfile from the exist-db community
      # This is the only way to override settings, for example ARG CACHE_MEM=1024 in your Dockerfile won't work, neither will setting cache in conf.xml
      - JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8 -Dsun.jnu.encoding=UTF-8 -Djava.awt.headless=true -Dorg.exist.db-connection.cacheSize=2048M -Dorg.exist.db-connection.pool.max=20 -Dlog4j.configurationFile=/exist/etc/log4j2.xml -Dexist.home=/exist -Dexist.configurationFile=/exist/etc/conf.xml -Djetty.home=/exist -Dexist.jetty.config=/exist/etc/jetty/standard.enabled-jetty-configs -XX:+UseG1GC -XX:+UseStringDeduplication -XX:+ExitOnOutOfMemoryError -XX:MaxRAMPercentage=75.0

Regards, Eduard


________________________________
From: Eduard Drenth <ed...@fr...>
Sent: Thursday, October 16, 2025 5:33 PM
To: Lars Windauer <lar...@ex...>; Exist-open <Exi...@li...>
Subject: Re: [Exist-open] exist multiple cores

Have been trying to upload a 386M document into exist 6.4.0 in order to test query performance compared to querying 70000 separate documents.

Question before I continue trying different settings/jvm's/etc.:


  *
Is querying one large document expected to be significantly faster compared to querying many separate small documents?

Regards, Eduard
________________________________
From: Eduard Drenth <ed...@fr...>
Sent: Thursday, October 16, 2025 6:46 AM
To: Lars Windauer <lar...@ex...>; Exist-open <Exi...@li...>
Subject: Re: [Exist-open] exist multiple cores

Thanks, I understand it's hard to tell yes.

Could try this https://github.com/duncdrum/distroless-exist/blob/main/Dockerfile for java 21 with exist 6.4. But until now I rely on what is in docker hub instead of building images myself.

Could queries be faster when I merge ± 70.000 seperate TEI documents into one teiCorpus document?

Regards, Eduard
________________________________
From: Lars Windauer <lar...@ex...>
Sent: Tuesday, October 14, 2025 12:28 PM
To: Eduard Drenth <ed...@fr...>; Exist-open <Exi...@li...>
Subject: Re: [Exist-open] exist multiple cores

It’s really weird why the app should be slow on a system with more ghz. There must be some kind of other limitation in the new environment.

With regard to Java 21: Where did you read that eXist-db 6.x would be bound to max Java 17? I know at least two people running eXist-db 6 with Java 21. I’m aware of some discussion about some edge cases that could cause trouble with Java 21 but afaik it’s working fine at least for those two people. Did you ever try it yourself? Nevertheless, if the performance was fine in the old environment but is not in the new one, the Java version will very likely not be the problem because you did not run Java 21 in the old system.

Sorry I can’t help more but without actually looking in the new and old environment and running a few tests it’s kind of impossible to say, what the issue could be.

Fingers crossed you can sort it out!

Best,

Lars



> On 8. Oct 2025, at 17:34, Eduard Drenth <ed...@fr...> wrote:
>
> Thanks for the info, good to know about one xquery being bound to one core.
>
> In the mean time we turned hyperthreading off, that helped, but not enough yet.
>
> It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
>
> The queries seem cpu bound as top shows cpu percentage above 200%.
>
> Increasing memory does not help, IO does not seem to be a problem.
>
> It is doable for now, but I am happy that new hardware is being ordered, hope that helps.
>
> Unfortunately I cannot benefit from the many performance improvements in java either, because exist 6 is bound to java 17 max.
>
> Regards,
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> (058) 213 14 14
> +31 62 094 34 28
> https://www.fryske-akademy.nl/
> https://frysker.nl/
> https://frisian.eu/<Outlook-d4b4dmlh.svg>
> gpg: https://keyserver.ubuntu.com/pks/lookup?search=eduard+drenth&fingerprint=on&op=index
> From: Lars Windauer <lar...@ex...>
> Sent: Tuesday, October 7, 2025 10:26 PM
> To: Exist-open <Exi...@li...>
> Subject: Re: [Exist-open] exist multiple cores
>  Dear Eduard,
>
> eXist-db can benefit from multiple cores without the need to configure anything. Having said this, please be aware that a single XQuery (dbbroker) can always run on only one core! So a single XQuery can’t utilise 20 cores but on a 20 core system you can have 20 parallel queries that each use one core.
>
> Can you provide us with information about the old and new environment? Gigahertz of your CPUs, how much RAM does the system have and how much of this is assigned to eXist-db. What about the hard drives, is eXist-db running from a SSD with the same filesystem as in the old environment?
>
> Best,
>
> Lars
>
>
> > On 6. Oct 2025, at 16:42, Eduard Drenth <ed...@fr...> wrote:
> >
> > Dear all,
> >
> > We are in the process of migrating exist from our own hardware to a provider.
> >
> > Performance drops significantly, the main cause is that exist seems to be using one core only.
> >
> > Can exist benefit from multiple cores? And if yes, how is that configured?
> >
> > Regards,
> >
> > Eduard Drenth, Software Architekt
> >
> > ed...@fr...
> > _______________________________________________
> > Exist-open mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] exist multiple cores

[Eduard D. <ed...@fr...>]

Have been trying to upload a 386M document into exist 6.4.0 in order to test query performance compared to querying 70000 separate documents.

Question before I continue trying different settings/jvm's/etc.:


  *
Is querying one large document expected to be significantly faster compared to querying many separate small documents?

Regards, Eduard
________________________________
From: Eduard Drenth <ed...@fr...>
Sent: Thursday, October 16, 2025 6:46 AM
To: Lars Windauer <lar...@ex...>; Exist-open <Exi...@li...>
Subject: Re: [Exist-open] exist multiple cores

Thanks, I understand it's hard to tell yes.

Could try this https://github.com/duncdrum/distroless-exist/blob/main/Dockerfile for java 21 with exist 6.4. But until now I rely on what is in docker hub instead of building images myself.

Could queries be faster when I merge ± 70.000 seperate TEI documents into one teiCorpus document?

Regards, Eduard
________________________________
From: Lars Windauer <lar...@ex...>
Sent: Tuesday, October 14, 2025 12:28 PM
To: Eduard Drenth <ed...@fr...>; Exist-open <Exi...@li...>
Subject: Re: [Exist-open] exist multiple cores

It’s really weird why the app should be slow on a system with more ghz. There must be some kind of other limitation in the new environment.

With regard to Java 21: Where did you read that eXist-db 6.x would be bound to max Java 17? I know at least two people running eXist-db 6 with Java 21. I’m aware of some discussion about some edge cases that could cause trouble with Java 21 but afaik it’s working fine at least for those two people. Did you ever try it yourself? Nevertheless, if the performance was fine in the old environment but is not in the new one, the Java version will very likely not be the problem because you did not run Java 21 in the old system.

Sorry I can’t help more but without actually looking in the new and old environment and running a few tests it’s kind of impossible to say, what the issue could be.

Fingers crossed you can sort it out!

Best,

Lars



> On 8. Oct 2025, at 17:34, Eduard Drenth <ed...@fr...> wrote:
>
> Thanks for the info, good to know about one xquery being bound to one core.
>
> In the mean time we turned hyperthreading off, that helped, but not enough yet.
>
> It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
>
> The queries seem cpu bound as top shows cpu percentage above 200%.
>
> Increasing memory does not help, IO does not seem to be a problem.
>
> It is doable for now, but I am happy that new hardware is being ordered, hope that helps.
>
> Unfortunately I cannot benefit from the many performance improvements in java either, because exist 6 is bound to java 17 max.
>
> Regards,
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> (058) 213 14 14
> +31 62 094 34 28
> https://www.fryske-akademy.nl/
> https://frysker.nl/
> https://frisian.eu/<Outlook-d4b4dmlh.svg>
> gpg: https://keyserver.ubuntu.com/pks/lookup?search=eduard+drenth&fingerprint=on&op=index
> From: Lars Windauer <lar...@ex...>
> Sent: Tuesday, October 7, 2025 10:26 PM
> To: Exist-open <Exi...@li...>
> Subject: Re: [Exist-open] exist multiple cores
>  Dear Eduard,
>
> eXist-db can benefit from multiple cores without the need to configure anything. Having said this, please be aware that a single XQuery (dbbroker) can always run on only one core! So a single XQuery can’t utilise 20 cores but on a 20 core system you can have 20 parallel queries that each use one core.
>
> Can you provide us with information about the old and new environment? Gigahertz of your CPUs, how much RAM does the system have and how much of this is assigned to eXist-db. What about the hard drives, is eXist-db running from a SSD with the same filesystem as in the old environment?
>
> Best,
>
> Lars
>
>
> > On 6. Oct 2025, at 16:42, Eduard Drenth <ed...@fr...> wrote:
> >
> > Dear all,
> >
> > We are in the process of migrating exist from our own hardware to a provider.
> >
> > Performance drops significantly, the main cause is that exist seems to be using one core only.
> >
> > Can exist benefit from multiple cores? And if yes, how is that configured?
> >
> > Regards,
> >
> > Eduard Drenth, Software Architekt
> >
> > ed...@fr...
> > _______________________________________________
> > Exist-open mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] exist multiple cores

[Eduard D. <ed...@fr...>]

Thanks, I understand it's hard to tell yes.

Could try this https://github.com/duncdrum/distroless-exist/blob/main/Dockerfile for java 21 with exist 6.4. But until now I rely on what is in docker hub instead of building images myself.

Could queries be faster when I merge ± 70.000 seperate TEI documents into one teiCorpus document?

Regards, Eduard
________________________________
From: Lars Windauer <lar...@ex...>
Sent: Tuesday, October 14, 2025 12:28 PM
To: Eduard Drenth <ed...@fr...>; Exist-open <Exi...@li...>
Subject: Re: [Exist-open] exist multiple cores

It’s really weird why the app should be slow on a system with more ghz. There must be some kind of other limitation in the new environment.

With regard to Java 21: Where did you read that eXist-db 6.x would be bound to max Java 17? I know at least two people running eXist-db 6 with Java 21. I’m aware of some discussion about some edge cases that could cause trouble with Java 21 but afaik it’s working fine at least for those two people. Did you ever try it yourself? Nevertheless, if the performance was fine in the old environment but is not in the new one, the Java version will very likely not be the problem because you did not run Java 21 in the old system.

Sorry I can’t help more but without actually looking in the new and old environment and running a few tests it’s kind of impossible to say, what the issue could be.

Fingers crossed you can sort it out!

Best,

Lars



> On 8. Oct 2025, at 17:34, Eduard Drenth <ed...@fr...> wrote:
>
> Thanks for the info, good to know about one xquery being bound to one core.
>
> In the mean time we turned hyperthreading off, that helped, but not enough yet.
>
> It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
>
> The queries seem cpu bound as top shows cpu percentage above 200%.
>
> Increasing memory does not help, IO does not seem to be a problem.
>
> It is doable for now, but I am happy that new hardware is being ordered, hope that helps.
>
> Unfortunately I cannot benefit from the many performance improvements in java either, because exist 6 is bound to java 17 max.
>
> Regards,
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> (058) 213 14 14
> +31 62 094 34 28
> https://www.fryske-akademy.nl/
> https://frysker.nl/
> https://frisian.eu/<Outlook-d4b4dmlh.svg>
> gpg: https://keyserver.ubuntu.com/pks/lookup?search=eduard+drenth&fingerprint=on&op=index
> From: Lars Windauer <lar...@ex...>
> Sent: Tuesday, October 7, 2025 10:26 PM
> To: Exist-open <Exi...@li...>
> Subject: Re: [Exist-open] exist multiple cores
>  Dear Eduard,
>
> eXist-db can benefit from multiple cores without the need to configure anything. Having said this, please be aware that a single XQuery (dbbroker) can always run on only one core! So a single XQuery can’t utilise 20 cores but on a 20 core system you can have 20 parallel queries that each use one core.
>
> Can you provide us with information about the old and new environment? Gigahertz of your CPUs, how much RAM does the system have and how much of this is assigned to eXist-db. What about the hard drives, is eXist-db running from a SSD with the same filesystem as in the old environment?
>
> Best,
>
> Lars
>
>
> > On 6. Oct 2025, at 16:42, Eduard Drenth <ed...@fr...> wrote:
> >
> > Dear all,
> >
> > We are in the process of migrating exist from our own hardware to a provider.
> >
> > Performance drops significantly, the main cause is that exist seems to be using one core only.
> >
> > Can exist benefit from multiple cores? And if yes, how is that configured?
> >
> > Regards,
> >
> > Eduard Drenth, Software Architekt
> >
> > ed...@fr...
> > _______________________________________________
> > Exist-open mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Willem v. d. W. <wi...@kw...>]

that the one in image that I shared? or how should I find it?

Willem

On 2025/10/15 13:55, Adam Retter wrote:
>
>     In our case we are not updating when the locking occurs.
>
>
> If all of your operations are read-only then it should not be possible 
> for a deadlock to occur.
> Would it be possible to see a dump of the lock table when that happens 
> Willem?
>
> -- 
> Adam Retter
>
> eXist Core Developer
> { United Kingdom }
> ad...@ex...

Re: [Exist-open] Urgent assistance with locking behaviour of existdb / deadlocking

[Pieter L. <pie...@be...>]

I don't know how to get that dump, would that be with jstack {pid} > 
C:\jstack.txt? I am not too keen on breaking my instance as I have a lot 
to do with it. Also, exist completely freezes so I do not know whether 
it would be possible to extract anything.

-p

On 10/15/2025 1:55 PM, Adam Retter wrote:
> Would it be possible to see a dump of the lock table when that happens 
> Pieter?
>
> On Wed, 15 Oct 2025 at 12:52, Pieter Lamers via Exist-open 
> <exi...@li...> wrote:
>
>     Hi Adam,
>
>     It is not trivial to create a reproduceable case for the
>     deadlocking problem but I am seeing deadlocks all the time when I
>     do not take the necessary precautions when updating. I think it
>     has to do with module imports. I think the deadlock occurs when
>     the update of a library module is attempted which is also holding
>     a read-lock because of an active process. Or the other way around:
>     you make a request which involves a library which is being
>     updated. At least one of these scenarios and maybe both can break
>     my setup.
>
>     This does not seem to be Willem's problem tho.
>
>     Best,
>     Pieter
>
>     On 10/14/2025 9:59 AM, Adam Retter wrote:
>>
>>     Hi Willem,
>>
>>     In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the
>>     database) is treated as any other document, and so it is locked
>>     for the duration of operations that happen upon it. Unfortunately
>>     this means that whilst a query is executing the document holding
>>     that query will keep a READ lock on the document.
>>
>>     A document may have many either a single write lock, or one or
>>     more read locks. This means that the query can only be updated by
>>     one person/client at a time, but that multiple clients can run
>>     the query in parallel.
>>
>>     I am not sure how a deadlock could occur with this. Can you
>>     provide some further information about your issue, and how to
>>     reproduce it please?
>>
>>     Thanks, Adam.
>>
>>     On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen
>>     <wi...@kw...> wrote:
>>
>>         Hi
>>
>>         We use existdb extensively to run xqueries on our external
>>         databases,
>>         couchdb and others. We dont use the existdb datastore for our
>>         data.
>>
>>         We used to run on an old 3.3.3 version of existdb, and
>>         recently migrated
>>         to 6.2. Since then we have found that existdb locks the
>>         xquery that is
>>         being executed, causing deadlocks that eventually locks up
>>         the server.
>>         Is there an easy way to tell existdb to revert to the older
>>         locking
>>         behaviour that does not lock up the xquery being run?
>>
>>
>>         Regards
>>
>>         -- 
>>         Willem van der Westhuizen
>>         +27 82 9200718
>>
>>
>>
>>         _______________________________________________
>>         Exist-open mailing list
>>         Exi...@li...
>>         https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>>
>>
>>     -- 
>>     Adam Retter
>>
>>     eXist Core Developer
>>     { United Kingdom }
>>     ad...@ex...
>>
>>
>>     _______________________________________________
>>     Exist-open mailing list
>>     Exi...@li...
>>     https://lists.sourceforge.net/lists/listinfo/exist-open
>
>     -- 
>     Pieter Lamers
>     John Benjamins Publishing Company
>     Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
>     Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
>     Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
>     tel: +31 20 630 4747
>     web:www.benjamins.com <http://www.benjamins.com>
>
>
>     _______________________________________________
>     Exist-open mailing list
>     Exi...@li...
>     https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
> -- 
> Adam Retter
>
> eXist Core Developer
> { United Kingdom }
> ad...@ex...

-- 
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web:www.benjamins.com

Re: [Exist-open] Urgent assistance with locking behaviour of existdb / deadlocking

[Adam R. <ad...@ex...>]

Would it be possible to see a dump of the lock table when that happens
Pieter?

On Wed, 15 Oct 2025 at 12:52, Pieter Lamers via Exist-open <
exi...@li...> wrote:

> Hi Adam,
>
> It is not trivial to create a reproduceable case for the deadlocking
> problem but I am seeing deadlocks all the time when I do not take the
> necessary precautions when updating. I think it has to do with module
> imports. I think the deadlock occurs when the update of a library module is
> attempted which is also holding a read-lock because of an active process.
> Or the other way around: you make a request which involves a library which
> is being updated. At least one of these scenarios and maybe both can break
> my setup.
>
> This does not seem to be Willem's problem tho.
>
> Best,
> Pieter
>
> On 10/14/2025 9:59 AM, Adam Retter wrote:
>
>
> Hi Willem,
>
> In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is
> treated as any other document, and so it is locked for the duration of
> operations that happen upon it. Unfortunately this means that whilst a
> query is executing the document holding that query will keep a READ lock on
> the document.
>
> A document may have many either a single write lock, or one or more read
> locks. This means that the query can only be updated by one person/client
> at a time, but that multiple clients can run the query in parallel.
>
> I am not sure how a deadlock could occur with this. Can you provide some
> further information about your issue, and how to reproduce it please?
>
> Thanks, Adam.
>
> On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen <wi...@kw...>
> wrote:
>
>> Hi
>>
>> We use existdb extensively to run xqueries on our external databases,
>> couchdb and others. We dont use the existdb datastore for our data.
>>
>> We used to run on an old 3.3.3 version of existdb, and recently migrated
>> to 6.2. Since then we have found that existdb locks the xquery that is
>> being executed, causing deadlocks that eventually locks up the server.
>> Is there an easy way to tell existdb to revert to the older locking
>> behaviour that does not lock up the xquery being run?
>>
>>
>> Regards
>>
>> --
>> Willem van der Westhuizen
>> +27 82 9200718
>>
>>
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>
>
> --
> Adam Retter
>
> eXist Core Developer
> { United Kingdom }
> ad...@ex...
>
>
> _______________________________________________
> Exist-open mailing lis...@li...://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> --
> Pieter Lamers
> John Benjamins Publishing Company
> Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
> Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
> Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
> tel: +31 20 630 4747
> web: www.benjamins.com
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Adam R. <ad...@ex...>]

> My personal experience is that
> lock ups occur when I try to update XQuery resources while they are in
> use. Therefore I do not update the applications while they are running
> production.
>

Theoretically that should not be possible as the XQuery whislt it is
executing will hold a READ lock. Whilst the READ lock is held, any requests
to update the XQuery will wait until the READ lock is released, each
pending update operation will then be executed one after another, with each
one taking the WRITE lock.

If there is a deadlock, there must be more resources involved than just the
XQuery.

Would it be possible to see a dump of the lock table when that happens
Pieter?

-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Adam R. <ad...@ex...>]

>
> In our case we are not updating when the locking occurs.
>

If all of your operations are read-only then it should not be possible for
a deadlock to occur.
Would it be possible to see a dump of the lock table when that happens
Willem?


-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] Urgent assistance with locking behaviour of existdb - or how to change the architecture of a scheduled ETL service

[Willem v. d. W. <wi...@kw...>]

Maybe there is a different question to ask. We are using exist and 
xquery extensively with an etl engine that we wrote to read data from 
couchdb and place into MySQL for reporting purposes. We are currently 
using a shell script to launch 20 instances of the the etl with 
different parameters to pick up a batch and process it. Would there be a 
better way in exist to write the scheduler to make those calls, and then 
to wait for each instance to be complete before calling the next, that 
would not involve calling the same rest service muliple times from curl?

Regards

Willem

On 2025/10/15 11:48, Willem van der Westhuizen wrote:
> Definitely not updating. Just running,
>
> On 2025/10/15 10:30, Juri Leino wrote:
>> Hi Willem!
>>
>> The behaviour you are describing is usually a sign for an attempt to 
>> replace/update a running XQuery.  If this is the case the easiest 
>> solution is to make sure that the query you want to replace / change 
>> is not being run at the moment.
>>
>> I agree that we should investigate if we even have to lock the XQuery 
>> in the first place. But that is a separate discussion.
>>
>> Regards,
>> Juri
>>
>> On 12.10.25 13:01, Willem van der Westhuizen wrote:
>>> Hi
>>>
>>> We use existdb extensively to run xqueries on our external 
>>> databases, couchdb and others. We dont use the existdb datastore for 
>>> our data.
>>>
>>> We used to run on an old 3.3.3 version of existdb, and recently 
>>> migrated to 6.2. Since then we have found that existdb locks the 
>>> xquery that is being executed, causing deadlocks that eventually 
>>> locks up the server. Is there an easy way to tell existdb to revert 
>>> to the older locking behaviour that does not lock up the xquery 
>>> being run?
>>>
>>>
>>> Regards
>>>
>>
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Willem v. d. W. <wi...@kw...>]

Definitely not updating. Just running,

On 2025/10/15 10:30, Juri Leino wrote:
> Hi Willem!
>
> The behaviour you are describing is usually a sign for an attempt to 
> replace/update a running XQuery.  If this is the case the easiest 
> solution is to make sure that the query you want to replace / change 
> is not being run at the moment.
>
> I agree that we should investigate if we even have to lock the XQuery 
> in the first place. But that is a separate discussion.
>
> Regards,
> Juri
>
> On 12.10.25 13:01, Willem van der Westhuizen wrote:
>> Hi
>>
>> We use existdb extensively to run xqueries on our external databases, 
>> couchdb and others. We dont use the existdb datastore for our data.
>>
>> We used to run on an old 3.3.3 version of existdb, and recently 
>> migrated to 6.2. Since then we have found that existdb locks the 
>> xquery that is being executed, causing deadlocks that eventually 
>> locks up the server. Is there an easy way to tell existdb to revert 
>> to the older locking behaviour that does not lock up the xquery being 
>> run?
>>
>>
>> Regards
>>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Urgent assistance with locking behaviour of existdb / deadlocking

[Pieter L. <pie...@be...>]

Hi Adam,

It is not trivial to create a reproduceable case for the deadlocking 
problem but I am seeing deadlocks all the time when I do not take the 
necessary precautions when updating. I think it has to do with module 
imports. I think the deadlock occurs when the update of a library module 
is attempted which is also holding a read-lock because of an active 
process. Or the other way around: you make a request which involves a 
library which is being updated. At least one of these scenarios and 
maybe both can break my setup.

This does not seem to be Willem's problem tho.

Best,
Pieter

On 10/14/2025 9:59 AM, Adam Retter wrote:
>
> Hi Willem,
>
> In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is 
> treated as any other document, and so it is locked for the duration of 
> operations that happen upon it. Unfortunately this means that whilst a 
> query is executing the document holding that query will keep a READ 
> lock on the document.
>
> A document may have many either a single write lock, or one or more 
> read locks. This means that the query can only be updated by one 
> person/client at a time, but that multiple clients can run the query 
> in parallel.
>
> I am not sure how a deadlock could occur with this. Can you provide 
> some further information about your issue, and how to reproduce it please?
>
> Thanks, Adam.
>
> On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen 
> <wi...@kw...> wrote:
>
>     Hi
>
>     We use existdb extensively to run xqueries on our external databases,
>     couchdb and others. We dont use the existdb datastore for our data.
>
>     We used to run on an old 3.3.3 version of existdb, and recently
>     migrated
>     to 6.2. Since then we have found that existdb locks the xquery
>     that is
>     being executed, causing deadlocks that eventually locks up the
>     server.
>     Is there an easy way to tell existdb to revert to the older locking
>     behaviour that does not lock up the xquery being run?
>
>
>     Regards
>
>     -- 
>     Willem van der Westhuizen
>     +27 82 9200718
>
>
>
>     _______________________________________________
>     Exist-open mailing list
>     Exi...@li...
>     https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
>
> -- 
> Adam Retter
>
> eXist Core Developer
> { United Kingdom }
> ad...@ex...
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

-- 
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web:www.benjamins.com

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Willem v. d. W. <wi...@kw...>]

One further issue,

If the calls are not too rapid, then it does not happen. But many 
different calls to the same xquery will increase the likelihood of a lockup.

Willem

On 2025/10/15 10:30, Juri Leino wrote:
> Hi Willem!
>
> The behaviour you are describing is usually a sign for an attempt to 
> replace/update a running XQuery.  If this is the case the easiest 
> solution is to make sure that the query you want to replace / change 
> is not being run at the moment.
>
> I agree that we should investigate if we even have to lock the XQuery 
> in the first place. But that is a separate discussion.
>
> Regards,
> Juri
>
> On 12.10.25 13:01, Willem van der Westhuizen wrote:
>> Hi
>>
>> We use existdb extensively to run xqueries on our external databases, 
>> couchdb and others. We dont use the existdb datastore for our data.
>>
>> We used to run on an old 3.3.3 version of existdb, and recently 
>> migrated to 6.2. Since then we have found that existdb locks the 
>> xquery that is being executed, causing deadlocks that eventually 
>> locks up the server. Is there an easy way to tell existdb to revert 
>> to the older locking behaviour that does not lock up the xquery being 
>> run?
>>
>>
>> Regards
>>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Pieter L. <pie...@be...>]

Hi,

Might it be that there are any modules that include each other causing 
circular dependencies? I believe eXist has become a little pickier about 
this since version 4 and we've ironed them all out of our code base.

Best,
Pieter

On 10/15/2025 12:24 PM, Willem van der Westhuizen wrote:
>
> Hi,
>
> In our case we are not updating when the locking occurs. We simply 
> call the rest services, and then they call other rest services outside 
> of exist. We use local curl statements to call the rest services, and 
> that causes the lockup. And we can see that the waiting for locks are 
> growing. It seems that if the rest services are called from the 
> outside, and they themselves call rest services, the later versions 
> lock up. And when the server reached about 160 processed it always 
> crashes.    I have attached the screen shots of the monex. If you look 
> at the logs, its frozen, nothing running.
>
>
>
> On 2025/10/14 11:23, Pieter Lamers via Exist-open wrote:
>> Hi Willem,
>>
>> I would suggest using the latest release or build rather than 6.2.0 
>> and checking if the same behavior occurs. My personal experience is 
>> that lock ups occur when I try to update XQuery resources while they 
>> are in use. Therefore I do not update the applications while they are 
>> running production.
>>
>> Best,
>> Pieter
>>
>> On 10/12/2025 1:01 PM, Willem van der Westhuizen wrote:
>>> Hi
>>>
>>> We use existdb extensively to run xqueries on our external 
>>> databases, couchdb and others. We dont use the existdb datastore for 
>>> our data.
>>>
>>> We used to run on an old 3.3.3 version of existdb, and recently 
>>> migrated to 6.2. Since then we have found that existdb locks the 
>>> xquery that is being executed, causing deadlocks that eventually 
>>> locks up the server. Is there an easy way to tell existdb to revert 
>>> to the older locking behaviour that does not lock up the xquery 
>>> being run?
>>>
>>>
>>> Regards
>>>
>>

-- 
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web:www.benjamins.com

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Willem v. d. W. <wi...@kw...>]

Hi,

In our case we are not updating when the locking occurs. We simply call 
the rest services, and then they call other rest services outside of 
exist. We use local curl statements to call the rest services, and that 
causes the lockup. And we can see that the waiting for locks are 
growing. It seems that if the rest services are called from the outside, 
and they themselves call rest services, the later versions lock up. And 
when the server reached about 160 processed it always crashes.    I have 
attached the screen shots of the monex. If you look at the logs, its 
frozen, nothing running.



On 2025/10/14 11:23, Pieter Lamers via Exist-open wrote:
> Hi Willem,
>
> I would suggest using the latest release or build rather than 6.2.0 
> and checking if the same behavior occurs. My personal experience is 
> that lock ups occur when I try to update XQuery resources while they 
> are in use. Therefore I do not update the applications while they are 
> running production.
>
> Best,
> Pieter
>
> On 10/12/2025 1:01 PM, Willem van der Westhuizen wrote:
>> Hi
>>
>> We use existdb extensively to run xqueries on our external databases, 
>> couchdb and others. We dont use the existdb datastore for our data.
>>
>> We used to run on an old 3.3.3 version of existdb, and recently 
>> migrated to 6.2. Since then we have found that existdb locks the 
>> xquery that is being executed, causing deadlocks that eventually 
>> locks up the server. Is there an easy way to tell existdb to revert 
>> to the older locking behaviour that does not lock up the xquery being 
>> run?
>>
>>
>> Regards
>>
>

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Juri L. <ju...@ex...>]

Hi Willem!

The behaviour you are describing is usually a sign for an attempt to 
replace/update a running XQuery.  If this is the case the easiest 
solution is to make sure that the query you want to replace / change is 
not being run at the moment.

I agree that we should investigate if we even have to lock the XQuery in 
the first place. But that is a separate discussion.

Regards,
Juri

On 12.10.25 13:01, Willem van der Westhuizen wrote:
> Hi
>
> We use existdb extensively to run xqueries on our external databases, 
> couchdb and others. We dont use the existdb datastore for our data.
>
> We used to run on an old 3.3.3 version of existdb, and recently 
> migrated to 6.2. Since then we have found that existdb locks the 
> xquery that is being executed, causing deadlocks that eventually locks 
> up the server. Is there an easy way to tell existdb to revert to the 
> older locking behaviour that does not lock up the xquery being run?
>
>
> Regards
>

Re: [Exist-open] exist multiple cores

[Lars W. <lar...@ex...>]

It’s really weird why the app should be slow on a system with more ghz. There must be some kind of other limitation in the new environment.

With regard to Java 21: Where did you read that eXist-db 6.x would be bound to max Java 17? I know at least two people running eXist-db 6 with Java 21. I’m aware of some discussion about some edge cases that could cause trouble with Java 21 but afaik it’s working fine at least for those two people. Did you ever try it yourself? Nevertheless, if the performance was fine in the old environment but is not in the new one, the Java version will very likely not be the problem because you did not run Java 21 in the old system. 

Sorry I can’t help more but without actually looking in the new and old environment and running a few tests it’s kind of impossible to say, what the issue could be. 

Fingers crossed you can sort it out! 

Best, 

Lars



> On 8. Oct 2025, at 17:34, Eduard Drenth <ed...@fr...> wrote:
> 
> Thanks for the info, good to know about one xquery being bound to one core.
> 
> In the mean time we turned hyperthreading off, that helped, but not enough yet.
> 
> It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
> 
> The queries seem cpu bound as top shows cpu percentage above 200%.
> 
> Increasing memory does not help, IO does not seem to be a problem.
> 
> It is doable for now, but I am happy that new hardware is being ordered, hope that helps.
> 
> Unfortunately I cannot benefit from the many performance improvements in java either, because exist 6 is bound to java 17 max.
> 
> Regards,
> Eduard Drenth, Software Architekt
> 
> ed...@fr...
> 
> Doelestrjitte 8
> 8911 DX  Ljouwert
> (058) 213 14 14
> +31 62 094 34 28
> https://www.fryske-akademy.nl/
> https://frysker.nl/
> https://frisian.eu/<Outlook-d4b4dmlh.svg>
> gpg: https://keyserver.ubuntu.com/pks/lookup?search=eduard+drenth&fingerprint=on&op=index
> From: Lars Windauer <lar...@ex...>
> Sent: Tuesday, October 7, 2025 10:26 PM
> To: Exist-open <Exi...@li...>
> Subject: Re: [Exist-open] exist multiple cores
>  Dear Eduard, 
> 
> eXist-db can benefit from multiple cores without the need to configure anything. Having said this, please be aware that a single XQuery (dbbroker) can always run on only one core! So a single XQuery can’t utilise 20 cores but on a 20 core system you can have 20 parallel queries that each use one core. 
> 
> Can you provide us with information about the old and new environment? Gigahertz of your CPUs, how much RAM does the system have and how much of this is assigned to eXist-db. What about the hard drives, is eXist-db running from a SSD with the same filesystem as in the old environment? 
> 
> Best, 
> 
> Lars
> 
> 
> > On 6. Oct 2025, at 16:42, Eduard Drenth <ed...@fr...> wrote:
> > 
> > Dear all,
> > 
> > We are in the process of migrating exist from our own hardware to a provider.
> > 
> > Performance drops significantly, the main cause is that exist seems to be using one core only.
> > 
> > Can exist benefit from multiple cores? And if yes, how is that configured?
> > 
> > Regards,
> > 
> > Eduard Drenth, Software Architekt
> > 
> > ed...@fr...
> > _______________________________________________
> > Exist-open mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-open
> 
> 
> 
> 
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Pieter L. <pie...@be...>]

Hi Willem,

I would suggest using the latest release or build rather than 6.2.0 and 
checking if the same behavior occurs. My personal experience is that 
lock ups occur when I try to update XQuery resources while they are in 
use. Therefore I do not update the applications while they are running 
production.

Best,
Pieter

On 10/12/2025 1:01 PM, Willem van der Westhuizen wrote:
> Hi
>
> We use existdb extensively to run xqueries on our external databases, 
> couchdb and others. We dont use the existdb datastore for our data.
>
> We used to run on an old 3.3.3 version of existdb, and recently 
> migrated to 6.2. Since then we have found that existdb locks the 
> xquery that is being executed, causing deadlocks that eventually locks 
> up the server. Is there an easy way to tell existdb to revert to the 
> older locking behaviour that does not lock up the xquery being run?
>
>
> Regards
>

-- 
Pieter Lamers
John Benjamins Publishing Company
Postal Address: P.O. Box 36224, 1020 ME AMSTERDAM, The Netherlands
Visiting Address: Klaprozenweg 75G, 1033 NN AMSTERDAM, The Netherlands
Warehouse: Kelvinstraat 11-13, 1446 TK PURMEREND, The Netherlands
tel: +31 20 630 4747
web: www.benjamins.com

Re: [Exist-open] exist multiple cores

[Adam R. <ad...@ex...>]

Thanks for the info, good to know about one xquery being bound to one core.
>


It's not as simple as that I am afraid. The shared resources within
eXist-db are guarded by locks.

At the dbx file level, eXist-db uses a mutex as a lock, and so can only
ever have 1 thread, that thread can be either reading or writing, but not
both.
At the higher Document and Collection object level, each of those also has
at least one ReadWrite lock, there may be a single writer thread, or
multiple reader threads, but throughput here is limited by the underlying
dbx level when needing to load or save those objects.

If you are not performing store/load operations, then key to performance
can be how you layout your data in documents and collections. Both use a
hierarchical locking scheme along their URI path, so you want to avoid
common parent collections where contention for locks is high.


In the mean time we turned hyperthreading off, that helped, but not enough
> yet.
>

Makes sense. For eXist-db you will want the most performance per-core
possible.


It is weird, the new cpu is 3.8ghz compared to the old 2.4ghz.
> The queries seem cpu bound as top shows cpu percentage above 200%.
>

It would be interesting to know where that time is spent. Is it lock
contention, or is it meaningful work?



> Increasing memory does not help, IO does not seem to be a problem.
>

If you are CPU bound, changing those worn't help you.


Unfortunately I cannot benefit from the many performance improvements in
> java either, because exist 6 is bound to java 17 max.
>

eXist-db 6 actually requires Java 8.

-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] Urgent assistance with locking behaviour of existdb

[Adam R. <ad...@ex...>]

Hi Willem,

In eXist-db 5.x.x and 6.x.x, the XQuery (if stored in the database) is
treated as any other document, and so it is locked for the duration of
operations that happen upon it. Unfortunately this means that whilst a
query is executing the document holding that query will keep a READ lock on
the document.

A document may have many either a single write lock, or one or more read
locks. This means that the query can only be updated by one person/client
at a time, but that multiple clients can run the query in parallel.

I am not sure how a deadlock could occur with this. Can you provide some
further information about your issue, and how to reproduce it please?

Thanks, Adam.

On Tue, 14 Oct 2025 at 08:53, Willem van der Westhuizen <wi...@kw...>
wrote:

> Hi
>
> We use existdb extensively to run xqueries on our external databases,
> couchdb and others. We dont use the existdb datastore for our data.
>
> We used to run on an old 3.3.3 version of existdb, and recently migrated
> to 6.2. Since then we have found that existdb locks the xquery that is
> being executed, causing deadlocks that eventually locks up the server.
> Is there an easy way to tell existdb to revert to the older locking
> behaviour that does not lock up the xquery being run?
>
>
> Regards
>
> --
> Willem van der Westhuizen
> +27 82 9200718
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Adam Retter

eXist Core Developer
{ United Kingdom }
ad...@ex...

Re: [Exist-open] Autocompleting src and href attributes

[Alasdair D. <ala...@gm...>]

Hi Dannes,

You’re in luck, grab yourself a bowl of popcorn and enjoy the movie.

I save using the eXide Save button. You will see the screencast shows the unusual behaviour.

See: https://share.icloud.com/photos/001aTqWO4WgYunQQaVKSTHJdA

Thanks in advance,

Alasdair

Sent from my iPhone

> On 14 Oct 2025, at 6:01 am, Dannes Wessels <di...@ex...> wrote:
> 
> Hi,
> 
>> On 29 Sep 2025, at 03:56, Alasdair Dougall <ala...@gm...> wrote:
>> 
>> The issue is on saving and closing the file, it expand the href to:
> 
> How do you do the save?
> 
> Cheers
> 
> Dannes
> 

Re: [Exist-open] Autocompleting src and href attributes

[Dannes W. <di...@ex...>]

Hi,

> On 29 Sep 2025, at 03:56, Alasdair Dougall <ala...@gm...> wrote:
> 
> The issue is on saving and closing the file, it expand the href to:

How do you do the save?

Cheers

Dannes

[Exist-open] Urgent assistance with locking behaviour of existdb

[Willem v. d. W. <wi...@kw...>]

Hi

We use existdb extensively to run xqueries on our external databases, 
couchdb and others. We dont use the existdb datastore for our data.

We used to run on an old 3.3.3 version of existdb, and recently migrated 
to 6.2. Since then we have found that existdb locks the xquery that is 
being executed, causing deadlocks that eventually locks up the server. 
Is there an easy way to tell existdb to revert to the older locking 
behaviour that does not lock up the xquery being run?


Regards

-- 
Willem van der Westhuizen
+27 82 9200718