Re: [Exist-open] Urgent assistance with locking behaviour of existdb - or how to change the architecture of a scheduled ETL service

[Claudius T. <cla...@gm...>]

Hi!

Would it help to have the ETL engine as an eXist-db module?


Claudius Teodorescu

On Sat, 18 Oct 2025 at 11:29, Willem van der Westhuizen <wi...@kw...>
wrote:

> Maybe there is a different question to ask. We are using exist and
> xquery extensively with an etl engine that we wrote to read data from
> couchdb and place into MySQL for reporting purposes. We are currently
> using a shell script to launch 20 instances of the the etl with
> different parameters to pick up a batch and process it. Would there be a
> better way in exist to write the scheduler to make those calls, and then
> to wait for each instance to be complete before calling the next, that
> would not involve calling the same rest service muliple times from curl?
>
> Regards
>
> Willem
>
> On 2025/10/15 11:48, Willem van der Westhuizen wrote:
> > Definitely not updating. Just running,
> >
> > On 2025/10/15 10:30, Juri Leino wrote:
> >> Hi Willem!
> >>
> >> The behaviour you are describing is usually a sign for an attempt to
> >> replace/update a running XQuery.  If this is the case the easiest
> >> solution is to make sure that the query you want to replace / change
> >> is not being run at the moment.
> >>
> >> I agree that we should investigate if we even have to lock the XQuery
> >> in the first place. But that is a separate discussion.
> >>
> >> Regards,
> >> Juri
> >>
> >> On 12.10.25 13:01, Willem van der Westhuizen wrote:
> >>> Hi
> >>>
> >>> We use existdb extensively to run xqueries on our external
> >>> databases, couchdb and others. We dont use the existdb datastore for
> >>> our data.
> >>>
> >>> We used to run on an old 3.3.3 version of existdb, and recently
> >>> migrated to 6.2. Since then we have found that existdb locks the
> >>> xquery that is being executed, causing deadlocks that eventually
> >>> locks up the server. Is there an easy way to tell existdb to revert
> >>> to the older locking behaviour that does not lock up the xquery
> >>> being run?
> >>>
> >>>
> >>> Regards
> >>>
> >>
> >>
> >> _______________________________________________
> >> Exist-open mailing list
> >> Exi...@li...
> >> https://lists.sourceforge.net/lists/listinfo/exist-open
> >
> >
> > _______________________________________________
> > Exist-open mailing list
> > Exi...@li...
> > https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>


-- 
Cu stimă,
Claudius Teodorescu