Re: [Exist-open] 504 error in relatively small query

[Ralph C. <ra...@ra...>]

Thank you all (apologies for not writing a specific thanks to @Lizzi, 
Vincent but also really useful).

However, @Jean-Paul Rehr you have also very correctly pointed out some 
xQuery issues, which have made a massive difference. It runs at the 
collection level now and returns the ~7000 orphaned records 0.17s. 
Amazing!

I've not come across the id() function before. Why does it operate so 
quickly?

Thank you again

On 13/03/2023 16:12, Jean-Paul Rehr wrote:

> I feel like your query could run much faster in eXist with some 
> adjustments.
> - I would avoid reopening the document in the where and put it in a 
> variable
> - not(exists()) can be replaced by empty() and make evaluation more 
> efficient
> -  /id() is faster than @xml:id =
> 
> let $listPerson := doc('/db/apps/****/data/listPerson.xml')
> for $ref in subsequence(collection('/db/apps/****/data')//ref[@type eq 
> '2'],1,####)
> where empty($listPerson/id($ref/@target))
> return
> $ref
> 
> Cheers,
> JPR
> 
> On Mon, Mar 13, 2023 at 3:18 PM Ralph Corrigan <ra...@ra...> 
> wrote:
> 
> Hi all, I was going to post this in an xQuery forum, but at small 
> numbers this query works fine and I feel confident that the issue here 
> is in my exist-db config.
> 
> I'm trying to return orphaned <ref> elements from a recent update that 
> was set up wrong. I'm using the following query to get the results:
> 
> for $ref in 
> subsequence(collection('/db/apps/****/data')//ref[@type='2'],1,####)
> 
> where 
> not(exists(doc('/db/apps/****/data/listPerson.xml')//person[@xml:id=$ref/@target]))
> 
> return
> $ref
> 
> This returns absolutely fine up to ~1200 items in the subsequence $ref 
> but by the time we get closer to 2000 (and more) this returns a 504 
> gateway time-out error in roughly 60s. This does not seem to me to be a 
> significantly onerous task (especially considering other things that 
> are handled well).
> 
> I've tried
> 
> declare option exist:timeout "600000" (which should be ten minutes)
> 
> to extend the timeout, but I'm getting the same error after ~60s.
> 
> Any ideas, as the total I'm trying to get to is ~12000 records of which 
> I have no idea how many are orphaned....
> 
> Cheers
> 
> Ralph
> 
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open