Re: [Exist-development] query collection containing one million documents

[Patrick S. <pat...@zi...>]

  Hi Joe,

thank you for the response.
> Sounds like there are some theories about getting to the bottom of the
> slowness, but another tack might be to adopt this approach from the
> Performance Tuning article, "4.9. Use fn:id to lookup xml:id
> attributes" - see http://exist-db.org/tuning.html#N10382.  This would
> require either using @xml:id instead of @id, or declaring @id as type
> ID in a DTD - maybe more trouble than it's worth.  But I have a hunch
> that the id() function is faster than a range index; unfortunately, I
> don't actually see that spelled out anywhere.
That might solve this problem. Unfortunately the given query was just an 
example. The following query gives me similar execution times though I 
defined an index on "@value":

   declare namespace xq="http://www.xml-cml.org/schema";
   for $i in 
collection("tuscreen")//*[@value="Cc1c2c(ncn(c2=O)C)sc1C(=O)NCc1ccc(cc1)CN1CCOCC1"]
     return
       $i

09 Jul 2010 16:47:17,746 [P1-9] DEBUG (XQuery.java [compile]:161) - 
Compilation took 3 ms
09 Jul 2010 16:47:19,971 [P1-9] TRACE (Optimize.java 
[visitGeneralComparison]:219) - exist:optimize: found optimizable: 
org.exist.xquery.GeneralComparison
09 Jul 2010 16:47:19,972 [P1-9] TRACE (Optimize.java [before]:240) - 
exist:optimize: context step: null
09 Jul 2010 16:47:19,972 [P1-9] TRACE (Optimize.java [before]:241) - 
exist:optimize: context var: null
09 Jul 2010 16:47:21,267 [P1-9] TRACE (GeneralComparison.java 
[preSelect]:266) - Using QName index on type xs:string
09 Jul 2010 16:47:21,267 [P1-9] TRACE (GeneralComparison.java 
[preSelect]:298) - Using QName range index for key: 
Cc1c2c(ncn(c2=O)C)sc1C(=O)NCc1ccc(cc1)CN1CCOCC1
09 Jul 2010 16:47:21,428 [P1-9] TRACE (Optimize.java [eval]:120) - 
exist:optimize: pre-selection: 1
09 Jul 2010 16:47:21,530 [P1-9] DEBUG (XQuery.java [execute]:231) - 
Execution took 3.784 ms
09 Jul 2010 16:47:21,530 [P1-9] DEBUG (HTTPUtils.java 
[addLastModifiedHeader]:61) - mostRecentDocumentTime: 1278439917815
09 Jul 2010 16:47:21,531 [P1-9] DEBUG (HTTPUtils.java 
[addLastModifiedHeader]:61) - mostRecentDocumentTime: 1278439917815
09 Jul 2010 16:47:21,531 [P1-9] INFO  (RpcConnection.java [doQuery]:303) 
- query took 3785ms.
09 Jul 2010 16:47:21,531 [P1-9] DEBUG (RpcConnection.java [queryP]:2434) 
- found 1



Index definition is:
<collection xmlns="http://exist-db.org/collection-config/1.0">
<index xmlns:atom="http://www.xml-cml.org/schema">
<fulltext default="none" attributes="no"/>
<!-- Range indexes by qname -->
<create qname="@id" type="xs:string"/>
<create qname="@value" type="xs:string"/>
</index>
</collection>


Patrick