[Exist-development] query collection containing one million documents

[Patrick S. <pat...@zi...>]

  Hi,

I am using eXist to index one million xml documents of about 2.3GiB 
within one collection.

Unfortunately the query execution time is rather poor. Even a very 
simple XQuery results in execution times around 4sec:

   declare namespace xq="http://www.xml-cml.org/schema";
   for $i in collection("tuscreen")//xq:molecule[@id="298038"]
     return
<smile>{$i}</smile>

Logging says:

09 Jul 2010 15:11:55,783 [P1-9] DEBUG (XQuery.java [compile]:155) - 
Query diagnostics:
for <2>
     $i in collection("tuscreen")
     (# exist:optimize #)
     {
         descendant-or-self::xq:molecule[attribute::id = "298038"]
     }

return <3>
     element {"smile"} {
         {
             $i
         }
     }
09 Jul 2010 15:11:55,784 [P1-9] DEBUG (XQuery.java [compile]:161) - 
Compilation took 4 ms
09 Jul 2010 15:11:58,119 [P1-9] TRACE (Optimize.java 
[visitGeneralComparison]:219) - exist:optimize: found optimizable: 
org.exist.xquery.GeneralComparison
09 Jul 2010 15:11:58,119 [P1-9] TRACE (Optimize.java [before]:240) - 
exist:optimize: context step: 
descendant-or-self::xq:molecule[attribute::id = "298038"]
09 Jul 2010 15:11:58,119 [P1-9] TRACE (Optimize.java [before]:241) - 
exist:optimize: context var: null
09 Jul 2010 15:11:59,496 [P1-9] TRACE (GeneralComparison.java 
[preSelect]:266) - Using QName index on type xs:string
09 Jul 2010 15:11:59,496 [P1-9] TRACE (GeneralComparison.java 
[preSelect]:298) - Using QName range index for key: 298038
09 Jul 2010 15:11:59,666 [P1-9] TRACE (Optimize.java [eval]:120) - 
exist:optimize: pre-selection: 1
...
9 Jul 2010 15:11:59,837 [P1-9] DEBUG (XQuery.java [execute]:231) - 
Execution took 4.050 ms


Is there a way to speed up queries?

Thank you for any help

Patrick