[Exist-open] Odd problem with large files.

[Hugh C. <hca...@em...>]

Dear eXisters,

I'm having an odd problem with some large XML files.  These are TEI  
XML files that represent large volumes of documents.  File sizes  
range from 3-6 Mb, and between 100-400+ documents per file.  Each  
document is a <div1> with an id attribute in the form csr<volume #>- 
NNNN.  Initial tests with one, then two volumes seem to work fine,  
but now that I've loaded 6 of them, I'm seeing a problem with the  
later ones.  One of the things I want to be able to do is look up  
these div1 elements by their id attributes and return, then transform  
those XML fragments.  This is the issue I'm having, for example with  
volume 6:  A query like:

for $div in //div1[@id='csr06-0051']
return $div

returns the right div1 element, but

for $div in //div1[@id='csr06-0052']
return $div

which should pull up csr06-0051's following sibling returns nothing,  
even though that div1 is demonstrably there.  in fact, I can do

for $div in //div1[@id='csr06-0051']/following::div1[1]
return $div

and I get the right div1.  So it *is* there, just not retrievable by  
@id.  I'm seeing the stacktrace below in my logs.  This is 1.1.2dev,  
the latest build (rebuilt it today to see if it had any effect).  I  
haven't narrowed down yet whether there's something about the files  
themselves that's causing the problem or whether it's the fact that  
there are a bunch of these large files and the later ones are acting  
squirrelly because of that.  I did try putting each file in its own  
collection, which had no effect.

Any ideas would be most appreciated.

Thanks,
Hugh Cayless


/**
  * Hugh A. Cayless, Ph.D
  * Digital Library Research & Development
  * UNC Chapel Hill
  * hca...@em...
  */


Log snippet:

2007-08-07 16:38:33,822 [SocketListener0-3] DEBUG (XQuery.java
[compile]:161) - Query diagnostics:
for  <1>
     $div in [root-node]/descendant-or-self::text[attribute::id =
"volume_06"]/descendant-or-self::div1[attribute::id = "csr06-0053"]
return <2>
     $div

2007-08-07 16:38:33,822 [SocketListener0-3] DEBUG (XQuery.java 
[compile]:167) - Compilation took 2 ms
2007-08-07 16:38:33,830 [SocketListener0-3] ERROR  
(NativeElementIndex.java [findDescendantsByTagName]:798) - Found  
wrong prefix len: 4. Previous: 1.3.4.51
java.io.IOException: Found wrong prefix len: 4. Previous: 1.3.4.51
         at org.exist.numbering.DLNBase.<init>(DLNBase.java:154)
         at org.exist.numbering.DLN.<init>(DLN.java:122)
         at org.exist.numbering.DLNFactory.createFromStream 
(DLNFactory.java:55)
         at  
org.exist.storage.NativeElementIndex.findDescendantsByTagName 
(NativeElementIndex.java:726)
         at org.exist.xquery.LocationStep.getAttributes 
(LocationStep.java:505)
         at org.exist.xquery.LocationStep.eval(LocationStep.java:297)
         at org.exist.xquery.AbstractExpression.eval 
(AbstractExpression.java:59)
         at org.exist.xquery.GeneralComparison.quickNodeSetCompare  
(GeneralComparison.java:518)
         at org.exist.xquery.GeneralComparison.eval 
(GeneralComparison.java:366)
         at org.exist.xquery.AbstractExpression.eval 
(AbstractExpression.java:59)
         at org.exist.xquery.PathExpr.eval(PathExpr.java:241)
         at org.exist.xquery.Predicate.selectByNodeSet(Predicate.java: 
321)
         at org.exist.xquery.Predicate.evalPredicate(Predicate.java:195)
         at org.exist.xquery.LocationStep.applyPredicate 
(LocationStep.java:179)
         at org.exist.xquery.LocationStep.eval(LocationStep.java:325)
         at org.exist.xquery.AbstractExpression.eval 
(AbstractExpression.java:59)
         at org.exist.xquery.PathExpr.eval(PathExpr.java:241)
         at org.exist.xquery.ForExpr.eval(ForExpr.java:183)
         at org.exist.xquery.BindingExpression.eval 
(BindingExpression.java:158)
         at org.exist.xquery.AbstractExpression.eval 
(AbstractExpression.java:59)
         at org.exist.xquery.PathExpr.eval(PathExpr.java:241)
         at org.exist.xquery.AbstractExpression.eval 
(AbstractExpression.java:59)
         at org.exist.xquery.XQuery.execute(XQuery.java:217)
         at org.exist.xquery.XQuery.execute(XQuery.java:184)
         at org.exist.xmlrpc.RpcConnection.doQuery(RpcConnection.java: 
264)
         at org.exist.xmlrpc.RpcConnection.queryP(RpcConnection.java: 
1554)
         at org.exist.xmlrpc.RpcConnection.queryP(RpcConnection.java: 
1521)
         at org.exist.xmlrpc.RpcServer.queryP(RpcServer.java:877)
         at org.exist.xmlrpc.RpcServer.queryP(RpcServer.java:859)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke 
(NativeMethodAccessorImpl.java:39)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke 
(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:585)
         at org.exist.xmlrpc.AuthenticatedHandler.execute 
(AuthenticatedHandler.java:120)
         at org.exist.xmlrpc.AuthenticatedHandler.execute 
(AuthenticatedHandler.java:67)
         at org.apache.xmlrpc.XmlRpcWorker.invokeHandler 
(XmlRpcWorker.java:128)
         at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java: 
185)
         at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java: 
151)
         at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java: 
139)
         at org.exist.xmlrpc.RpcServlet.doPost(RpcServlet.java:93)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
         at org.mortbay.jetty.servlet.ServletHolder.handle 
(ServletHolder.java:428)
         at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch 
(WebApplicationHandler.java:473)
         at org.mortbay.jetty.servlet.ServletHandler.handle 
(ServletHandler.java:568)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1530)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1482)
         at org.mortbay.http.HttpServer.service(HttpServer.java:909)
         at org.mortbay.http.HttpConnection.service 
(HttpConnection.java:820)
         at org.mortbay.http.HttpConnection.handleNext 
(HttpConnection.java:986)
         at org.mortbay.http.HttpConnection.handle 
(HttpConnection.java:837)
         at org.mortbay.http.SocketListener.handleConnection 
(SocketListener.java:245)
         at org.mortbay.util.ThreadedServer.handle 
(ThreadedServer.java:357)
         at org.mortbay.util.ThreadPool$PoolThread.run 
(ThreadPool.java:534)