From: d s <mis...@ya...> - 2007-09-20 23:21:39
|
Hi, I have the same problem as Bert, but in my case the size is much smaller I only have 300MB, 14.500 files, 6500 collections/subsollections. I've increased the JVM to 1GB and also have some indexing. But still when I did the query it has "java.lang.OutOfMemoryError: GC overhead limit exceeded" or " java.lang.OutOfMemoryError: Java heap space" Bert,would you be able to explain more detail about your solution? much appreciated. I have some calculation in my query, will that cost a problem? MyData.xml <parent> <subparentA>test</subparent> <subparentB>test <subB id="123">subB <a>abc1234</a> <b>abc</b> <date1>...</date1> <date2>...</date2> . . . <z>abc</z> </subB> <subB id="456">subB <a>abc1234</a> <b>abc</b> <date1>...</date1> <date2>...</date2> . . . <z>abc</z> </subB> </subparentB> </parent> Query: declare namespace x="http://www.foo.com"; for $a in //x:parent let $sub:= $a//x:subB[@id="123"] let $calcdate := xs:dateTime($sub//date1) + xs:dateTime($sub//date2) ..more calcultation where $a//x:sub[@id="123"] return ($a//x:subparentA, <sub id="{$sub/@id}"> <a>{xs:string($sub//x:a)}</a> <b>{xs:string($sub//x:b)}</b> <dateresult>{$caldate cast as xs:date}</dateresult> . . . </sub>) Any suggestions how can I solve this problem? Cheers, Re: [Exist-open] OutOfMemoryError during XQuery From: Bert Schultheiss <bertschultheiss@ho...> - 2007-08-07 07:54 Dear Wolfgang, Thank you for your answer. To solve my problem for now, I rewrote the query using a local function that obtains all of the documents having DataSet, a local function that returns the xquery for a single document, and a "main" that for all document-uris first obtains the xquery and next applies eval on the xquery. Now the xquery runs with out any memory problem. Actually, I failed to find the stack trace on the server: not tomcat's log file, not in WEB-INF/logs ... But if I run the xquery using the client in embedded mode, I get: java.lang.OutOfMemoryError: Java heap space at org.exist.storage.btree.Paged$Page.read(Paged.java:978) at org.exist.storage.index.BFile.getDataPage(BFile.java:570) at org.exist.storage.index.BFile.access$1600(BFile.java:82) at org.exist.storage.index.BFile$MultiPageInput.advance (BFile.java:2364) at org.exist.storage.index.BFile$MultiPageInput.readShort (BFile.java:2285) at org.exist.storage.StorageAddress.read(StorageAddress.java:92) at org.exist.storage.NativeElementIndex.findElementsByTagName (NativeElementIndex.java:466) at org.exist.xquery.LocationStep.getChildren(LocationStep.java:443) at org.exist.xquery.LocationStep.eval(LocationStep.java:217) at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56) at org.exist.xquery.PathExpr.eval(PathExpr.java:216) at org.exist.xquery.DynamicCardinalityCheck.eval (DynamicCardinalityCheck.java:71) at org.exist.xquery.Atomize.eval(Atomize.java:67) at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:66) at org.exist.xquery.functions.FunMin.eval(FunMin.java:99) at org.exist.xquery.InternalFunctionCall.eval (InternalFunctionCall.java:49) at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56) at org.exist.xquery.PathExpr.eval(PathExpr.java:216) at org.exist.xquery.LetExpr.eval(LetExpr.java:106) at org.exist.xquery.LetExpr.eval(LetExpr.java:152) at org.exist.xquery.ForExpr.eval(ForExpr.java:249) at org.exist.xquery.BindingExpression.eval(BindingExpression.java:109) at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56) at org.exist.xquery.PathExpr.eval(PathExpr.java:216) at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:56) at org.exist.xquery.XQuery.execute(XQuery.java:201) at org.exist.xquery.XQuery.execute(XQuery.java:171) at org.exist.xmldb.LocalXPathQueryService.execute (LocalXPathQueryService.java:354) at org.exist.xmldb.LocalXPathQueryService.execute (LocalXPathQueryService.java:142) at org.exist.client.QueryDialog$QueryThread.run (QueryDialog.java:401) Kind regards, Bert Thread View Thread Author Date [Exist-open] OutOfMemoryError during XQuery Bert Schultheiss <bertschultheiss@ho...> Re: [Exist-open] OutOfMemoryError during XQuery From: Bert Schultheiss <bertschultheiss@ho...> - 2007-08-06 06:49 Dear all, Any clue? Actually, I think eXist is working really well - I am impressed - I just have this single outOfMemory during an XQuery problem ... Bert Re: [Exist-open] OutOfMemoryError during XQuery From: Wolfgang Meier <wolfgangmm@gm...> - 2007-08-06 15:04 > 3. The XQuery: > for $d in collection("/db/test")//DataSet > return > <entry> > <docuri>{ document-uri($d) }</docuri> > <s1>{ max($d/l[./s1>1.999]/t) }</s1> > <s2>{ min($d/l[./s2<-10.9999]/t) }</s2> > </entry> > > For one specific document, eXist answers. > With 2 files eXist answers. > Why do I get an OutOfMemoryError when querying all four documents? The only possible explanation I see is that max() and min() consume too much memory as they need to atomize the element's value. However, I just checked the implementation and found that max and min are never comparing more than two values. So the memory leak has to be elsewhere. It would be good to watch the query in a memory profiler. Certainly something is wrong. I'll try to test this once I have some time again. > 4. The stack trace This is the client side exception. We need the exception shown in the server's log. Wolfgang --------------------------------- Looking for a deal? Find great prices on flights and hotels with Yahoo! FareChase. |