Hello!

 

We are software developers working on Pentaho technologies via olap4j / XML/A.

Because sometimes Mondrian is not appropriate we also use Microsoft SQL Server Analysis Services.

 

Situation:

 

We are running a XML/A client using olap4j 1.2 and Microsoft SQL Server Analysis Services 2012.

We have a simple star schema on SSAS but one dimension can have more then 1.000.000 members. I know bad design L , but the CUBES are made from another company.

 

So, when there is lots of members we get ERROR :

 

08:21:15.189 c.r.m.server.run.SchemaTester INFO       Level: [Dokument - finančni].[Dokument - finančni].[©tevilka dokumenta] -> Type: REGULAR

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit

       at java.util.Arrays.copyOf(Arrays.java:2271)

       at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)

       at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)

       at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)

       at org.olap4j.driver.xmla.proxy.XmlaOlap4jHttpProxy.getResponse(XmlaOlap4jHttpProxy.java:150)

       at org.olap4j.driver.xmla.proxy.XmlaOlap4jAbstractHttpProxy.get(XmlaOlap4jAbstractHttpProxy.java:181)

       at org.olap4j.driver.xmla.proxy.XmlaOlap4jHttpProxy.get(XmlaOlap4jHttpProxy.java:42)

       at org.olap4j.driver.xmla.XmlaOlap4jConnection.executeMetadataRequest(XmlaOlap4jConnection.java:876)

       at org.olap4j.driver.xmla.XmlaOlap4jConnection.populateList(XmlaOlap4jConnection.java:851)

       at org.olap4j.driver.xmla.DeferredNamedListImpl.populateList(DeferredNamedListImpl.java:136)

       at org.olap4j.driver.xmla.DeferredNamedListImpl.getList(DeferredNamedListImpl.java:90)

       at org.olap4j.driver.xmla.DeferredNamedListImpl.size(DeferredNamedListImpl.java:116)

       at com.resevo.dev.olap.SchemaTester.main(SchemaTester.java:134)

 

Call line:for(Member child : member.getChildMembers()) {...}

 

The actual response from server is 97mb of gzip-ed data that gets extracted to about 1.25gb byte array. So we're breaching limit of single array instance, not limitations of memory available JVM (8gb).

 

Question:

Is there any other way to lets say load dimension members in smaller chunks like in SQL via  [LIMIT { number | ALL }] [OFFSET number] or do you suggest any other way?

 

Are there any plans to change reading of XMLA responses from current DOM/Xerces implementation to a StAX/streaming one so we don't have to keep full response in memory?

 

Thanks in advance.

 

Kind regards,

 ToMaZ


GSM: 041 334 356
Toma¾ Bergant
ResEvo d.o.o.
Fin¾garjeva 1a
SI-4248 Lesce
WEB:
http://www.ResEvo.com
email:
tomaz.bergant@ResEvo.com