ServingXML / Support Requests / #8 OutOfMemory Exception using an sx:subtreeRecordReader

#8 OutOfMemory Exception using an sx:subtreeRecordReader

Status: open

Owner: nobody

Labels: None

Priority: 5

Updated: 2011-03-18

Created: 2011-03-18

Creator: KKK

Private: No

Hi!

I am having trouble with a huge xml-file (about 650 MB) including
round about 1.000.000 million records which i want to convert into
a flatFile.

servingXML aborts with an OutOfMemory-Exception because running out of java heap space...but i thought, servingXML uses a streaming architecture, that doesn't allocate java heap space for a whole file but only for the currently processed record...

My servingXML-resource works fine if try it with an XML-file of only 100.000 records...

What can i change in my resources-file to solve this problem?
Is there any suggestion for processing those large xml-files?
(for Details: see attachements)

Thank you very much in advance.

Kind regards
K. Krauthausen

Discussion

KKK - 2011-03-18

servingXML resource

test_sx_resourcefile.xml

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

KKK - 2011-03-18

java exception that is raised while processing 1.000.000 records

exception.txt

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

KKK - 2011-03-18

short example xml-source file to be converted (only two records instead of 1 million)

source.xml

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

KKK - 2011-03-18

summary: OutOfMemory Exception using a sx:subtreeRecordReader --> OutOfMemory Exception using an sx:subtreeRecordReader
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

OutOfMemory Exception using an sx:subtreeRecordReader

Group

Searches

Help

#8 OutOfMemory Exception using an sx:subtreeRecordReader

Discussion