Re: [Xmltask-users] Memory leaks using xmltask

[Brian A. <br...@oo...>]

Thx for this. I'll run some tests and have a look at what's happening. I
can't say much until I've had a look, but the below is sufficiently simple
that I should be able to determine pretty quickly...

On Wed, April 2, 2008 21:42, Pau...@no... wrote:
> Our real-world use case is merging into one file a number of XML files
> that total approx 3-4Mb in size, maybe more.
>
> A test case I tried is below:
>
>     <target name="try-xmltask">
>         <xmltask dest="xmltask_test.xml">
>             <insert path="/"  >
>             <![CDATA[
>             <test/>
>             ]]>
>             </insert>
>         </xmltask>
>         <script language="jython">
> result = ",".join([str(i) for i in range(10000)])
> project.setProperty("xml.test.list", str(result))
>         </script>
>         <echo>${xml.test.list}</echo>
>         <for param="number" list="${xml.test.list}">
>             <sequential>
>                 <xmltask source="xmltask_test.xml"
>                          dest="xmltask_test.xml">
>                    <insert path="/test">
>                        <![CDATA[
>             <element>@{number}</element>
>             ]]>
>                    </insert>
>                 </xmltask>
>             </sequential>
>         </for>
>     </target>
>
>>From the memory profiling I did, it seems to increase from ~80Mb to
> ~140Mb.
>
> Are buffers used in any use of xmltask? What is persisted there? When
> might they get cleaned up?
>
> thanks
>
> paul
>
>
>
>
> ________________________________
>
> 	From: ext Brian Agnew [mailto:br...@oo...]
> 	Sent: Sunday, March 30, 2008 8:26 AM
> 	To: Mackay Paul (Nokia-D-MSW/Vancouver)
> 	Cc: xml...@li...
> 	Subject: Re: [Xmltask-users] Memory leaks using xmltask
>
>
> 	There's been no explicit memory profiling done.
>
> 	I suspect that XMLTask doesn't clear up the XML doc it's been
> manipulating. I will check that and rectify.
>
> 	Note that buffers are persisted across XMLTask calls and so
> those will remain. XMLTask will read in the document as a DOM, and due
> to the random access nature of XPath and the ad hoc manipulation of XML,
> I don't think there's an alternative to that.
>
> 	What size of docs are you dealing with, and what specifically
> are you doing ? With that info I can start to investigate further. Can
> you forward me your test detailed below ?
>
> 	Brian
>
> 	Pau...@no... wrote:
>
> 		Hi,
>
> 		Has anyone tried testing memory usage of xmltask? We run
> a long build using Ant with large XML data content. Recently we
> experienced OOM errors in our builds. I did some profiling and found
> that running xmltask takes a portion of heap memory which does not get
> garbage collected. We had the max heap size configured to 200Mb.
> Increasing that has allowed our build to proceed further, but it would
> be helpful to know if there are any dangling objects not being cleaned
> up. This may also be an Ant problem, but specifically the memory
> increases seemed to be around the use of xmltask. I wrote a specific
> test that simply inserted an element 10000 times into an initially empty
> document and this also grew the heap memory over time.
>
> 		Also has there been any examination of the memory
> profile of processing large XML documents? Although most of the memory
> is garbage collected, reading a large document as DOM can take many Mbs
> of memory temporarily.
>
> 		Any suggestions most welcome.
>
> 		thanks
>
> 		paul
>
>
> 		________________________________
>
>
>
> ------------------------------------------------------------------------
> -
> 		Check out the new SourceForge.net Marketplace.
> 		It's the best place to buy or sell services for
> 		just about anything Open Source.
>
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketp
> lace
> 		________________________________
>
>
> 		_______________________________________________
> 		Xmltask-users mailing list
> 		Xml...@li...
>
> https://lists.sourceforge.net/lists/listinfo/xmltask-users
>
>
>
> 	--
> 	Brian Agnew                  http://www.oopsconsultancy.com
> 	OOPS Consultancy Ltd
> 	Tel: +44 (0)7720 397526
> 	Fax: +44 (0)20 8682 0012
>
>
>


-- 
Brian Agnew                  http://www.oopsconsultancy.com
OOPS Consultancy Ltd
Tel: +44 (0)7720 397526
Fax: +44 (0)20 8682 0012