Re: [Xmltk-devs] Sorting a huge XML file

[Makoto O. <oni...@la...>]

Dinesh,

I hit the same problem. As far as I know/guess:
- the initial memory size of xsort is fixed at 38MB.
- xsort with default memory setting works if the output
  size doesn't exceed around 100MB.
- so, there are some bugs in disk-sort-merge algorithm
  in xsort. I can't fix it soon.

A easiest solution is to set the memory size.
xsort -m {memory size MB} -c "top"   -e "*" -k ...;

makoto


From: Makoto ONIZUKA <oni...@la...>
Subject: Re: [Xmltk-devs] Sorting a huge XML file
Date: Sat, 20 Jan 2007 15:48:58 +0900 (JST)
Message-ID: <200...@la...>

>Dinesh,
>
>I have tested the xsort using a synthetic XML data 
>that satisfies your condition; the total size of
>BLOCK1 and BLOCK2 exceeds 100MB. 
>
>Two exapmles below work correctly.
>  xsort -c "top"   -e "*" -k ...
>  xsort -c "top/*" -e "*" -k ...
>
>So, can you give me more details of your example?
>
>makoto
>
>
>From: "Venkataramanaidu, Dinesh (Dinesh)** CTR **" <vd...@al...>
>Subject: [Xmltk-devs] Sorting a huge XML file
>Date: Wed, 10 Jan 2007 15:37:22 +0530
>Message-ID: <3BE...@ii...>
>
>>Hello XML Toolkit developers,  
>>
>>While looking for a tool to sort XML tags i hit upon
>>xsort utility in sourceforge. This tool impressed me very much after
>>my initial testing of this tool with sample XML files.
>>Now i am trying to use xsort utility against bigger XML files, 
>>and encountering a problem.
>>
>>In the below XML template, if the total size of descendents of BLOCK1(or
>>BLOCK2)
>>exceeds ~100MB then running xsort on this XML file simply returns <TOP/>, 
>>while the same thing works perfectly fine when the  descendents 
>>size less than 100MB.(For my testing, I am just embedding XML files of
>>certain 
>>size as the descendents for each of the BLOCK.)
>>
>>
>><TOP> 
>>  <BLOCK1 id="1"> 
>>    ......
>>  </BLOCK1> 
>>  <BLOCK2 id="2"> 
>>    ......
>>  </BLOCK2> 
>></TOP>
>>
>>Inputs from your side regarding this issue will be of great help to me to.
>>My advance thank for your inputs. 
>>
>>Thanks,
>>Dinesh.V
>>
>>-------------------------------------------------------------------------
>>Take Surveys. Earn Cash. Influence the Future of IT
>>Join SourceForge.net's Techsay panel and you'll get the chance to share your
>>opinions on IT & business topics through brief surveys - and earn cash
>>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>>_______________________________________________
>>Xmltk-devs mailing list
>>Xml...@li...
>>https://lists.sourceforge.net/lists/listinfo/xmltk-devs
>>