I've read the format specs at your site, and concluded the following: VTD-XML can only handle up to 1.01GB XML documents, where the maximum size of an element is 1MB, and maximum depth of 255. I assume this is by design, but what should one do if he should file with larger limits into consideration?
Just a thought - would you care to use VInt (originally defined by Java Lucene Docs: http://lucene.apache.org/java/docs/fileformats.html#VInt\). This will make the format virtually limit-less, and keep VTDs for smaller files smaller (since VInt will use less than 30 bits for smaller numbers).
Hi, VTD-XML 's maximum document size (if ns is off) is 2GB...
the max doc size is 1GB when ns is turned on
the size of text is unlimited, as it internally concatenates/aggregates multiple 1 MB records so there is no limit on that ...
Also we are extending limits of VTD (by re-organize the layout of VTD to accommodate docs up to 128 GB)
The lucene stuff looks interesting, will investigate more...
you are welcome to join VTd-XML-users list and post any questions over there...
Nah, I'm subscribed to too many mailing lists already, so as long as I'm not a dev or a constant user of VTD-XML (yet), I will rather not too...
However, I'd really like to know what you will end up doing with regards to VInt. If you could just post here once you've done your invastigation, I'd really appreciate it.
Log in to post a comment.
Sign up for the SourceForge newsletter:
You seem to have CSS turned off.
Please don't fill out this field.