Hi,
I'm using TinyXML to load a 1MB XML file, and it takes about 100 seconds for the TiXmlDocument::LoadFile function to process/complete.
I've tried enlarging the buffer size to 256KB, and it doesn't help very much.
Is there a way to speed loads up?
Thanks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've done a little bit of work trying to figure out why the library was so slow, and I'm embarassed to say it was trivial to discover it.
I was NOT using the STL string implementation, because as a rule, I find the STL is not quite ready for prime-time yet; most compilers have problems with the STL, a lot of STL 'vendors' have buggy implementations, etc...
So I figured it would be safe to use the tried and true CRT (C runtime library - strlen, strcat, etc...).
But here is where the problem lay - The CRT 'string' implemenation in TinyXML calls strlen and strcat quite a lot when it is building the document's nodes. This causes the XML strings to be scanned for a terminating NUL byte over and over, where the STL implementation of a string stores the length of the string inside its implementation and can use it in a 'cached' fashion, dramatically speeding up loading and parsing.
Possibly, a good solution would be to rework the CRT 'string' inside of TinyXML and have it do the same.
Thank you.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm kinda new to c++ [well, actually kinda old; returning to c++ after 5 years of java and discovering that c++ is not in any way like riding a bicycle!], and am very impressed with TinyXML, it's proving a real life-saver.
However I share your speed problem, and then some. It's my own fault, the XML document I'm generating/working with is over 3mb and it's taking very many minutes to parse.
So, any fast and dirty tips/patches to speed up the process would be warmly received.
many thanks,
ed.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I'm using TinyXML to load a 1MB XML file, and it takes about 100 seconds for the TiXmlDocument::LoadFile function to process/complete.
I've tried enlarging the buffer size to 256KB, and it doesn't help very much.
Is there a way to speed loads up?
Thanks.
I've done a little bit of work trying to figure out why the library was so slow, and I'm embarassed to say it was trivial to discover it.
I was NOT using the STL string implementation, because as a rule, I find the STL is not quite ready for prime-time yet; most compilers have problems with the STL, a lot of STL 'vendors' have buggy implementations, etc...
So I figured it would be safe to use the tried and true CRT (C runtime library - strlen, strcat, etc...).
But here is where the problem lay - The CRT 'string' implemenation in TinyXML calls strlen and strcat quite a lot when it is building the document's nodes. This causes the XML strings to be scanned for a terminating NUL byte over and over, where the STL implementation of a string stores the length of the string inside its implementation and can use it in a 'cached' fashion, dramatically speeding up loading and parsing.
Possibly, a good solution would be to rework the CRT 'string' inside of TinyXML and have it do the same.
Thank you.
See my post
Big files loading speed-up (non-STL)
http://sourceforge.net/forum/forum.php?thread_id=797723&forum_id=42748
I have made some speed improvements to the loading of the file as well as to the TiXmlOutStream class.
Since I'm not familiar with STL, I'm only using the Ansi C functions.
Please contact me if you're interested and I will send you the modifications.
- Rob
you really should familiarise your self with STL - they are real life savers, not to mentione that they are highly optimized ;)
Hi,
I'm kinda new to c++ [well, actually kinda old; returning to c++ after 5 years of java and discovering that c++ is not in any way like riding a bicycle!], and am very impressed with TinyXML, it's proving a real life-saver.
However I share your speed problem, and then some. It's my own fault, the XML document I'm generating/working with is over 3mb and it's taking very many minutes to parse.
So, any fast and dirty tips/patches to speed up the process would be warmly received.
many thanks,
ed.