Re: [Flightgear-devel] A puzzling problem with FG XML parsing

[Ron J. <wi...@je...>]

On Monday 13 September 2010 19:57:55 Alasdair wrote:
> I have recently been delving into FG's innards, with a view to fixing
> some problems with the speech handling capabilities. This has led
> indirecly to my having to understand how XML files are parsed.
>
> Now in SG's XML folder is a program called easyxml.cxx which reads the
> XML files before passing contents to expat's xmlparse.c. The problem I
> have discovered is that an entire XML file is read in _one_ chunk into a
> 16k buffer declared at line 222. A dozen or so lines later a read
> operation is performed which reads the _entire_ XML file into that
> buffer. Now since preferences.xml is more than 3 times the size of this
> only one third of it gets parsed, with means that the final partial line
> that makes it into the buffer is in an indeterminate state. That happens
> arount line 410 of preferences.xml. Furthermore garbage is passed to the
> parser when when the end of the buffer is reached on readback.
>
> I am so astonished with this discovery that I can barely believe the
> evidence of my eyes and my tests.
>
> Can someone confirm that I have not gone loopy? Because if I am right
> this could explain a lot of strange goings on. Where am I going wrong?
>
> Kind regards,
>
> Alasdair Campbell

Sorry Alasdair, you've looped.  :)  Or more correctly, you missed the loop!

Simplifying from line 223 we have:
 while (!input.eof())
 {
    input.read(buf,16384);
    XML_Parse(parser, buf, input.gcount();
}

Everything else is just error checking and such.  So we read the file in 16k 
chunks until we've read the whole thing.  I haven't looked at what goes on in 
XML_Parse() for partial lines and tags, but we feed a lot of files through 
easyxml that are larger than 16k bytes...

Ron