tcllib

This problem is now more urgent than it had been before. I'm working with
JavaScript that is writing the HTML as it runs, including adding new
<SCRIPT>...</SCRIPT> code inside of JavaScript document.write statements.
htmlparse is getting seriously confused inside the JavaScript, and the
resulting nodes are truly mucking up my parse tree. Some nodes aren't even
at the right level any more, which makes my post processing lose them,
which is very bad news for my application.

Looking at your reference, it's pretty clear that the script tag cannot
end until a </[a-zA-Z] is found. Looks like the recognizer needs to be a
bit more discriminating.

I was able to work around the problem by adding spaces around the < in the
files I had access to (which would not always be the case). Pragmatically,
I don't know what the fix should be, but handling of SCRIPT might need to
be more special than it is.

Your right its valid, but there are some problems lurking in that area,
see:
http://www.w3.org/TR/html4/appendix/notes.html#notes-specifying-data

It works because the DTD for HTML4 declares <script> as having a CDATA
value, but may blow up in apps that don't fully respect the DTD. So its
dangerous ground...

I don't see anything in http://www.w3.org/TR/html4/interact/scripts.html
that would indicate that. Can you provide a reference? None of the examples
that I have seen on the web follow such a convention. (The CDATA section is
deprecated, according the the W3 page above.)

Wrong. The above is incorrect HTML. The JavaScript is incorrect, a < has
always to be quoted as &lt; unless it is inside a CDATA section.