[Tclxml-users] event based parsing BUG

[Mats B. <ma...@pr...>]

Hi all,

I've found a really nasty bug in the xml-tcl parser when doing event based
parsing together with the '-final 0' option.
This typically happens when having a proc registered with fileevent
on sockets that just feeds the parser, and when any element callback
calls 'update' it triggers the event queue which feeds the parser
with xml, which therefore comes in the wrong order. Hope you get it...
It's a kind of reentrance problem.

This problem is isolated in the script below. The output from a simple
example is shown below, with and without the 'update' call.

I have a proposed fix for this by embedding the parsing in another
proc which is reentry safe. I think that this should go into the
tcl-xml parser, either as a separate method, or the default 'parse' method.
Opinion???

Best Wishes,   Mats

PS: You need my patched TclXML from "http://hem.fyristorg.com/matben/download/TclXML.tar.gz"
Glad to see that my fixes for '-final 0' and error callbacks work :-)

# OUTPUT ----------------------------------------------------

set data1 {<stream><base>Some text here</base><junk chop='off' }
set data2 {type='public'/><cut>more stuff</cut></stream>}
after idle [list $p parse $data2]
$p parse $data1

==> (Start) 'stream': 
==> (Start) 'base': 
==> (Data 'update') 'Some text here'             <-- the idle handler is called here
==> (Start) 'junk': type public chop off
==> (End) 'junk'
==> (Start) 'cut': 
==> (Data 'update') 'more stuff'
==> (End) 'cut'
==> (Error) errcode=illegalendtag, errmsg=end tag "stream" does not match open
element "base" around line 0
==> (End) 'base'

Correct:

==> (Start) 'stream': 
==> (Start) 'base': 
==> (Data) 'Some text here'
==> (End) 'base'
==> (Start) 'junk': type public chop off
==> (End) 'junk'
==> (Start) 'cut': 
==> (Data) 'more stuff'
==> (End) 'cut'
==> (End) 'stream'

I think we quit here
I think we quit here
    while executing
"HandleError illegalendtag {end tag "stream" does not match open element "base"
around line 0}"
    ("uplevel" body line 1)
    invoked from within
"uplevel #0 $options(-errorcommand) [list illegalendtag "end tag \"$tag\" does
not match open element \"[lindex $state(stack) end]\" around line $state..."
    (procedure "ParseEvent:ElementClose" line 9)
    invoked from within


#- EXAMPLE SCRIPT --------------------------------------

package require xml

set _wspc {[ \t\n\r]*}
proc HandleStart {name attlist args} {
    puts "==> (Start) '$name': $attlist"
}
proc HandleEnd {name args} {
    global  myParser
    puts "==> (End) '$name'"
}
proc HandleText {data} {
    global  _wspc
    if {![regexp "^${_wspc}$" $data]} {
	puts "==> (Data 'update') '$data'"
	update
    }
}
proc HandleError {errcode errmsg} {
    puts "==> (Error) errcode=$errcode, errmsg=$errmsg"
    return -code error {I think we quit here}
}
set p [xml::parser]
$p configure -final 0 -elementstartcommand HandleStart  \
  -elementendcommand HandleEnd -characterdatacommand HandleText \
  -errorcommand HandleError

set data1 {<stream><base>Some text here</base><junk chop='off' }
set data2 {type='public'/><cut>more stuff</cut></stream>}

after idle [list $p parse $data2]
$p parse $data1
#after idle [list wrap::parsereentrant $p $data2]
#wrap::parsereentrant $p $data1

#- FIX SCRIPT ----------------------------------------

namespace eval wrap {

    # A reference counter for reentries.
    variable refcount 0
    
    # Stack for xml.
    variable stack ""
}

proc wrap::parsereentrant {p xml} {
    
    variable refcount
    variable stack
    incr refcount
    if {$refcount == 1} {
	
	# This is the main entry: do parse original xml.
	puts "++> main entry: refcount=$refcount"
	$p parse $xml
	
	# Parse everything on the stack (until empty?).
	while {[string length $stack] > 0} {
	    puts "++> parse stack"
	    set tmpstack $stack
	    set stack ""
	    $p parse $tmpstack
	}
	puts "++> main exit: refcount=$refcount"
    } else {
	
	# Reentry, put on stack for delayed execution.
	puts "++> reentry: refcount=$refcount"
	append stack $xml
    }
    incr refcount -1
    return {}
}