tclxml-users — Discussion on using the TclXML package

[Tclxml-users] Fw: [tdom] Re: tclDOM or TDOM dilemma on Windows

[yahalom e. <Yah...@xo...>]

Hello

my personal experince with both packages tdom and dom points towards =
tdom.
I used the tcl implementation of dom on a commercial web application.
it used to much memory  needed to much cpu and it was slowwww. getting =
the C implementation to works failed (well I am not an expert in =
compling etc.. when things gets too compicated I loose it).
I moved to tdom (took me some time) but since then I am happy. parsing =
is fast and efficient.
I do had problem upgrading to 0.7 .I work with mod_dtcl and apache and =
version 0.7 crashes apache so I work with 0.5. I reported on this =
problem but it was not solved so support is a bit of an issue.
regarding tdom being in kitten. It is there but I tried it and if failed =
for me on linux.error :
387324couldn't load file "/tmp/tclWNC0ZN": /tmp/tclWNC0ZN: undefined =
symbol: Tcl_GetChannel
 dom worked (tcl parsing).


Thanks

Yahalom Emet
Software Developer
Xor Technologies Ltd.

Tel: +972 (0)4-6619164
E-mail: yah...@xo...
Web: http://www.xortechnologies.com


  ----- Original Message -----=20
  From: ro...@po...=20
  To: td...@ya...=20
  Cc: tcl...@li...=20
  Sent: Wednesday, January 08, 2003 6:38 AM
  Subject: Re: [tdom] Re: tclDOM or TDOM dilemma on Windows


  On  8 Jan, Steve Ball wrote:
  > Fra...@ce... wrote:
  >> My problem is to chose between the tclDOM and the tDOM families of
  >> development for using it on Windows environment, without having to =
compile
  >> anything (sorry I do that as a hobby and I have no time to =
configure Linux
  >> nor even to play with a Visual C compiler - although it was my job =
several
  >> years ago ;-} ).
  >=20
  > I'm also working with Jean-Claude Wippler to include these
  > packages in kitten, so that one can write a StarKit that
  > makes use of XML & XSLT.

  kitten already includes versions of both tDOM and TclXML et
  al. Therefor you could in deed write a Starkit that makes use of XML &
  XSLT, no matter which of the packages you choose.

  > At the C level I believe TclXML, et al, is more advanced.
  > The TclXML family is fully TEA2 compliant, is Stubs enabled
  > and sports a layered architecture that allows extensions.

  Well, tDOM has a TEA2 build system (thanks to Zoran), is stubs enabled
  and has an architecture that allows extensions.

  > There's also the issue of support.  Many more developers use
  > libxml2 & libxslt, so it is better tested.

  So you're suggesting, that especially libxslt is more compliant and
  bug free than the tDOM XSLT engine, don't you? For sure you have also
  some hard facts for this, beside the fussy marketing speech? (Visual
  Basic is used by much more developers than tcl and therefor much more
  mature than tcl, or what is the argument?)

  Before the release of the current tDOM version 0.7.5 I run it against
  a suite of much more than 1500 xslt test files. The suite is compiled
  together from the (enormous) test suite of the xalan XSLT project
  (this alone are almost 1300 texts), the NIST XSLT test suite (more
  than 170 tests), the XSLTmark test suite, the libxslt tests, the
  various examples of Michael Kays "XSLT - Programmer Reference" book
  and a couple of other sources. I compared the tDOM results with the
  libxslt-1.0.22 results - the current version at that time, they are at
  libxslt-1.0.23 now - (and saxon, xalan-j and sablotron btw). I found,
  that libxslt had notable more failures than the tDOM xslt engine.

  Don't get that wrong. I confirm, that libxslt provides a good, almost
  compliant XSLT processor. (As, for example also the sablotron folks
  do.) Most of the libxslt failures, that I found, are related to not so
  common XSLT features or constructs and therefor libxslt will do most
  of the 'real world' XSLT right.=20

  But I state, that tDOM also provides a good, almost compliant
  XSLT processor - and probably a bit better, at least in some areas,
  than libxslt. Steve, please would you share the test results with us,
  that make you belive, it isn't this way?

  > I have no
  > comparative usage numbers for TclDOM/TclXSLT vs tDOM,
  > but you can check for yourself on the SourceForge project
  > page.

  The sourceforge tDOM project is only a placeholder, registered by
  Jochen more than 2 years ago, to protect the project name. Since the
  tDOM sourceforge project space was never used, it suggest, that there
  is no code and no users. Both is definitely not true ;-)

  > As far as performance goes, tDOM may be slightly better in
  > some circumstances but compared to the performance of Java tools
  > the difference is trivial.

  Hm. Steve, would you please name only a view cirumstances, for which
  tDOM isn't _at least_ slightly better than TclXML etc., as far as
  performance goes? ;-)

  What speed difference could be named 'trivial' depends of course on
  the viewpoint. But I, for once, would not confirm, that the speed
  difference between tDOM and TclXML compaired with the speed of Java
  tools is completely "trivial". If the java virtual machine has started
  and the classes are already loaded (the java virtual machine has
  'warmed up'), the rough numbers are as following: Given a modern, fast
  Java dom implementation, libxml is around two times faster, tDOM with
  the expat parser 3 times and tDOM with the simple parser 4 times.

  Another point is the memory footprint. A DOM tree needs a lot of
  memory. A libxml DOM tree needs typically more than double as much
  memory, as the tDOM DOM tree of the same document. For example, a just
  chosen random 17 MByte XML file needs clearly more than 100 MByte
  memory, as libxml DOM tree, and less then 50 MByte as tDOM DOM tree
  (numbers measured at linux). Sure, if you process only small XML files
  on a box with reasonable memory, that may not bother you much.

  Both the raw xml parsing/DOM building speed and the memory footprint
  are results with only moderate variations. It don't really matter,
  what half-way realistic XML document you use, tDOM is faster, while
  using lesser memory.

  XPath and XSLT are much more complicated. There is clearly a greater
  variation of the results. But the overall picture - my results, please
  provide your own - is pretty clear: tDOM xslt is notable faster than
  libxslt. I would say XSLT benchmarking is still in its first
  days. XSLTmark (http://www.datapower.com/xml_community/xsltmark.html)
  is probably the most known XSLT benchmark suite, at the moment. This
  are 40 different Stylesheets, which run 10 to 100 times against
  different source documents from a few bytes up to 2 MBytes. With the
  default test configuration, the XSLT transformation time sums up for
  libxslt to around 140 seconds and for tDOM to 75 seconds.

  My to some degree extensive XSLT benchmarking in deed shows, that
  libxslt is for example mostly much faster than sablotron - well,
  sablotron isn't that bad in compliance, as already said, but that the
  sablotron folks claim at there homepage: "Sablotron is a fast [XSLT
  processor]" is ludicrous and only possible, because nobody measures by
  itself --- do you (and especially Steve: do you)? I found tDOMs xslt
  engine often cleary faster than libxslt and only very seldom the other
  way around. And there's a hole group of stylesheet/(bigger) document
  combinations, for which libxslt is ridiculous slow.

  If you like more 'real live' numbers, Simon Hefi reports
  (http://groups.yahoo.com/group/tdom/message/286), that tDOM transforms
  the almost 1,7 MByte DocBook document Securing.xml more than double as
  fast as libxslt. At my a bit ancient box, libxslt (xsltproc) needs
  more than 50 seconds for that transformation, and tDOM only 25. By the
  way, a "warmed up" saxon also needs only a bit more than 60
  seconds. In such cases. I think it would be possible to say: the speed
  difference of libxslt to Java tools is "trivial", compared with the
  tDOM speed.

  > Finally, at least for TclXML & TclDOM there is a pure-Tcl
  > version for completely compilation-free deployment.

  It should not be suppressed, that especially the scripted TclDOM has
  its limits with respect to speed and its immens memory demand.


  Don't get me wrong. For private use and a small 'content management
  system' I guess both packages are probably very well suited; choose,
  whatever you like more. Steve, greased with the words, as it is, has
  in deed some tutorial material, which may help you on track. For sure
  an additional convenient plus for TclXML et al. is, that it is
  included in the ActiveTcl (for windows: probably not so far in the
  future) distributions. The long mail is mostly because Steve tends to
  get the numbers a bit wrong.

  rolf




        Yahoo! Groups Sponsor=20
              ADVERTISEMENT
            =20
      =20
      =20

  To unsubscribe from this group, send an email to:
  tdo...@eg...



  Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.=20

[Tclxml-users] Re: tclDOM or TDOM dilemma on Windows

[Steve B. <Ste...@zv...>]

Fra...@ce... wrote:
> My problem is to chose between the tclDOM and the tDOM families of
> development for using it on Windows environment, without having to compile
> anything (sorry I do that as a hobby and I have no time to configure Linux
> nor even to play with a Visual C compiler - although it was my job several
> years ago ;-} ).
> 
> tclDOM family is now at version 2.5 but the version on  ActiveState is
> behind and didn't appear to work for my first little tests. However there is
> a lot of documentation available. TclXML2 parser seems to be linked to
> Gnome, which perhaps implies no plan for availability on Windows??? 

Not at all - late last year Andreas Kupries did some fine
work to make TclXML/expat, TclDOM/libxml2 and TclXSLT (libxslt)
part of ActiveTcl.  He has these building on all platforms supported
by ActiveTcl; Linux, Windows, Solaris and HP/UX.  Hence the
next release of ActiveTcl should have all of these packages
included for the Windows platform.  As to when that will become
available, well you'd have to ask him or Jeff Hobbs.

I'm also working with Jean-Claude Wippler to include these
packages in kitten, so that one can write a StarKit that
makes use of XML & XSLT.

> tDOM comes with a Windows DLL ready and although the syntax is different my
> first tests were successful. The documentation seems to be scarce.

My personal belief is that the two packages should have been merged
years ago, but my efforts to make that happen failed :-(

I'm certainly working on more documentation for TclXML, et al.
Also, my company (Zveno) has training material available for
these packages and it can also provide commercial support.

> What would be your recommendations? main functional and operational
> differences between the 2 approaches? availability plans for Windows?

At the Tcl scripting level both functionally and operationally
I don't think there's a big difference - trivially different syntax
but the basic functions of parsing, DOM and XSLT are supported
by both packages.

At the C level I believe TclXML, et al, is more advanced.
The TclXML family is fully TEA2 compliant, is Stubs enabled
and sports a layered architecture that allows extensions.

There's also the issue of support.  Many more developers use
libxml2 & libxslt, so it is better tested.  I have no
comparative usage numbers for TclDOM/TclXSLT vs tDOM,
but you can check for yourself on the SourceForge project
page.

As far as performance goes, tDOM may be slightly better in
some circumstances but compared to the performance of Java tools
the difference is trivial.

Finally, at least for TclXML & TclDOM there is a pure-Tcl
version for completely compilation-free deployment.
Alot of people find this very useful.

> Best Regards and many thanks to Steve Ball and Jochen Loewer for there
> respective big efforts.

No worries!

Cheers,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

[Tclxml-users] tclDOM or TDOM dilemma on Windows

[<Fra...@ce...>]

Happy new year!

After spending some time trying to parse and transform html and xml by =
hand
(regexp) in tcl - for a kind of personal web content management system =
- I
discovered all nice work done in many packages: html, htmlparse, http =
and
more importantly tclXML and its associated packages tclDOM and tclXSLT =
and
the alternative package tDOM.

My problem is to chose between the tclDOM and the tDOM families of
development for using it on Windows environment, without having to =
compile
anything (sorry I do that as a hobby and I have no time to configure =
Linux
nor even to play with a Visual C compiler - although it was my job =
several
years ago ;-} ).

tclDOM family is now at version 2.5 but the version on  ActiveState is
behind and didn't appear to work for my first little tests. However =
there is
a lot of documentation available. TclXML2 parser seems to be linked to
Gnome, which perhaps implies no plan for availability on Windows???=20

tDOM comes with a Windows DLL ready and although the syntax is =
different my
first tests were successful. The documentation seems to be scarce.

What would be your recommendations? main functional and operational
differences between the 2 approaches? availability plans for Windows?

Best Regards and many thanks to Steve Ball and Jochen Loewer for there
respective big efforts.

Fran=E7ois JUNIQUE=20

Re: [Tclxml-users] tcldom 2.5 question

[Andreas K. <aku...@sh...>]

> Larry W. Virden wrote:
> > I am seeing the following behavior that seems strange to me - perhaps
> > I am misunderstanding something?
> > 
> > $ tclsh
> > % package require dom::tcl
> > can't read "::dom::strictDOM": no such variable
> > % package require dom::generic
> > 2.5
> > % package require dom::tcl
> > 2.5
> > 
> > If dom::tcl requires dom::generic to be loaded, then should it not do
> > a package require on it?
> 
> Yes, that appears to be correct.  Please submit a bug report
> on SF.
> 
> I haven't picked it up before now because I normally
> use 'package require dom'.

I do the same, i.e. 'package require dom'. So far I considered this
the official / blessed invocation of dom which loads everything
required and possible.

Despite this I am _not_ adverse to having 'package require' work for
all of the subordinate packages too.

This is something I did not touch when creating the TEA 2 build system
for all the tclxml/dom/xslt packages.

-- 
So long,
	Andreas Kupries <aku...@sh...>
			<http://www.purl.org/NET/akupries/>
	Developer @	<http://www.activestate.com/>
-------------------------------------------------------------------------------

Re: [Tclxml-users] problems configuring the newest tclxml

[Andreas K. <aku...@sh...>]

> Larry W. Virden wrote:

>> When I attempt to configure the tclxml/expat directory, I get this
>> error:

>> checking for Tclxml configuration... configure: WARNING: "Cannot
>> find Tclxml configuration definitions"

>> This is _after_ having installed the tclxml tcl and tclxml C code.

> Did you use exactly the same configure command that you used for the
> generic layer installation?
 
> I have found that one needs to specify *both* the --prefix and
> --exec_prefix options to get it all to work properly.  Don't know
> why exactly, and Andreas has gone on holidays so we can't ask him
> until January.

I don't know either. I believe this is somehow embedded in the
configure macros Jeff created for TEA 2.

-- 
Sincerely,
	Andreas Kupries <aku...@sh...>
	Developer @	<http://www.activestate.com/>
	Private		<http://www.purl.org/NET/akupries/>
-------------------------------------------------------------------------------

Re: [Tclxml-users] tcldom 2.5 question

[Steve B. <Ste...@zv...>]

Larry W. Virden wrote:
> I am seeing the following behavior that seems strange to me - perhaps
> I am misunderstanding something?
> 
> $ tclsh
> % package require dom::tcl
> can't read "::dom::strictDOM": no such variable
> % package require dom::generic
> 2.5
> % package require dom::tcl
> 2.5
> 
> If dom::tcl requires dom::generic to be loaded, then should it not do
> a package require on it?

Yes, that appears to be correct.  Please submit a bug report
on SF.

I haven't picked it up before now because I normally
use 'package require dom'.

Cheers,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

[Tclxml-users] tcldom 2.5 question

[Larry W. V. <lv...@ca...>]

I am seeing the following behavior that seems strange to me - perhaps
I am misunderstanding something?

$ tclsh
% package require dom::tcl
can't read "::dom::strictDOM": no such variable
% package require dom::generic
2.5
% package require dom::tcl
2.5

If dom::tcl requires dom::generic to be loaded, then should it not do
a package require on it?
-- 
Tcl - The glue of a new generation.  <URL: http://wiki.tcl.tk/ >
Larry W. Virden <mailto:lv...@ca...> <URL: http://www.purl.org/NET/lvirden/>
Even if explicitly stated to the contrary, nothing in this posting should 
be construed as representing my employer's opinions.
-><-

Re: [Tclxml-users] problems configuring the newest tclxml

[Larry W. V. <lv...@ca...>]

From: Steve Ball <Ste...@zv...>

> Did you use exactly the same configure command that you
> used for the generic layer installation?

Yes

> I have found that one needs to specify *both* the --prefix
> and --exec_prefix options to get it all to work properly.

I didn't do this though - I will try that.
-- 
Tcl - The glue of a new generation.  <URL: http://wiki.tcl.tk/ >
Larry W. Virden <mailto:lv...@ca...> <URL: http://www.purl.org/NET/lvirden/>
Even if explicitly stated to the contrary, nothing in this posting should 
be construed as representing my employer's opinions.
-><-

Re: [Tclxml-users] problems configuring the newest tclxml

[Steve B. <Ste...@zv...>]

Larry W. Virden wrote:
> When I attempt to configure the tclxml/expat directory, I get this
> error:
> 
> checking for Tclxml configuration... configure: WARNING: "Cannot find Tclxml configuration definitions"
> 
> This is _after_ having installed the tclxml tcl and tclxml C code.

Did you use exactly the same configure command that you
used for the generic layer installation?

I have found that one needs to specify *both* the --prefix
and --exec_prefix options to get it all to work properly.
Don't know why exactly, and Andreas has gone on holidays
so we can't ask him until January.

HTHs,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

[Tclxml-users] problems configuring the newest tclxml

[Larry W. V. <lv...@ca...>]

When I attempt to configure the tclxml/expat directory, I get this
error:

checking for Tclxml configuration... configure: WARNING: "Cannot find Tclxml configuration definitions"

This is _after_ having installed the tclxml tcl and tclxml C code.
-- 
Tcl - The glue of a new generation.  <URL: http://wiki.tcl.tk/ >
Larry W. Virden <mailto:lv...@ca...> <URL: http://www.purl.org/NET/lvirden/>
Even if explicitly stated to the contrary, nothing in this posting should 
be construed as representing my employer's opinions.
-><-

Re: [Tclxml-users] fast xml parsing into a tclList

[<mis...@sy...>]

Ok I think I should explain  a little more what xml2tcl does:

I get big XML-replies with about lets say 5MByte and all I need is the
characterData inbetween tags which is put into a tclList. But I still need
the nested structure of the XML-document in my new List.

The xml2tcl command takes one argument (big XML-doc) and returns
a nested tclList. As I said this takes about 6MByte/Sec. (PIII 866)

Jochen: Your tdom parser seems to be even faster. Has it the same
base functionality as expat? I will try use it with my code.

If anyone is interested in the code or has some webspace to put it
I will be glad to contribute it.

Thanks Mischa

p.s.: Steve I looked at the code  you mentioned but at the moment I
can't do the upgrade and maintain it - sorry.

[Tclxml-users] Re: Question on -final usage in tclxml-2.4

[Mats B. <ma...@pr...>]

Update:

I just tested this once again with a slightly modified callback, 
which shows that there are no extra newlines; it's just a callback
with a completely empty (nothing) cdata. It's the newline of the
puts we have seen. 
See script below and verbatim output from the same script.

/Mats

package require xml

proc cdata {data args} {
    puts "(data)==>'$data'"
}
set parser [::xml::parser parseit \
  -characterdatacommand cdata -final 0]

$parser parse "<the>"
$parser parse "world"
$parser parse "</the>"

From cvs Dec 10, but without patch #596959
(I still used a hacked up pkgIndex.tcl file, therefore 3.0):

% package require xml
3.0
(Build) 2 % 
(Build) 3 % proc cdata {data args} {
>     puts "(data)==>'$data'"
> }
(Build) 4 % set parser [::xml::parser parseit \
>   -characterdatacommand cdata -final 0]
parseit
(Build) 5 % 
(Build) 6 % $parser parse "<the>"
(data)==>''
(Build) 7 % $parser parse "world"
(data)==>'world'
(Build) 8 % $parser parse "</the>"
(data)==>''

[Tclxml-users] Re: Question on -final usage in tclxml-2.4

[Mats B. <ma...@pr...>]

Hi all,

Just one clarification. I have not written tclxml. I just found the
problem with -final 0 for my Jabber client and fixed it to the best
of my knowledge. Fixing bugs in others code is not the funniest, and
tclxml is one of the most complex piece of tcl code I've ever seen.
End of statement.

Steve Ball wrote:
> 
> Mats Bengtsson wrote:
> > Just do:
> >       if {0} {
> >       # Patch from bug report #596959, Marshall Rose
> >       if {[string compare [lindex $sgml 4] ""]} {
> >           set sgml [linsert $sgml 0 {} {} {} {} {}]
> >       }
> >       }
> >
> > I don't know the reason for this code. This must be sorted out
> > by the author. This would never have happened if there were a test
> > case with -final 0 and chopped off xml. So, please someone,
> > add this so we wont see bugs like this again.
> 
> It is quite clear that making this change will break
> regression tests.  At this stage we need three things:

The bug #596959 is a fix for chopped of xml as I understand it from
the desciption at SF bugtracker. But this is fixed in a much more elegant
way by my earlier patch (see sgml::tokenise) which enables you to feed
the parser with xml that is chopped off anywhere.

The core part of the parser is the 
 	regsub -all $elemExpr $sgml $elemSub sgml
in sgml::tokenise, with
    variable tokExpr <(/?)( too long.... )>
    variable substExpr "\}\n{\\2} {\\1} {\\3} \{"
with elemExpr=tokExpr, and elemSub=substExpr.
I've traced this code to Brent Welch's book 2ed, and he refers
this trick to Stephen Uhler.
The trick here is that each tag is matched by each regsub'stitution,
and all inbetween subsequent matches, which is cdata, comes after
the open brace to the right, and before the close brace to the left.
The \n was used in Welch's book to produce tcl code with a command
on each line which is used to render html. But I don't see why it
is used here! Why is not the sgml handled as a list?
My suspicion is that the extra spurios \n come from this regexp.
Rewriting this is a major task, and right now I don't have time or
energy to do this.

> 
> 1. A test case for bug #596959
> 2. A test case for bug #413341
> 3. Make sure that *both* tests pass

I think both bugs refer to the same problem. I add some
xml as a postscript to this mail.

> 
> According to Marshall's bug report, the Jabber client's
> use of TclXML tickled the bug so applying your change
> will break that program.

I'm not compleletly sure of this, see above.

> Don't you set '-final 1' at all?  Setting that configuration
> option to 1 from 0 does a final check that the document
> is well-formed.  Consider:
> 
>         $parser parse "<the>"
>         $parser parse "world"
> 
> "<the>world" is not well-formed XML, but you will never
> know that because the parsing ends and no error is raised.
> 

With -final 0 you usually never know when the document ends.
You usually know that after you've got the last xml, and then
its too late. Kind of catch 22. In the Jabber case, you read
xml from a network socket, so there you only know that it was the last
chunk when the connection closed down, or if you got the final
</stream>.

It could perhaps be useful to have a new command that checks
if document so far was final (isducumentfinal).
This could check [llength $state(stack)] which should be 0
when finished, since state(stack) is a list with all unmatched
open tags in hierarchy.

/Mats


set parser [::xml::parser parseit \
        -characterdatacommand cdata -final 0]
# ex 1
$parser parse "<the>"
$parser parse "world"
$parser parse "</the>"
# ex 2
$parser parse <stream><ju
$parser parse "nk attr='lo"
$parser parse "se>Hello Wo"
$parser parse "rld</junk"
$parser parse "></stream>"

or something like this, with appropriate callbacks.

Re: [Tclxml-users] Error compiling TclXML & TclExpat with MS Visual Studio 6.0

[Steve B. <Ste...@zv...>]

Rob Flynn wrote:
> I recently downloaded TclXML v2.5 and have attempted to compile it with VS
> 6.0.
> 
> I was able to compile expatlib without errror, however, trying to compile
> TclXml results in the following error:
> Linking...
>    Creating library Release/tclxml.lib and object Release/tclxml.exp
> tclxml.obj : error LNK2001: unresolved external symbol _tclxmlStubs
> Release/tclxml.dll : fatal error LNK1120: 1 unresolved externals
> Error executing link.exe.

Sounds like a problem with linking against the stubs library...
but I don't much about Windoze... (thank heavens!)

> I eventually managed to force both TclXML.dll and TclExpat.dll to be
> generated.  Upon trying to use the code I found that I was only able to
> parse documents once per parser.  I began resetting the parser after each
> parse attempt which resolved the situation.

You're supposed to reset the parser before parsing another
document.  Either that or destroy the old parser and create
a new, fresh one.

> Later, however, I ran into a situation where the parser (i don't recall if
> it was in tclxml or tclexpat) seemed to cause a violation of memory.  I'm it
> tried to return a NULL pointer when a non-NULL was expected.  I managed to
> comment out a few lines and force some returns to be specific values and was
> able to get the library to parse properly.  I, however, feel that this
> really isn't an optimum solution.

'::xml::parserclass info default' will tell you what is the
default parser.  That may help in tracking down the cause
of the problem.

Passing a NULL pointer sounds like a bug to me, and at the
very least the offending code should check that and deal with
it without crashing.  Please submit a bug report on SF,
including sample script, XML document and the patch that
you applied.

Cheers,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

Re: [Tclxml-users] fast xml parsing into a tclList

[Steve B. <Ste...@zv...>]

Mischa Diehm wrote:
> I was very happy to find the TclXml package with it's nice
> features. I have to parse XML and get the structure back in a
> tclList for further processing. I used callbacks to get the job done
> and it worked perfectly well. But if I get big XML-Structures (like over
> one MByte) the parsing  takes way too much time. After checking the
> documentation and trying to get my calls lighter ( as far as my knowledge
> goes )
> and some checks on what causes the speed loss I realized that it is due to
> the
> parser itself.

That's perfectly understandable if you were using the
Tcl implementation of the parser.  However, TclXML already
has a wrapper for Expat - did you try that?

> I wrote a little package xml2tcl in C using the C-expat-1.95.0
> implementation.
> On a PIII with 863Mhz I can now parse about 6 MByte/Sec which is fast
> enaugh
> for my use. Is there any interest for this here and where can I put the
> source or
> the shared lib?

As I said, TclXML already has a wrapper for Expat - a very
old version heavily modified by Ajuba.  It would be very
helpful to update the TclXML/expat code to use the version
that's now on SourceForge (as I recall that is the
expat-1.95.0 version that you mention).

Submit the patches on the TclXML SF site and I will
incorporate them.  If you are willing to be responsible
for ongoing maintenance then I'll add you as a developer
to the project.

Cheers,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

[Tclxml-users] fast xml parsing into a tclList

[<mis...@sy...>]

Hi,

I was very happy to find the TclXml package with it's nice
features. I have to parse XML and get the structure back in a
tclList for further processing. I used callbacks to get the job done
and it worked perfectly well. But if I get big XML-Structures (like over
one MByte) the parsing  takes way too much time. After checking the
documentation and trying to get my calls lighter ( as far as my knowledge
goes )
and some checks on what causes the speed loss I realized that it is due to
the
parser itself.
I wrote a little package xml2tcl in C using the C-expat-1.95.0
implementation.
On a PIII with 863Mhz I can now parse about 6 MByte/Sec which is fast
enaugh
for my use. Is there any interest for this here and where can I put the
source or
the shared lib?

Greets Mischa

[Tclxml-users] Error compiling TclXML & TclExpat with MS Visual Studio 6.0

[Rob F. <tc...@so...>]

All -

I recently downloaded TclXML v2.5 and have attempted to compile it with VS
6.0.

I was able to compile expatlib without errror, however, trying to compile
TclXml results in the following error:
Linking...
   Creating library Release/tclxml.lib and object Release/tclxml.exp
tclxml.obj : error LNK2001: unresolved external symbol _tclxmlStubs
Release/tclxml.dll : fatal error LNK1120: 1 unresolved externals
Error executing link.exe.

I eventually managed to force both TclXML.dll and TclExpat.dll to be
generated.  Upon trying to use the code I found that I was only able to
parse documents once per parser.  I began resetting the parser after each
parse attempt which resolved the situation.

Later, however, I ran into a situation where the parser (i don't recall if
it was in tclxml or tclexpat) seemed to cause a violation of memory.  I'm it
tried to return a NULL pointer when a non-NULL was expected.  I managed to
comment out a few lines and force some returns to be specific values and was
able to get the library to parse properly.  I, however, feel that this
really isn't an optimum solution.

Can anyone here offer me any advice on what's going wrong?

- Rob

Re: [Tclxml-users] Re: Question on -final usage in tclxml-2.4

[Steve B. <Ste...@zv...>]

Mats Bengtsson wrote:
> I checked out a fresh copy of tclxml (version 2.5 released today),
> and found that most (all?) of my changes already were applied
> (by Steve Ball presumably). I've been running my patched tclxml
> daily for a year, with a lot of chopped off xml, and that works fine.
> However, the cvs version does not work as previously noted.
> There are some more changes to the cvs version compared to my version.
> The main difference seems to be in sgmlparser, and after some
> search I found the code:
> 	# Patch from bug report #596959, Marshall Rose
> to be the reason.
> Just do:
> 	if {0} {
> 	# Patch from bug report #596959, Marshall Rose
> 	if {[string compare [lindex $sgml 4] ""]} {
> 	    set sgml [linsert $sgml 0 {} {} {} {} {}]
> 	}
> 	}
> 
> I don't know the reason for this code. This must be sorted out
> by the author. This would never have happened if there were a test
> case with -final 0 and chopped off xml. So, please someone,
> add this so we wont see bugs like this again.

It is quite clear that making this change will break
regression tests.  At this stage we need three things:

1. A test case for bug #596959
2. A test case for bug #413341
3. Make sure that *both* tests pass

According to Marshall's bug report, the Jabber client's
use of TclXML tickled the bug so applying your change
will break that program.

> There remains a question of how to actually use the -final option.
> I use it always when there is a risc of incomplete xml.
> Is there any reason for using -final 1? What's the advantage?

The simpler case when you know that the document is complete.
See also below.

> Test case on mac (I've a special pkgIndex file that sets version to 3.0 in order
> to not interfere with other installations):

Until we release v3.0...

> % package require xml
> 3.0
> (Build) 2 % 
> (Build) 3 % proc cdata {data args} {
> 
>>        puts $data
>>}
> 
> (Build) 4 % 
> (Build) 5 % set parser [::xml::parser parseit \
> 
>>        -characterdatacommand cdata -final 0]
> 
> parseit
> (Build) 6 % 
> (Build) 7 % $parser parse "<the>"
> 
> (Build) 8 % $parser parse "world"
> world
> (Build) 9 % $parser parse "</the>"

Don't you set '-final 1' at all?  Setting that configuration
option to 1 from 0 does a final check that the document
is well-formed.  Consider:

	$parser parse "<the>"
	$parser parse "world"

"<the>world" is not well-formed XML, but you will never
know that because the parsing ends and no error is raised.

Cheers,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

Re: [Tclxml-users] Re: Question on -final usage in tclxml-2.4

[Ted N. S. A. GA <te...@ag...>]

In message <3DF...@pr...>you write:
>
>Hi all,
>
>I checked out a fresh copy of tclxml (version 2.5 released today),
>and found that most (all?) of my changes already were applied
>(by Steve Ball presumably). I've been running my patched tclxml
>daily for a year, with a lot of chopped off xml, and that works fine.
>However, the cvs version does not work as previously noted.
>There are some more changes to the cvs version compared to my version.
>The main difference seems to be in sgmlparser, and after some
>search I found the code:
>	# Patch from bug report #596959, Marshall Rose
>to be the reason.
>Just do:
>	if {0} {
>	# Patch from bug report #596959, Marshall Rose
>	if {[string compare [lindex $sgml 4] ""]} {
>	    set sgml [linsert $sgml 0 {} {} {} {} {}]
>	}
>	}
>
>I don't know the reason for this code. This must be sorted out
>by the author. This would never have happened if there were a test
>case with -final 0 and chopped off xml. So, please someone,
>add this so we wont see bugs like this again.
>
>There remains a question of how to actually use the -final option.
>I use it always when there is a risc of incomplete xml.
>Is there any reason for using -final 1? What's the advantage?
>
>In case there are remaining problems you can always extract
>my patched TclXML form my whiteboard application.
>See "http://hem.fyristorg.com/matben/"
>
>Best Wishes,   Mats

Mats,

Thanks for the input.  I applied your change to dike out
the Rose patch from 2.5 (though I do find the fact that
Marshall Rose is using tclxml a nice endorsement..)

It does not crash any longer, but I can't say that I consider
the result correct.  It's not what I expect at any rate..

Here's the test case with -final 1 (default)


####
package require xml

proc cdata {data args} {
	puts $data
}

set parser [::xml::parser parseit \
	-characterdatacommand cdata ]

$parser parse "<the>world</the>"
####

It outputs:

solabel10% ./doit | od -c
0000000   w   o   r   l   d  \n
0000006

This makes sense to me.

Here's the test case with -final 0..


####
package require xml

proc cdata {data args} {
	puts $data
}

set parser [::xml::parser parseit \
	-characterdatacommand cdata -final 0 ]

$parser parse "<the>"
$parser parse "world"
$parser parse "</the>"
$parser configure -final 1
####

It outputs:

solabel10% ./doit2 | od -c
0000000  \n   w   o   r   l   d  \n  \n
0000010

Note the extraneous leading newline, and the
extraneous trailing one (one comes from the "puts" of
course..)

				Ted

[Tclxml-users] Re: Question on -final usage in tclxml-2.4

[Mats B. <ma...@pr...>]

Hi all,

I checked out a fresh copy of tclxml (version 2.5 released today),
and found that most (all?) of my changes already were applied
(by Steve Ball presumably). I've been running my patched tclxml
daily for a year, with a lot of chopped off xml, and that works fine.
However, the cvs version does not work as previously noted.
There are some more changes to the cvs version compared to my version.
The main difference seems to be in sgmlparser, and after some
search I found the code:
	# Patch from bug report #596959, Marshall Rose
to be the reason.
Just do:
	if {0} {
	# Patch from bug report #596959, Marshall Rose
	if {[string compare [lindex $sgml 4] ""]} {
	    set sgml [linsert $sgml 0 {} {} {} {} {}]
	}
	}

I don't know the reason for this code. This must be sorted out
by the author. This would never have happened if there were a test
case with -final 0 and chopped off xml. So, please someone,
add this so we wont see bugs like this again.

There remains a question of how to actually use the -final option.
I use it always when there is a risc of incomplete xml.
Is there any reason for using -final 1? What's the advantage?

In case there are remaining problems you can always extract
my patched TclXML form my whiteboard application.
See "http://hem.fyristorg.com/matben/"

Best Wishes,   Mats


Test case on mac (I've a special pkgIndex file that sets version to 3.0 in order
to not interfere with other installations):

% package require xml
3.0
(Build) 2 % 
(Build) 3 % proc cdata {data args} {
>         puts $data
> }
(Build) 4 % 
(Build) 5 % set parser [::xml::parser parseit \
>         -characterdatacommand cdata -final 0]
parseit
(Build) 6 % 
(Build) 7 % $parser parse "<the>"

(Build) 8 % $parser parse "world"
world
(Build) 9 % $parser parse "</the>"

[Tclxml-users] Re: Question on -final usage in tclxml-2.4

[Mats B. <ma...@pr...>]

> > This seems like a pretty basic thing.  Any suggestions?
> 
> There is already an open bug on "-final" in the SF bug
> tracker (#413341).  Please feel free to append your comments
> to that bug.  The bug is currently assigned to Mats, and may be
> worthwhile reminding him (gently) to fix it ;-)
> 
> Of course, if you can provide a patch I would be only too
> happy to incorporate it myself and close the bug report if
> it is fixed.

I have just managed to get cvs to work on my Mac OS X box
(well it worked right away, but I had to get used to it),
and I was actually thinking of fixing this, eventually.
Must be mindreading or something.

I'll post to this list when fixed.

/Mats

Re: [Tclxml-users] Question on -final usage in tclxml-2.4

[Steve B. <Ste...@zv...>]

Ted Nolan SRI Augusta GA wrote:
> Thanks.  It seems pretty clear to me after looking at it for
> a bit that the following patch is required, but not suficient:
[...snip...]
> This fixes the 
> 
>   "can't read "opts(-statevariable)": no such variable while executing"
> 
> problem, but something else is wrong as well.

I'll apply that patch.

Can you also supply a test case(s), in the form of a patch
to the test suite, that demonstrates the bug?

Cheers,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

Re: [Tclxml-users] Question on -final usage in tclxml-2.4

[Ted N. S. A. GA <te...@ag...>]

>
>There is already an open bug on "-final" in the SF bug
>tracker (#413341).  Please feel free to append your comments
>to that bug.  The bug is currently assigned to Mats, and may be
>worthwhile reminding him (gently) to fix it ;-)
>
>Of course, if you can provide a patch I would be only too
>happy to incorporate it myself and close the bug report if
>it is fixed.
>

Steve,

Thanks.  It seems pretty clear to me after looking at it for
a bit that the following patch is required, but not suficient:

solabel10% diff -c sgmlparser.tcl $a
*** sgmlparser.tcl	Thu Sep 12 17:13:46 2002
--- /home/solabel10/ted/wdc/src/uc/lib/tclxml2.4/sgmlparser.tcl	Mon Dec  9 12:46:41 2002
***************
*** 151,157 ****
      if {[info exists options(-statevariable)]} {
  	# Mats: Several rewrites here to handle -final 0 option.
  	# If any cached unparsed xml (state(leftover)), prepend it.
! 	upvar #0 $opts(-statevariable) state
  	if {[string length $state(leftover)]} {
  	    regsub -all $elemExpr $state(leftover)$sgml $elemSub sgml
  	    set state(leftover) {}
--- 151,157 ----
      if {[info exists options(-statevariable)]} {
  	# Mats: Several rewrites here to handle -final 0 option.
  	# If any cached unparsed xml (state(leftover)), prepend it.
! 	upvar #0 $options(-statevariable) state
  	if {[string length $state(leftover)]} {
  	    regsub -all $elemExpr $state(leftover)$sgml $elemSub sgml
  	    set state(leftover) {}
solabel10% 

This fixes the 

  "can't read "opts(-statevariable)": no such variable while executing"

problem, but something else is wrong as well.


			Ted

Re: [Tclxml-users] Question on -final usage in tclxml-2.4

[Steve B. <Ste...@zv...>]

Dear Ted,

You wrote:
> I am transitioning a program I had running with tclexpat to use
> tclxml.  I had some problems with -final in tclexpat, (had to patch the
> code someway or another) and seem to be finding something similar here
> also.
> 
> Here is my first test script
[...snip...]

> As expected, it prints
> 
> 	world

Good - that's a start ;-)

> Here is my second test script
> 
> ====Start====
> #!/usr/local/bin/tclsh8.4
> 
> lappend auto_path lib
> 
> package require xml
> 
> proc cdata {data args} {
> 	puts $data
> }
> 
> set parser [::xml::parser parseit \
> 	-characterdatacommand cdata -final 0]
> 
> $parser parse "<the>"
> $parser parse "world"
> $parser parse "</the>"
> ====End====
> 
> It does not print "world" and errors out with a long backtrace:

Your script should include:

$parser configure -final 1

at the end, just to wind-up the parser.  As it is, I would expect
it to at least not produce an error, if not to emit "world".

> This seems like a pretty basic thing.  Any suggestions?

There is already an open bug on "-final" in the SF bug
tracker (#413341).  Please feel free to append your comments
to that bug.  The bug is currently assigned to Mats, and may be
worthwhile reminding him (gently) to fix it ;-)

Of course, if you can provide a patch I would be only too
happy to incorporate it myself and close the bug report if
it is fixed.

> Please reply directly to me as well as to the list because sourceforge
> seems to be unhappy today, and I have not been able to subscribe so
> far.

Fixed.  You are now subscribed.  (New subscribers must be
approved, to avoid spammers having access to the list).

Cheers,
Steve Ball

-- 
Steve Ball            |   XSLT Standard Library   | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |   XML XSL Schemas
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099

[Tclxml-users] Question on -final usage in tclxml-2.4

[Ted N. S. A. GA <te...@ag...>]

Hello folks,

I am transitioning a program I had running with tclexpat to use
tclxml.  I had some problems with -final in tclexpat, (had to patch the
code someway or another) and seem to be finding something similar here
also.

I am using the pure tcl version of tclxml version 2.4.  It is installed
in a director "lib" beneath my current directory, as is tcllib1.2

Here is my first test script

====Start====
#!/usr/local/bin/tclsh8.4

lappend auto_path lib

package require xml

proc cdata {data args} {
	puts $data
}

set parser [::xml::parser parseit \
	-characterdatacommand cdata ]

$parser parse "<the>world</the>"
====End====

As expected, it prints

	world

Here is my second test script

====Start====
#!/usr/local/bin/tclsh8.4

lappend auto_path lib

package require xml

proc cdata {data args} {
	puts $data
}

set parser [::xml::parser parseit \
	-characterdatacommand cdata -final 0]

$parser parse "<the>"
$parser parse "world"
$parser parse "</the>"
====End====

It does not print "world" and errors out with a long backtrace:


====Start====
  can't read "opts(-statevariable)": no such variable
      while executing
  "upvar #0 $opts(-statevariable) state"
      (procedure "::sgml::tokenise" line 29)
      invoked from within
  "::sgml::tokenise $xml $tokExpr $substExpr -internaldtdvariable ::xml::tclparser::parseit(internaldtd) -statevariable ::xml::tclparser::parseit -final ..."
      ("eval" body line 1)
      invoked from within
  "eval {::sgml::tokenise $xml $tokExpr $substExpr} $tokenOptions"
      (procedure "parse" line 57)
      invoked from within
  "parse parseit <the>"
      (in namespace inscope "::xml::tclparser" script line 1)
      invoked from within
  "::namespace inscope ::xml::tclparser parse parseit <the>"
      ("eval" body line 1)
      invoked from within
  "eval $classinfo(-parsecommand) [list $name] $args"
      ("parse" arm line 5)
      invoked from within
  "switch -- $method {
  
  	configure {
  	    # BUG: We're not checking for legal options
  	    array set data $args
  	    eval $classinfo(-configurecommand) [..."
      (procedure "::xml::ParserCmd" line 7)
      invoked from within
  "::xml::ParserCmd parseit parse <the>"
      ("eval" body line 1)
      invoked from within
  "eval ::xml::ParserCmd parseit [list $method] $args"
      (procedure "parseit" line 1)
      invoked from within
  "$parser parse "<the>""
      (file "./doit2" line 14)
====End====

This seems like a pretty basic thing.  Any suggestions?

Please reply directly to me as well as to the list because sourceforge
seems to be unhappy today, and I have not been able to subscribe so
far.

			Thanks,

			Ted Nolan