[Tclxml-users] Re: Accurate?

[Steve B. <Ste...@zv...>]

Hi Cameron,

You wrote:
> 
> I'll not have time to work up code examples.  I'd welcome
> 'em from you.

OK.  The code examples might be of interest to the TclXML mailing list,
so I'm cc'ing the list.

The aim of Zwik is to have a content management system
that is configured using XML documents.  That way, the XML
configuration files can themselves be managed by the system.
Also, the one common factor of users of the system is that they
know XML, but not any particular programming language.

Firstly, let's look at the config file.  This file specifies
which assets get processed, how they're processed and where 
the results go.  To do this, the config file requires a range of
primitives to perform the processing, such as invoking an
XSLT processor.

So, here's a small config file:

<actions>
  <action>
    <directory match="^/zveno/information$"/>
    <name>Make Website</name>

    <parameters>
      <targetdir>/zveno/information/public</targetdir>
    </parameters>

    <build:sequence>
      <action ref="makeZvenoInfo"/>
    </build:sequence>
  </action>

  <action id="makeZvenoInfo">
    <name>Make Zveno Info Section</name>
    <parameters>
      <sourcedir>/zveno/information/zveno_information</sourcedir>
      <targetdir>/zveno/information/public/zveno_information</targetdir>
    </parameters>
    <build:all match="\.xml$">
      <action ref="makecontent"/>
    </build:all>
  </action>

  <action id="makecontent">
    <name>Make Content Page</name>
    <parameters>
      <targetextension>.html</targetextension>
    </parameters>
    <build:xslt stylesheet="content.xsl"
stylesheetdir="/zveno/information/scripts"/>
  </action>

</actions>

Zwik presents a Web interface to the user, where they navigate through
the asset directories.  Thus there is the notion of a "current asset",
which may be a directory or a file.  Each asset will have the
appropriate actions displayed (as buttons or links).  The first
action in the above file will be displayed for a particular asset
directory (for an example, see http://waycool.zveno.com:8020/Source.
BTW, please don't publish the actual URL since this is our production
system!  We don't have any access controls in there yet).

The content of each 'action' element specifies what happens when
the action is invoked.  This is similar to templates in XSLT.
As you will notice, these usually refer to some 'built-in' element,
such as 'build:sequence' or 'build:xslt'.  That's where the magic is.

We have to be able to implement these 'built-in' elements.  The customer
may also wish to add their own custom elements.  To do this, I have
setup
a facility where an XML Namespace is associated with a Tcl namespace.
When an element from a particular XML Namespace is used it gets
translated 
into a call to a procedure in the corresponding Tcl namespace.
There are some namespace declarations missing from the above code
fragment:

<actions
    xmlns="http://www.zveno.com/zwik/2000/12"			1
    xmlns:build="http://www.zveno.com/zwik/2000/12/build"	2
    attach-tcl="build ::build">					3

The declaration at [1] puts all of the unqualified elements in
the Zwik namespace.  This is more for convenience and future use.

The declaration at [2] sets up the XML Namespace for a Tcl package
that will define built-in elements supplied by Zwik.  The 'attach-tcl'
attribute at [3] now does the association of the XML Namespace to
the Tcl namespace.  This attribute's value is a list of 
XML-Namespace-prefix/Tcl-Namespace pairs.  Now any reference to an
element in the "http://www.zveno.com/zwik/2000/12/build" namespace,
such as 'build:xslt', will be translated to a Tcl command line,
'::build::xslt ...'.  There are some calling conventions (arguments,
return values, etc) which make the rest of it hang together.

Keep in mind that the magic of XML Namespaces means that the
'build' prefix can be changed.  It could just as easily be 'foo':

<actions
    xmlns="http://www.zveno.com/zwik/2000/12"
    xmlns:foo="http://www.zveno.com/zwik/2000/12/build"
    attach-tcl="foo ::build">


The Tcl procedure which implements the built-in element looks like this:

proc build::xslt {cfgNode src args} {
.
.
.
    return $html
}

The 'cfgNode' argument is a handle to the DOM node which is the
action element in the config file which invoked the procedure.
The procedure can walk the DOM tree to get additional parameters
from the config file.  'src' is the asset that we are performing
the action upon.  'args' are arguments gathered from the config file.
Note that some of those 'action' templates from above had a
'parameters' element.  The children of those elements become option
arguments to the procedure - the name of the element is the option
and the content is the value.  For example,

<parameters>
  <targetdir>website</targetdir>
</parameters>

becomes the arguments:

-targetdir website

These arguments are propagated by the build:sequence and build:all
actions, so nested actions can override arguments.

The only piece of the puzzle is how Zwik does the XML Namespace ->
Tcl namespace mapping.  This is a little complicated, but I'll try to
simplify.  When the system starts the package managing the action
subsystem reads the config file into a TclDOM tree and then examines
the 'attach-tcl' attribute (BTW, this system extends to other languages
very neatly.  We could add 'attach-python', 'attach-perl', 
'attach-java', etc).  Each pair of values is placed in a Tcl array for
quick lookup:

    set attachList [dom::element getAttributeNS $actions $zwikNS
attach-tcl]
    if {![llength $attachList]} {
	set attachList [dom::element getAttribute $actions attach-tcl]
    }
    if {[llength $attachList]} {
	foreach {prefix tclns} $attachList {
	    set xmlns [dom::prefix2namespaceURI $actions $prefix]
	    set mappings($xmlns) $tclns
	}
    }

Here '$zwikNS' is the Zwik XML Namespace, 
'http://www.zveno.com/zwik/2000/12'.  'prefix2namespaceURI' is
a utility which looks up the XML Namespace URI used in a document
given the XML Namespace prefix.  This is necessary because the
XML Namespace declaration can occur in an ancestor node - the DOM
package may have to go searching for it.

When an action is invoked its template must be checked to see
if it in turn invokes an action.  This code from the same package
does that:

    # Process the template.  Each element not in the Zwik NS is mapped
to
    # an evaluating language

    foreach child [::dom::node children $actionNode] {
	set elementNS [::dom::node cget $child -namespaceURI]	;# 1
	if {[string compare $elementNS $zwikNS] && [info exists
mappings($elementNS)]} {
	    # Evaluate this node

	    set cmd $mappings($elementNS)::[::dom::node cget $child -localName]
;# 2
	    set result [uplevel #0 $cmd [list $child $asset] [array get opts]]
	}
    }

    return $result

Notice how in [1] we use TclDOM's facilities to find the XML Namespace
URI of each element in the template.  At [2] we can use the 'mappings'
array built earlier to quickly look up the corresponding Tcl namespace
and form the Tcl command line to evaluate.  Note that XML Namespace
prefixes are not used at all here - the document author is free to
choose
whatever prefix they like.

That's basically it!  Phew!  There's a bit more to it than that,
but the rest is engineering detail.

> Here's a draft:
> =======================================================================
>    he founded, Invisible Worlds, funded initial development by Zveno of
>    TclXML. TclXML is the XML parser for Tcl, comparable to SAX or JAXP.

Invisible Worlds funded additional/continuing development of TclXML.
The package was already well established before Invisible Worlds
came along.  Initial funding was by the Co-operative Research Centre
for Advanced Computational Systems and the Australian National
University.
Credit where credit is due...

>    TclXML and more
>    More precisely, TclXML is a collection of "tools for processing and
>    manipulating XML documents using Tcl" (see the Resources, below, for
>    hyperlinks to more detailed information). TclXML includes two parsers:
>      * TclExpat, based on James Clark's high-performance Expact XML
>        parser; and
>      * a "native" TclXML parser, coded entirely in Tcl.

Please don't call it 'TclExpat', as it has been discontinued.

TclXML is now a framework for wrapping parser implementations,
and the pure Tcl and 'expat' parsers are the first implementations
to be wrapped.

>    Like many other WS tools, TclXML is open-sourced, governed by a
>    BSD-style traditional in the Tcl community.
> 
>    Since version 2.0theta, TclXML has supported XML Namespaces
>    conveniently abstracting away XML syntax details. Tcl has its own
>    namespace constructs, so it has been natural for Ball to attach XML
>    Namespaces to Tcl namespaces. As he explains, once this is done,
> 
>      Then any reference to an element within that namespace is
>      translated to an evaluation of a Tcl procedure with the same
>      local-name in the Tcl Namespace.

That doesn't happen automatically, as the code fragments above
demonstrate.
TclXML simply exposes the XML Namespace declarations so that it is
easy for the Tcl app to discover them, saving the hassle of interpreting
the attribute definitions itself.

Cheers,
Steve

-- 
Steve Ball            | waX Me Lyrical XML Editor | Training & Seminars
Zveno Pty Ltd         |     Web Tcl Complete      |      XML XSL
http://www.zveno.com/ |      TclXML TclDOM        | Tcl, Web Development
Ste...@zv...  +---------------------------+---------------------
Ph. +61 2 6242 4099   |   Mobile (0413) 594 462   | Fax +61 2 6242 4099