webmacro-devel — Discuss development issues related to the core of WebMacro

Fwd: Re: Re: [Webmacro-devel] WM and traversing trees

[Marc P. <ma...@an...>]

FWD'd on Keats' behalf...

------- Forwarded message -------
From: ke...@su...
To: Lane Sharman <la...@op...>, ma...@an...
Subject: Re: Re: [Webmacro-devel] WM and traversing trees
Date: Mon, 21 Apr 2003 11:29:58 -0400 (EDT)

> FWIW, you can do the kind of runtime recursive traversal you are looking 
> for with the $Template tool.  (A #template directive version of this 
> might be prettier.)
>
> I like the filtering option on #foreach.  I've been toying with a 
> separate list filtering mechanism, but it might make sense to combine it 
> with the #foreach (although it's getting pretty loaded down with options) 
> .  May a #filter subdirective?
>
> Keats
>
> PS - I'm having trouble posting to the list, so if anybody want to 
> include my post in a reply that would be appreciated.

Re: [Webmacro-devel] Yep, #macro/parser BUG. **Proof**

[Marc P. <ma...@an...>]

On Mon, 21 Apr 2003 10:46:08 -0400, Eric B. Ridge <eb...@tc...> wrote:

> On Friday, April 18, 2003, at 10:09  PM, Eric B. Ridge wrote:
>
>> On Friday, April 18, 2003, at 09:19  PM, Brian Goetz wrote:
>>
>>> Do the same with your macro.  There must be some way #visitChildren can 
>>> expand to something that doesn't involve calling > #visitChildren....
>>
>> Did I miss something obvious or wouldn't that case be when $n has no 
>> children?
>
> I did miss something "obvious" (even tho I knew it already), and it's 
> that #macro's are expanded into the template at build time, not run time.
>
> So... this makes me think that making a #call directive for Macro is NOT 
> a good idea because #call doesn't accurately represent how macros work 
> and would likely lead to lots and lots of confusion along the same lines 
> as this #visitChidren example.

Definitely agreed. I missed the same thing too :)

-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] WM and traversing trees

[Marc P. <ma...@an...>]

On Mon, 21 Apr 2003 07:32:44 -0700, Lane Sharman <la...@op...> 
wrote:

>
>
> Marc Palmer wrote:
>
>>
>> Ah, a fresh new thread :-)
>>
>>
>> #foreachnode $node in $mytreeData topdown filtering ( $node.IsText && 
>> ($node.Parent.Name == "description"))
>> $node.Value
>> #end
>
>
> I actually like the aboove. If $myTreeData were to implement an interface 
> it would follow the KISS principle in WM and it would provide a lot of 
> power to the Web Head who has to deal with an tree structure.

Yes, I think so. I think however that I prefer not requiring a specific 
interface (nothing else in WM does) and using an extra clause I forgot to 
include:

#foreachnode $node in $mytreeData topdown filtering ( ... ) children in 
$node.Children

   $node.Value
#end

Thus we tell the directive what property will return a list 
(array/iter/enum) of children for the nodes. No interface required. Perhaps 
this would need to take the form of "children in 'Children'" so that the 
template writer cannot write $someUnrelatedObject.Children because that 
would result in infinite looping.

> good new thread, Marc!

Thanks :)

The beauty of this directive approach is that you can omit the filtering 
keyword and get a full flattened view of the tree - solving both problem 
scenarios in one go.

#foreachnode $loopVar in $srcData children in $propertyName [topdown | 
leftright] [filtering (expr)]

Marc
-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Yep, #macro/parser BUG. **Proof**

[Eric B. R. <eb...@tc...>]

On Friday, April 18, 2003, at 10:09  PM, Eric B. Ridge wrote:

> On Friday, April 18, 2003, at 09:19  PM, Brian Goetz wrote:
>
>> Do the same with your macro.  There must be some way #visitChildren 
>> can expand to something that doesn't involve calling > 
>> #visitChildren....
>
> Did I miss something obvious or wouldn't that case be when $n has no 
> children?

I did miss something "obvious" (even tho I knew it already), and it's 
that #macro's are expanded into the template at build time, not run 
time.

So... this makes me think that making a #call directive for Macro is 
NOT a good idea because #call doesn't accurately represent how macros 
work and would likely lead to lots and lots of confusion along the same 
lines as this #visitChidren example.

eric

Re: [Webmacro-devel] WM and traversing trees

[Lane S. <la...@op...>]

Marc Palmer wrote:

>
> Ah, a fresh new thread :-)
>
>
> #foreachnode $node in $mytreeData topdown filtering ( $node.IsText && 
> ($node.Parent.Name == "description"))
>   $node.Value
> #end 


I actually like the aboove. If $myTreeData were to implement an 
interface it would follow the KISS principle in WM and it would provide 
a lot of power to the Web Head who has to deal with an tree structure.

good new thread, Marc!

-Lane


>
>
> I suspect nobody will like that idea though. For XML-specific usage it 
> would be nicer with the expression using XPath or similar syntax:
>
> #foreachnode $node in $mytreeData topdown filtering 
> "parent::description/text()"
>   $node.Value
> #end
>
> ...or something like that anyway, but that is an XML specific solution 
> - which I am not interested in.
>
> Problem 2 can either be solved by having a helper class that can 
> return ALL nodes flattened, or by a similar tree iteration mechanism.
>
> Flattening a whole tree is often easy enough - and in fact in my 
> current XML problem scenario we have a method to do this - but it is 
> really rather wasteful of memory and processing. Yes you can cache the 
> results, but it seems a little crazy to take a perfectly good tree 
> structure and transfer it into an array with potentially thousands of 
> elements, just to display it.
>
>
> So, any other ideas on how to solve these two problems elegantly would 
> be much appreciated. You poor guys who aren't reading your email over 
> Easter... Brian and I have been mailbombin you :-(
>

[Webmacro-devel] WM and traversing trees

[Marc P. <ma...@an...>]

Ah, a fresh new thread :-)

OK from Brian's extremely helpful discussion on this issue already, I think 
we can now narrow the problem space into two sections. I would like to add 
that at this point I think it is helpful to remember that we are not just 
discussing XML usage here - we should consider any tree structure - 
although XML is the most prevalent example that springs to mind.

Problem 1. Accessing sections (branches) of a tree structure and rendering 
information from them.

Pre-requisites: Knowing the structure of the tree in advance (which types 
of nodes are children of others), and which types of nodes you want to get 
access to.

Problem 2. Displaying an entire tree, in a strict order (t-d/l-r or l-r/t-d 
depending on the data structure). Pre-requisite: You wish to visit all 
nodes in the tree.


These are in effect completely different scenarios.

Problem 1 requires a helper class with the ability to select nodes and 
return them flattened in a list, or a WM-supplied selection mechanism that 
can work on any kind of node object using introspection to work out if the 
node complies with the search criteria. The latter being far too specific 
usage for WM I think, but in a way it would be very cool - #foreachnode 
$varname [topdown | leftright] [filtering (expr)]:

#foreachnode $node in $mytreeData topdown filtering ( $node.IsText && 
($node.Parent.Name == "description"))
   $node.Value
#end

I suspect nobody will like that idea though. For XML-specific usage it 
would be nicer with the expression using XPath or similar syntax:

#foreachnode $node in $mytreeData topdown filtering 
"parent::description/text()"
   $node.Value
#end

...or something like that anyway, but that is an XML specific solution - 
which I am not interested in.

Problem 2 can either be solved by having a helper class that can return ALL 
nodes flattened, or by a similar tree iteration mechanism.

Flattening a whole tree is often easy enough - and in fact in my current 
XML problem scenario we have a method to do this - but it is really rather 
wasteful of memory and processing. Yes you can cache the results, but it 
seems a little crazy to take a perfectly good tree structure and transfer 
it into an array with potentially thousands of elements, just to display 
it.


So, any other ideas on how to solve these two problems elegantly would be 
much appreciated. You poor guys who aren't reading your email over 
Easter... Brian and I have been mailbombin you :-(

-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

On Sun, 20 Apr 2003 13:03:48 -0700, Brian Goetz <br...@qu...> wrote:

>> For what it's worth, relatively simple cases (with shallow depth and 
>> simple structure) can be implemented using #macro and without recursion 
>> (#foreach on all the child nodes you are interested in, and examine each 
>> only for the 1 or 2 deep children you are interested in).
>
> And these are probably the cases we care about anyway, aren't they?

I know what you're saying, but I think it would be foolish to limit the 
usefulness of WM for these tasks to the simple examples we can think of now 
(i.e. RSS).

What about 3 or 4 deep? It's not much more, but it increases the complexity 
of the WM code enormously - a new nested #foreach for each level, at the 
very least.

>> However the simplest way to deal with these things is usually to have a 
>> single loop "macro" and a long #if list checking tag type and 
>> calling/including the bit of WMScript to render a node of that tag type.
>
> And the long #if _can_ be a macro since it doesn't call back to the 
> looper.

Yes, but the loops can't (as far as I can see).

>> I have used the "hard coded" method so far with my RSS display code for 
>> WM, but I am using the recursion method on the wmwebapp docs because it 
>> is a less rigid structure, were <code> can appear inside a <para> or a 
>> <title> for example.
>
> How about an XPath tool?  I suspect that would go a long way towards
> making the sorts of problems you want to solve more tractable without
> building DOM support into the template language.

We are going to build XPath or an XPath-like tool into our XML helper. Our 
XML helper class presents a simple WM-friendly wrapper around nodes, 
without requiring the template writer to fully understand DOM.

Perhaps this is where some confusion is arising. I am not proposing that we 
manipulate DOM objects directly in WM script - only simplified wrappers of 
DOM Nodes. Here's the basic node wrapper API:

     boolean getIsCDATA();
     boolean getIsComment();
     boolean getIsDocumentFragment();
     boolean getIsDocumentType();
     boolean getIsElement();
     boolean getIsEntity();
     boolean getIsEntityReference();
     boolean getIsNotation();
     boolean getIsProcessingInstruction();
     boolean getIsText();

     String getName();
     XMLParentNode getParent();

     XMLNode getNextSibling();
     XMLNode getPreviousSibling();

     String getValue();
     String getText();

     XMLNode[] getChildren();
     XMLNode getFirstChild();
     XMLNode getLastChild();
     XMLNode[] getAll( String name );
     XMLNode getById( String id );
     XMLNode[] getChildren( String name );
     XMLNode getFirstChild( String name );
     String getAttribute( String name );
     String[] getAttributeNames();

These can all be used very simply and nicely from a WM template. Yes we 
cache the wrappers on a per-doc basic. These methods work very nicely in 
templates - and the XMLNode implementations implement toString and return 
getValue/getText (all child text nodes to any depth) depending on the 
current node type. i.e.

#foreach $node in $xmlHelper.Children( "item")
      <h1>$node.FirstChild.title</h1>
	<p>$node.FirstChild.description</p>
#end

We will add an XPath-like find method.. i.e. $node.find( "channel/item").

Do you know of a good free XPath lib we can pull in to use for this? I 
haven't found one yet. There's one in Apache Xalan 2 but I am pretty sure 
it depends on a lot of Xalan code so it would be serious overkill.

>> By community do you mean webmacro-devel or webmacro-users?
>
> Either!  But since this conversation is on -devel, that's the one that
> has a bigger chance of responding.

Hehe, but I meant we could post a question to the wm-users list asking "Do 
you think it would be good to be able to easily display pages containing 
data taken from XML documents?". Or "Do you think we should deliberately 
not make it easy for you to use XML data in your templates?"

:-)

>> Have you looked at Google's web services, or Amazon's web services? I 
>> will be building demo templates that retrieve XML information about a 
>> product search on Amazon and dump out the product image thumbnails, 
>> prices, descriptions etc. - this is KILLER stuff, and WM needs to be 
>> there doing it.
>
> Yes, I've even written an article about it.  Very slick.  Took me less
> than an hour to build an Amazon search web-app with Glue+IDEA+WM.
> Since it came back as Java objects, no need to even touch the XML.
> Glue rocks.

Yes I've heard good things about it - but you're taking the SOAP approach 
:) (it would seem)

It's easy (though it should be VERY easy) to do this in WM script alone, 
without any Java code apart from an XML helper to give you nodes.

>> Remember, the solution can be as simple as supporting an inline template 
>> directive - and whether "crazies" like me use it for the "right" reasons 
>> or not is not really an issue.
>
> The issue is when some "crazy" uses it for the "wrong" reason and then
> posts to the list something like "#include is broken -- proof!" :)

OK OK... but it also depends on the docs clearly showing what is/isn't 
possible. I made incorrect assumptions, unfortunately :-)

>> Inline template sections would be very useful for many things I think, 
>> although I agree it does move one step closer to WMScript becoming a 
>> little more like a "functional" language.
>
> The only reason inline templates never made it was that we didn't have
> a syntax for the substitution.  If we had inline templates without
> context substitution, it would take no more than five minutes for
> peopel to say "How about some way of calling this template with this
> parameter list..."
>
> Now that the parser has been upgraded to do argument lists, this would
> be pretty simple to write:
>
> #template t(arg-list) { block }
> #expand t(arg-list)
>
> Hnm, that was what you were asking for, right?

Something like that yes - where the net effect is EXACTLY the same as:

#set $myglobalVar1 = "something"
#set $myglobalVar2 = "something else"
#include as template "subtemplate.wmt"

...but without putting the vars into the "global" context.

>> Thanks for your discussion on this so far Brian. Have you dealt with
>> any XML / tree-based data in apps using WM yet? If so, what were
>> your solutions? If not... then perhaps you should try it. Get
>> yourself a free developer token from Amazon and start writing code
>> that puts this data into a WM template.
>
> This is a great example.  I used the Glue wsdl2java tool (which is
> nicely integrated with IDEA), and it generated Java classes for all
> the types used in the WSDL.  They are bean types, so you can justput
> an array of them right in your context, done!  Look Ma, no XML!

Ah yes, but this is using SOAP. This is completely different. You can use 
Amazon just as "XML RPC" with a straight forward request URI and get an XML 
doc back. This is far more efficient and is possible without any Java code, 
just WM script.

Glue looks very nice though for more serious apps.

Remember, I'm thinking web designers here, not java developers. The 
forthcoming webapp is a tool as much for web designers as it is for WM 
developers. As a result I want to make sure we have features and 
capabilities in there to address most if not all of the common problems 
facing web designers making complex web sites - which are not necessarily 
the high grade bespoke Java web applications people like you and I write.

> Working Amazon WS example:
>
> KeywordRequest request = new KeywordRequest();
> request.keyword = httpServletRequest.getParameter("query_string");
> request.devtag = MY_AMAZON_TOKEN;
> request.type = "heavy";
> request.page = "1";
> request.mode = "books";
> ProductInfo pi = amazon.KeywordSearchRequest(request);
> context.put("search_results", pi.Details);
>
> where Details is an array of ProductInfo objects, which are generated
> by GLUE directly from the WSDL.

Yes, nice.

Although here is something controversial. I was thinking about SOAP... 
calling a method on a helper in a template is conceptually not any 
different to calling a method on another server.

So, I think we should seriously consider writing a #soapcall directive that 
does just what you think. I'm still a SOAP noobie but I can't see why it 
shouldn't be possible.

Of course I don't mean this should be a core directive, but if we have 
#bean why not have something that is vaguely equivalent that can call 
across the net to any machine running any language? It could be a killer 
feature for people working on WM sites that must integrate with other 
platforms/languages such as Delphi.

If this is of interest I think we should start a new thread - "WM and 
SOAP/XML-RPC support".

Again, I don't see why we should force web designers to get some java code 
written by somebody (and also increase their runtime code overhead and 
deployment complexities) when we can make it possible to do this straight 
from WMScript.

Marc

-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

On Sun, 20 Apr 2003 17:20:36 -0700, Brian Goetz <br...@qu...> wrote:

>> I think it's time we found the right "tree" solution for WM.
>
> If traversing XML trees is your goal -- which I think is a reasonable
> restriction of the general goal -- I think some sort of XPath context
> tool will do it.
>
> XPath takes a query and returns a list of nodes.  The returned list is
> linear, so you can iterate over it with #foreach.

Yes.

[snip]

> This seems the right solution to me.  Leverage an existing XML-slicing
> mechanism, wrap DOM nodes with objects so they play nice with WM, and
> we're done.

Please see my new thread "WM and traversing trees"
-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

On Sun, 20 Apr 2003 12:48:26 -0700, Brian Goetz <br...@qu...> wrote:

>> Take an RSS news feed for example, typically these can contain up to 15 
>> news items. You may only want to display five of them. As the item nodes 
>> are usually under a channel node, this would be rather awkward to 
>> implement with a generic tree traversal directive.
>
> This is a good example, since it would be a common usage of any DOM
> traversal in WM and is pretty simple.
>
> Option 1: preprocess with a servlet and extract what you want.  This

As is probably abundantly clear, I do not favour options like this. This is 
because in many cases, particularly with the possible expansion of the WM 
user base to pure web-heads with no Java skills (via the webapp), people 
will need to / should be able to do stuff like this without coding Java.

> Option 2: Introduce a context tool that knows how to flatten trees into
> linear streams of SAX-like events:

Yuck-ish. A flattened tree is better than this, I think. As long as your 
node objects have a getParent() mechanism you can get an idea of your 
current node's relationship. The only problem here is that getting full 
parentage information would be difficult because we have no #break or 
#while loop.

> Option 3: Use XPath for selection.  Use an XPath select to whittle

Yes, or a similar mechanism

> Option 4: Some sort of traversal directive, which is going to end up
> looking a lot like XSL.  Combining it with XPath could be a win.

Yes - it doesn't have to be like XSL. One of the biggest problems with XML 
is its XSL template based non-linear approach. Very confusing for the 
author.

Have you seen XQuery? That is a template language for presenting XML data. 
It has a special syntax for expanding nodes by XPath query, based on the 
current node context. Of course I'm not proposing a new syntax for this, 
just showing what's out there. In XQuery you just write:

This is the title node under the current para node: $currentNode/$title

...interesingly XQuery is VERY like WM in some of its syntax! I would 
suggest an XQuery directive where you can embed XQuery script in a WM 
template, but the differences in syntax would be VERY annoying.

http://www.w3.org/TR/xquery/

Also, I am writing another message on these issues with a new thread 
subject...

-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Brian G. <br...@qu...>]

> I think it's time we found the right "tree" solution for WM.

If traversing XML trees is your goal -- which I think is a reasonable
restriction of the general goal -- I think some sort of XPath context
tool will do it.  

XPath takes a query and returns a list of nodes.  The returned list is
linear, so you can iterate over it with #foreach.

So you want to turn an RSS document into a table.  Great, grab an RSS
feed, execute the XPath query /rss/channel/item, and you've got an
iterable list of ITEM tags, each of which have description, pubdate,
guid, and link tags.

Doesn't this seem a promising way to approach the problem?  Specify a
way of slicing the XML with something like XPath, which produces a
stream of shallow elements which you can handle.  

I think what would be needed would be a DOM context tool, which would
have methods like this:

  #set $node = $DOM.parse($text)                    ## Turn text into a DOM
  #set $nodeArray = $DOM.selectNodes($node, $xPath) ## Execute an XPath query

The DOM objects returned could have a map-style get() method so you could
easily do this with no WM fuss:

#set $dom = $DOM.parse($rssDocument)
#set $itemNodes = $DOM.selectNodes($dom, "/rss/channel/item")
#foreach $node in $itemNodes { 
  $node.link $node.description $node.title
  ## Could select child nodes of $node with $DOM.selectNodes if desired
}

This seems _really_ easy, and requires no language changes.  I like
that!  I think most traversal issues that you'd want to do in a WM
template could be easily done this way.

Sure, XPath is pretty ugly, but if you just want to get a list of XYZ
nodes out of a document, its pretty easy to learn the basic cases.  Its
a lot better than trying to learn XSLT.

This seems the right solution to me.  Leverage an existing XML-slicing
mechanism, wrap DOM nodes with objects so they play nice with WM, and
we're done.

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Brian G. <br...@qu...>]

> For what it's worth, relatively simple cases (with shallow depth and simple 
> structure) can be implemented using #macro and without recursion (#foreach 
> on all the child nodes you are interested in, and examine each only for the 
> 1 or 2 deep children you are interested in).

And these are probably the cases we care about anyway, aren't they?  

> However the simplest way to deal with these things is usually to have a 
> single loop "macro" and a long #if list checking tag type and 
> calling/including the bit of WMScript to render a node of that tag type.

And the long #if _can_ be a macro since it doesn't call back to the looper.

> I have used the "hard coded" method so far with my RSS display code for WM, 
> but I am using the recursion method on the wmwebapp docs because it is a 
> less rigid structure, were <code> can appear inside a <para> or a <title> 
> for example.

How about an XPath tool?  I suspect that would go a long way towards
making the sorts of problems you want to solve more tractable without
building DOM support into the template language.

> By community do you mean webmacro-devel or webmacro-users?

Either!  But since this conversation is on -devel, that's the one that
has a bigger chance of responding.

> Have you looked at Google's web services, or Amazon's web services? I will 
> be building demo templates that retrieve XML information about a product 
> search on Amazon and dump out the product image thumbnails, prices, 
> descriptions etc. - this is KILLER stuff, and WM needs to be there doing 
> it.

Yes, I've even written an article about it.  Very slick.  Took me less
than an hour to build an Amazon search web-app with Glue+IDEA+WM.
Since it came back as Java objects, no need to even touch the XML.
Glue rocks.

> Remember, the solution can be as simple as supporting an inline template 
> directive - and whether "crazies" like me use it for the "right" reasons or 
> not is not really an issue. 

The issue is when some "crazy" uses it for the "wrong" reason and then
posts to the list something like "#include is broken -- proof!" :)

> Inline template sections would be very useful 
> for many things I think, although I agree it does move one step closer to 
> WMScript becoming a little more like a "functional" language.

The only reason inline templates never made it was that we didn't have
a syntax for the substitution.  If we had inline templates without
context substitution, it would take no more than five minutes for
peopel to say "How about some way of calling this template with this
parameter list..."

Now that the parser has been upgraded to do argument lists, this would
be pretty simple to write:

  #template t(arg-list) { block }
  #expand t(arg-list)

Hnm, that was what you were asking for, right?  

> Thanks for your discussion on this so far Brian. Have you dealt with
> any XML / tree-based data in apps using WM yet? If so, what were
> your solutions? If not... then perhaps you should try it. Get
> yourself a free developer token from Amazon and start writing code
> that puts this data into a WM template.

This is a great example.  I used the Glue wsdl2java tool (which is
nicely integrated with IDEA), and it generated Java classes for all
the types used in the WSDL.  They are bean types, so you can justput
an array of them right in your context, done!  Look Ma, no XML!

Working Amazon WS example:

        KeywordRequest request = new KeywordRequest();
        request.keyword = httpServletRequest.getParameter("query_string");
        request.devtag = MY_AMAZON_TOKEN;
        request.type = "heavy";
        request.page = "1";
        request.mode = "books";
        ProductInfo pi = amazon.KeywordSearchRequest(request);
        context.put("search_results", pi.Details);

where Details is an array of ProductInfo objects, which are generated
by GLUE directly from the WSDL.  

Re: [Webmacro-devel] Directive writing question

[Marc P. <ma...@an...>]

On Sun, 20 Apr 2003 12:34:53 -0700, Brian Goetz <br...@qu...> wrote:

>> Does anybody know the best way to enforce that an "RValue" directive arg 
>> is a Variable?
>
> Do you mean LValue?

No, RValue.

My #newobject takes:

1. an LVALUE (target)
2. optional keyword "inherits"
3. optional RValue (ancestor)

> The directive will be able to inspect the object, and I think can
> check the size of the "names" property.  I think some directives, like
> #const, do this already, so you might look there.

OK I'll have a look.

Marc

-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Brian G. <br...@qu...>]

> Take an RSS news feed for example, typically these can contain up to 15 
> news items. You may only want to display five of them. As the item nodes 
> are usually under a channel node, this would be rather awkward to implement 
> with a generic tree traversal directive.

This is a good example, since it would be a common usage of any DOM
traversal in WM and is pretty simple.

Option 1: preprocess with a servlet and extract what you want.  This
is a pretty decent option.  The servlet traverses the tree and puts an
array of RssFeedElement objects, with properties like Link,
Description, etc, in the context.  Then the template traverses it with
#foreach.  

This option only works when your document is simple, like an RSS feed,
which has a relatively flat structure.  

Option 2: Introduce a context tool that knows how to flatten trees into
linear streams of SAX-like events:
begin-x
  begin-y
    z
    z  
  end-y
end-x

Now, traverse it in WM with #foreach.  You have to maintain context by
yourself, though, and this sucks even worse in WM than it does in
ordinary languages.  

Option 3: Use XPath for selection.  Use an XPath select to whittle
down the subset of the document you want, and then apply the
techniques from Option 1 or 2.  This is nice because it eliminates
nearly all of the extraneous XML and gives you something suitable for
transforming into a linear array.
  
Option 4: Some sort of traversal directive, which is going to end up
looking a lot like XSL.  Combining it with XPath could be a win.

#domtree $dom {
  #node "content" $c {
    blah $c
  }
  #node "channel" $c { 
    blah blah $c
  }
}

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

On Sun, 20 Apr 2003 12:11:28 -0700, Brian Goetz <br...@qu...> wrote:

>> Recursion-aware template inlining (AKA #include as template where
>> script comes from an inline block) still looks like the most
>> attractive option to me.
>
> As far as I know, you're the first person to try and apply WM to this
> problem.  Not that its not a good idea -- but just don't be surprised
> that WM wasn't built to do that (yet).

Hehe, I'm not. If you look back through the pre-SF list archives you'll see 
me proferring the idea of an XML helper at least a year, maybe two years 
ago. Nobody bit on the idea then, no doubt because they hate XML or they 
could foresee the problems of navigating the tree effectively in WMScript.

> I think that dealing with tree structured data is just outside the
> limits of the current WM model, which is why its giving you so much
> grief.  You found a few back-door ways of dealing with the problem,
> but none that were designed for it.

Yep.

For what it's worth, relatively simple cases (with shallow depth and simple 
structure) can be implemented using #macro and without recursion (#foreach 
on all the child nodes you are interested in, and examine each only for the 
1 or 2 deep children you are interested in).

...or repeat the child loop in every #macro. That should probably work too 
:-)

However the simplest way to deal with these things is usually to have a 
single loop "macro" and a long #if list checking tag type and 
calling/including the bit of WMScript to render a node of that tag type.

I have used the "hard coded" method so far with my RSS display code for WM, 
but I am using the recursion method on the wmwebapp docs because it is a 
less rigid structure, were <code> can appear inside a <para> or a <title> 
for example.

> If the community thinks this is a good idea, we can explore it.  But
> my gut feeling is that these "steps forward" have a significant risk
> of introducing more problems than they solve.  I don't want to
> reinvent XSL with WM syntax.

By community do you mean webmacro-devel or webmacro-users?

I think the bottom line is, you can do this with other page building 
technologies out there, so WM will be incomplete without a reasonable 
solution for this.

The problem is that XSL is revolting. We do not need XSL to use XML as the 
data model for WM template output. XSL is extremely complicated, 
unintuitive, and slow. The idea of doing a "#include as text" that refers 
to a URL that will XSL process my XML source (i.e. Cocoon) and return it 
inline is really quite horrible (not even thinking about caching issues!).

Real people will want to do this real-life task, and increasingly so.

Have you looked at Google's web services, or Amazon's web services? I will 
be building demo templates that retrieve XML information about a product 
search on Amazon and dump out the product image thumbnails, prices, 
descriptions etc. - this is KILLER stuff, and WM needs to be there doing 
it.

I really can't see the problem.

Remember, the solution can be as simple as supporting an inline template 
directive - and whether "crazies" like me use it for the "right" reasons or 
not is not really an issue. Inline template sections would be very useful 
for many things I think, although I agree it does move one step closer to 
WMScript becoming a little more like a "functional" language.

>> You're probably right there. However I think to many people recursion is 
>> natural. If you don't really understand what it means to recurse, you 
>> don't see why there could be problems doing it, and it seems rather 
>> simple :)
>
> I think you must spend too much time with programmers.  Recursion
> confuses just about everyone the first ten times they see it.  Its
> just that programmers had that beaten out of them when then were much
> younger.

Hehe... I'm not so sure. Even HTML is, in a way, recursive. People are used 
to the idea of tree structures and hierarchies - HTML has this, so do 
directories. The notion of calling a single piece of code to "render" a 
node in a tree is not so off the wall.

I think it's time we found the right "tree" solution for WM.

I also think it's going to be hard work :)

Thanks for your discussion on this so far Brian. Have you dealt with any 
XML / tree-based data in apps using WM yet? If so, what were your 
solutions? If not... then perhaps you should try it. Get yourself a free 
developer token from Amazon and start writing code that puts this data into 
a WM template.

It's not fun.

A generic XML helper and simple tree traversal in WM is the way forward. It 
doesn't take long to work that out! Especially as the tree traversal -is- 
display logic, not processing.

Take care!
-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Directive writing question

[Brian G. <br...@qu...>]

> Does anybody know the best way to enforce that an "RValue" directive arg is 
> a Variable?

Do you mean LValue?

The directive will be able to inspect the object, and I think can
check the size of the "names" property.  I think some directives, like
#const, do this already, so you might look there.

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Brian G. <br...@qu...>]

> ...so maybe we would have to add the caveat "If you don't know what C 
> #define is... considering using #include" :-)

I'm not so sure that is true.  

> Thing is I'm a programmer, and I still got bitten by it. Perhaps
> because I am a programmer too :)

I think it is definitely _because_ you are a programmer.  You saw
something that admitted recursion and you immediately thought "cool,
tree traversal."  Designers will see #macro and say "cool, style
sheets."

> WebMacro must be able to do it (it can, using #include), and preferably 
> easily (without #include). Without this, you either have to:

You can do it with #include, but that wasn't really what #include was
designed for either.

> Recursion-aware template inlining (AKA #include as template where
> script comes from an inline block) still looks like the most
> attractive option to me.

As far as I know, you're the first person to try and apply WM to this
problem.  Not that its not a good idea -- but just don't be surprised
that WM wasn't built to do that (yet).  

I think that dealing with tree structured data is just outside the
limits of the current WM model, which is why its giving you so much
grief.  You found a few back-door ways of dealing with the problem,
but none that were designed for it.

If the community thinks this is a good idea, we can explore it.  But
my gut feeling is that these "steps forward" have a significant risk
of introducing more problems than they solve.  I don't want to
reinvent XSL with WM syntax.

> You're probably right there. However I think to many people recursion is 
> natural. If you don't really understand what it means to recurse, you don't 
> see why there could be problems doing it, and it seems rather simple :)

I think you must spend too much time with programmers.  Recursion
confuses just about everyone the first ten times they see it.  Its
just that programmers had that beaten out of them when then were much
younger.

[Webmacro-devel] Directive writing question

[Marc P. <ma...@an...>]

Hiya,

Does anybody know the best way to enforce that an "RValue" directive arg is 
a Variable?

In my (now working) #newobject directive I have the following optional 
syntax:

#newobject $newobj inherits $baseobj

(Which makes $newobj return properties from $baseobj if the property is not 
defined on $newobj).

Anyway, $baseobject has to be a compatible object type - an RValue but also 
a WebMacroObject or perhaps a java.util.Map. I can enforce this at 
evaluation time with instanceof but at build time, can how do I force that 
it is a Variable and not a literal or inline array?

As separate issue, writing this has made me realising something interesting 
- that I probably shouldn't do... I could make it "inherit" from -any- 
Variable even application-supplied custom objects. This would be a weird 
kind of "runtime inheritance" but only for properties, not method calls.

This would just require that if the property X is not defined on the 
referenced WebMacroObject, it will use introspection to read the property 
from the "ancestor" object if any. i.e. ...using #bean to demonstrate 
custom class in context...:

#bean $mybaseObj "com.mycompany.MyObject"
#set $mybaseObject.SomeProperty = "something"

#newobject $wmObj inherits $mybaseObj

The value is: $wmObj.SomeProperty

......Which would show "something" of course. Not rocket science, but it 
might be a useful facility... effectively applying a property "decorator" 
pattern to objects already in the context, such as adding name properties.

-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

On Sat, 19 Apr 2003 19:28:06 -0700, Brian Goetz <br...@qu...> wrote:

> That's probably true.  I agree 100% it could be documented better.  I 
> think that just saying "its like C #define" goes a long way towards 
> helping make it clear what's going on, at least to programmers.

...so maybe we would have to add the caveat "If you don't know what C 
#define is... considering using #include" :-)

Thing is I'm a programmer, and I still got bitten by it. Perhaps because I 
am a programmer too :)

> You make some good arguments about how subtle #macro is.  Its true that 
> if you don't understand the distinction between compile time and 
> execution time (and few folks do, because it all happens dynamically), it 
> is confusing.  I think these arguments actually supports _removing_ 
> #macro -- because its too complicated and too easy to make mistakes.  Our 
> goal was to keep the language butt-simple.

I was thinking exactly that. In fact for the "WM webapp" I was thinking of 
removing #bean and #macro from the default config (they can always be put 
back on a per-module basis) - for this very reason.

>> I really think that WM -must-have some kind of runtime-evaluated 
>> callable (and recursable) block mechanism otherwise we are really going 
>> to have huge problems with any kind of tree traversal.
>
> You're taking it as a given that WM is designed for traversing dynamic 
> structures like trees, and I'm not sure you'd get 100% agreement from 
> everyone on that.  This is a pretty big step beyond what most people are 
> doing with WM.
>
> Compare it to #foreach.  #foreach was added to allow iteration over 
> linear collections (array, vectors, etc.)  The language didn't have a 
> concept for looping or indexing (by design), so we added a facility for 
> interating over lists.
>
> Tree traversal is, conceptually, like list iteration.  Perhaps a 
> directive for "iterating" over trees would solve the problem?  You could 
> still use macros in the body, just use the directive to do the iteration. 
> I think the problem is that you're using the #macro for both expansion 
> (presentation) and iteration (structure.)

I don't think it would. I also don't think what I'm trying to do is "too 
much" to expect to be able to do in WM.

Taking any kind of simple XML - say an RSS news feed, VERY common on 
websites and handled "out of the box" by PHP distros - and rendering it in 
HTML is purely a display task.

WebMacro must be able to do it (it can, using #include), and preferably 
easily (without #include). Without this, you either have to:

(a) write some really horrible servlet code/helper that will return the XML 
rendered to HTML or to a series of "block" elements in an array that you 
render in the page. This in itself may not be too bad except that you need 
to write new code for every XML format you wish to display. Bad news.

-or-

(b) Not use WebMacro and use XSLT - or perhaps slightly better, write an 
XSLT directive... #xsltransform "http://myserver.com/mydata.xml" with 
"mytransformation.xsl"  - which would be quite elegant in a way but 
requires knowledge and usage of XSL in addition to WM, and all kinds of 
caching horribleness and slow processing.


A modified #foreach or some #tree directive is not desirable I think. Think 
about traversing a trivial XML document - you may not want to visit all 
nodes, and this could be very inefficient.

Take an RSS news feed for example, typically these can contain up to 15 
news items. You may only want to display five of them. As the item nodes 
are usually under a channel node, this would be rather awkward to implement 
with a generic tree traversal directive.

Of course you could just "ignore" the other nodes once you have had 5, but 
you still have the problem of (a) this is a very simple example, and (b) 
you would need some mechanism to tell the directive which property of each 
object in the initial list is the one containing the list of children. 
Sure, it's doable but it's quite ugly. It would look something like:

#foreach $node in $nodeList recursing with children "ChildrenList"
... and then a list of #if ($node.Type == "xxxx")
#end

..and this would probably require addition of a #continue directive to make 
coding the list of #if(s) nicer, so you can break out and continue the loop 
if the node is of no interest for whatever reason.

I don't think we could avoid the "with children 'PropName'" clause unless 
we enforced a certain node interface... which would be bad.

However, I suppose we could make #foreach detect items that have iterator() 
and elements() methods themselves - and using an option "recursing" keyword 
make it automatically present all of these elements too (recursing into 
them) as a flattened list. We would probably need options for top- 
down/left-right and left-right/top-down traversal.

Again I think this would be problematic because with XML nodes for example, 
you typically have different ways of getting children.

For example I might want to write code that iterates over the tree 
represented by $node.getChildren() or by $node.getChildrenByName( "mytag"). 
So again we are back to the "method name for child list" requirement.

Recursion-aware template inlining (AKA #include as template where script 
comes from an inline block) still looks like the most attractive option to 
me.

> You could even adapt #foreach to handle tree traversal relatively easily 
> by defining an iterator to traverse the tree in the desired order, and 
> write a context tool to take a tree and return such an interator, so you 
> could traverse it with #foreach.  Since your example simply flattens the 
> tree and visits each node, that would work, with no language changes, 
> just a new "tree flattening" context tool.

Yes, see above.

>> I really think webdesigners (not programmers) will have trouble 
>> understanding the "limitations" of #macro so we should find a way to 
>> make it abundantly clear what it cannot be used for. However, I would 
>> wager that most web designers would still think "Well why the hell is 
>> that, that's a real pain".
>
> I think it takes a pretty big leap before this would occur to a web 
> designer.  I think that most web designers would not even think to try 
> and build recursive macros to traverse tree structures.  Most web 
> designers would thing "cool, I don't have to type the same thing over and 
> over."  That's really what it was for anyway -- a simple substitution 
> mechanism, not a programming language construct.

You're probably right there. However I think to many people recursion is 
natural. If you don't really understand what it means to recurse, you don't 
see why there could be problems doing it, and it seems rather simple :)

> I wouldn't call that better -- look how many people balked at having to 
> distinguish between #include as (text|template|macro).
>
> I think that what's needed is to step back and see what is the underlying 
> problem here, if any -- and then decide if that's something that WM 
> should do.  The downside of trying to make WM all things to all people is 
> that you end up with JSP :(

...and we definitely don't need that. However the world is becoming 
increasingly XML aware, and competing technologies have this "out of the 
box". WebMacro needs it too. We can do it already in a clumsy way (#foreach 
loop in an included template) but I think we all agree we can do better.

Try finding a portal/blog site these days that does not use RSS or other 
XML formats in some way.

I will be writing the webapp online docs (served by the webapp) using a 
simple XML grammar. I thought of using the Wiki code to do render it, but 
that it looks like that would require a fair amount of Java coding (vs. 
none for the XML version, have XML helper written already) and it also 
makes the webapp dependent on a large amount of non-core code purely for 
documentation purposes.

The docs will be written like this:

<wmdoc>
     <title>Introduction to template writing</title>

     <para>
          WebMacro templates are written using WMScript. This is a simple 
language
          that uses directives, commands prefixed with
          <code>#</code>, and variables which
          are prefixed with <code>$</code>.
     </para>

     <para>
          <title>Using the most common directives</title>

          <para>
               The most commond directives used are:
               <bulletlist>
                    <listitem>
                         <directive>#foreach</directive>
                    </listitem>
                    <listitem>
                         <directive>#include</directive>
                    </listitem>
                    <listitem>
                         <directive>#set</directive>
                    </listitem>
               </bulletlist>
          </para>
     </para>
</wmdoc>

This can be rendered very simply using WebMacro script alone, especially if 
we later add some tree-supporting mechanism as discussed.

> Still, there are some good ideas that have been batted around here -- 
> definable functions, inline templates, representing hashmaps -- all of 
> which could help.

Yes the inline "map" syntax would be interesting.

I have written the #newobject directive by the way. Haven't quite got the 
"inherits $xxx" clause working, some problem with my complete lack of 
understanding of the macro build/eval process.

I'm reading the wiki docs on the pluggable directive mechanism. A couple of 
comments:

1. The HowToWriteADirective pages is blank, and should probably be obsolete 
given PluggableDirectiveMechanism exits.

2. The PDM page shows an example using PunctArg which is not listed in the 
list of primitive arg types, and I believe doesn't exist in our code.

See http://www.webmacro.org/PluggableDirectiveMechanism

Thanks


-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Brian G. <br...@qu...>]

>OK Brian, thanks for explaining this fully. I completely understand now. 
>However I am sure I will not be the last person to make this mistake - 
>it's certainly unintuitive if you don't have a deep understanding of the 
>way it is written.

That's probably true.  I agree 100% it could be documented better.  I think 
that just saying "its like C #define" goes a long way towards helping make 
it clear what's going on, at least to programmers.

You make some good arguments about how subtle #macro is.  Its true that if 
you don't understand the distinction between compile time and execution 
time (and few folks do, because it all happens dynamically), it is 
confusing.  I think these arguments actually supports _removing_ #macro -- 
because its too complicated and too easy to make mistakes.  Our goal was to 
keep the language butt-simple.

>I really think that WM -must- have some kind of runtime-evaluated callable 
>(and recursable) block mechanism otherwise we are really going to have 
>huge problems with any kind of tree traversal.

You're taking it as a given that WM is designed for traversing dynamic 
structures like trees, and I'm not sure you'd get 100% agreement from 
everyone on that.  This is a pretty big step beyond what most people are 
doing with WM.

Compare it to #foreach.  #foreach was added to allow iteration over linear 
collections (array, vectors, etc.)  The language didn't have a concept for 
looping or indexing (by design), so we added a facility for interating over 
lists.

Tree traversal is, conceptually, like list iteration.  Perhaps a directive 
for "iterating" over trees would solve the problem?  You could still use 
macros in the body, just use the directive to do the iteration.  I think 
the problem is that you're using the #macro for both expansion 
(presentation) and iteration (structure.)

You could even adapt #foreach to handle tree traversal relatively easily by 
defining an iterator to traverse the tree in the desired order, and write a 
context tool to take a tree and return such an interator, so you could 
traverse it with #foreach.  Since your example simply flattens the tree and 
visits each node, that would work, with no language changes, just a new 
"tree flattening" context tool.

>I really think webdesigners (not programmers) will have trouble 
>understanding the "limitations" of #macro so we should find a way to make 
>it abundantly clear what it cannot be used for. However, I would wager 
>that most web designers would still think "Well why the hell is that, 
>that's a real pain".

I think it takes a pretty big leap before this would occur to a web 
designer.  I think that most web designers would not even think to try and 
build recursive macros to traverse tree structures.  Most web designers 
would thing "cool, I don't have to type the same thing over and 
over."  That's really what it was for anyway -- a simple substitution 
mechanism, not a programming language construct.

>Here's an idea: Can we make #macro work out if it is being used "badly" 
>i.e. calling other #macros?

We certainly could.  But does calling other macros constitute "bad" 
usage?  I could easily see a macro for "table" which in turn uses a "row" 
macro in a #foreach.  That was part of the design intention.

Since this is a compile time error, it can be caught earlier with the 
CheckTemplates tool, for which there is an ANT task.  If you run that over 
your templates at build time, you can find out that you've got a problem 
before you deploy it.

>If so we should definitely trap this. BETTER still make it work in two 
>modes have an extra parameter #macro static MyMacro { } or #macro dynamic 
>$MyMacro { }, so that dynamic macros are evaluated at runtime instead, 
>static ones at build time?

I wouldn't call that better -- look how many people balked at having to 
distinguish between #include as (text|template|macro).

I think that what's needed is to step back and see what is the underlying 
problem here, if any -- and then decide if that's something that WM should 
do.  The downside of trying to make WM all things to all people is that you 
end up with JSP :(

Still, there are some good ideas that have been batted around here -- 
definable functions, inline templates, representing hashmaps -- all of 
which could help.



--
Brian Goetz
Quiotix Corporation
br...@qu...           Tel: 650-843-1300            Fax: 650-324-8032

http://www.quiotix.com

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

On Sat, 19 Apr 2003 11:37:05 -0700, Brian Goetz <br...@qu...> wrote:

> I'm really sorry you thought thats what we had in mind when we built
> #macro, but it wasn't.  You're right, you can't do it with #macro.
> I'm sorry you thought you could.  The WM #macro facility was designed
> to work like the C #define facility.
>
> And that's still pretty damn useful!

OK Brian, thanks for explaining this fully. I completely understand now. 
However I am sure I will not be the last person to make this mistake - it's 
certainly unintuitive if you don't have a deep understanding of the way it 
is written.

I really think that WM -must- have some kind of runtime-evaluated callable 
(and recursable) block mechanism otherwise we are really going to have huge 
problems with any kind of tree traversal.

Should we really worry about infinite loops when (a) you can do this with 
#include, and (b) Java will likely catch it anyway with StackOverflow, 
depending on how the parser/evaluation works.

The #template thing you mentioned sounds like a better option that the 
modified #include syntax, to me at least.

I really think webdesigners (not programmers) will have trouble 
understanding the "limitations" of #macro so we should find a way to make 
it abundantly clear what it cannot be used for. However, I would wager that 
most web designers would still think "Well why the hell is that, that's a 
real pain".

Here's an idea: Can we make #macro work out if it is being used "badly" 
i.e. calling other #macros? If so we should definitely trap this. BETTER 
still make it work in two modes have an extra parameter #macro static 
MyMacro { } or #macro dynamic $MyMacro { }, so that dynamic macros are 
evaluated at runtime instead, static ones at build time?

Is this possible, or best done with a separate macro? Doing it in #macro 
will show people that there is a different behaviour, and by default I 
think we should make it "dynamic" i.e. runtime evaluated. Or, we could have 
a setting for you folks who are #macro addicts - 
MacroDirective.DefaultMode: static

Perhaps we should just do #function MyFunc ($arg1, $arg2) and #functioncall 
MyFunc(1,2)

I haven't based a system around this, it's just that I want to produce the 
webapp docs from XML source because it's quick and easy, and #macro looked 
like it would fit the bill.

Right now I'm going to have to go for the very ugly #include :(

Thanks for your patience with my misunderstanding,
Marc

-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Brian G. <br...@qu...>]

> #macro countdown($n) {
>   #if ($n > 0) { $n #countdown($n-1) }
> }
> 
> #countdown(5)
> 
> This will expand, at compile time, to:
> 
>  5 4 3 2 1
> 
> Try it!  


By the way, there are some good tools for debugging WM templates which
some people don't know about.  I used StreamTemplate to verify the
above.  

Another tool is TemplateDumper, which dumps out the result of the
BUILD process.  It shows things like compile time optimization, too.

For this template:

#if ($a) { $a }

#if (1) { one }

The first #if is expanded into an IfDirective, but the second is completely
optimized away.  Here's the output:

#begin

#if :IfCondition:$property:a:IfBlock:
#begin
 $property:a
#end
 one

#end

Notice the second #if is gone!  The IfDirective is smart enough to see
that its arguments are compile-time constants.  Pretty cool!

Now look at the result of feeding the #countdown(5) example to
TemplateDumper:

#begin

 5 4 3 2 1

#end


Look ma, no #macros!  

[Webmacro-devel] Run-time #macros

[Brian G. <br...@qu...>]

Marc has raised an issue that no one else has so far regarding #macro.
He wants to write #macros to traverse DOM style trees at runtime.  You
can do this more clumsily with (lazy) #include, but that's kind of
nasty.

Justin had floated an idea at one point for a directive like #template:
where you could say

  #template {
    blah blah $a blah $b
  }

in a template, and then #include it later in the template.  It was a
crude macro facility with no substitution, or an #include facility
without using ancillary files.

Another suggestion made at one point would be to allow some sort of
substitution with #include -- like this (assuming a map-style syntax):

#include "foo.wm" using [ $a => "foo", $b => "moo" ] 

which would be like

#push $a
#push $b
#set $a = "foo"
#set $b = "moo"
#include "foo.wm"
#pop $b
#pop $a

Where #push and #pop do the obvious thing.  

Or, perhaps this is a better syntax for a temporary substitution:

#with ($a="foo", $b="bar") {
  #include "foo.wm"
}

Combining these would get the tool that Marc is looking for (lets
calls these Marcros.)  However, I think its worth having a discussion
about whether this is a good thing.  We don't even have a loop
directive because we don't want to put users in a situation where they
can create infinite loops at run time.  

Re: [Webmacro-devel] Revised #macro stack overflow test code

[Brian G. <br...@qu...>]

Marc;

I'm afraid you've made a fundamental error in assumption about what
#macro is for and how it works.  What you describe is NOT a bug -- it
illustrates its intended behavior.  

Webmacro templates processed twice -- once at parse time (this is
called building), which produces a parse tree of sorts, and once at
the time the template is expanded (run time.)  

Here's the key: MACROS ARE EXPANDED ENTIRELY AT BUILD TIME.  This is
why the ordinary #include doesn't pick up #macro definitions --
they're not in the template any more.

WM macros are almost EXACTLY like C #defines.  They are expanded
entirely at compile time.  However, many WM directives can be
optimized away at compile time if their arguments are all compile time
constants, which makes WM macros more powerful than C #defines, but
which might also confuse you into thinking that they are run-time
constructs.

WM macros can call themselves and other macros -- as long as we can reach
a base case at compile time.  Example:

#macro countdown($n) {
  #if ($n > 0) { $n #countdown($n-1) }
}

#countdown(5)

This will expand, at compile time, to:

 5 4 3 2 1

Try it!  

#macro was never designed to be a run-time template-function facility.
I'm really sorry you assumed they were, and built a whole structure
around them, but this is not a BUG in #macro -- its an attempt to use
it _way_ beyond its designed purpose.

> Here, I've tightened up the example a bit, and made it so #visitChildren 
> does not call itself directly - to deflect accusations regarding the 
> inability of a #macro to build itself given that it calls itself and hence 
> cannot hope to complete. Still it doesn't work, but I suspect still for 
> this reason, due to the way macros are built internally.

It doesn't work because #macro was not designed to do this.  

> At the very least recursion, if it cannot be supported, should be
> trapped and an exception thrown before your servlet log becomes
> 500KB in one second!

If WM had a #loop directive (btw, see the mail archives for why we
don't!), this would be like saying "At the very least, WM should
determine at compile time whether the user has coded an infinite
loop."  First of all, this is not "very least" behavior (does any
system?), and second of all, its provably impossible.

As you point out in your example, simply finding instances of
a-calls-a is not enough to find all the cases.  

Macros which call other macros, which is a feature at the very
periphery of its functionality in terms of anticipated usage, is a
dangerous feature with limited usefulness (cool examples like
#countdown aside.)  Use such features at your own risk.  Its not
inherently a bug for macros to call each other or themselves -- but
you must use it very carefully.

> However, I can't believe that we can't support macro
> recursion. Without it, any kind of DOM/XML based processing in WM
> templates is pretty much impossible / extremely ugly (using
> recursive #includes).

I'm really sorry you thought thats what we had in mind when we built
#macro, but it wasn't.  You're right, you can't do it with #macro.
I'm sorry you thought you could.  The WM #macro facility was designed
to work like the C #define facility.

And that's still pretty damn useful!  

Re: [Webmacro-devel] Yep, #macro/parser BUG. **Proof**

[Brian G. <br...@qu...>]

> Yes, exactly. I don't think I'm being stupid. This is standard node-tree 
> traversal logic. No children, no further recursion.
> 
> It's a bug I tell you!

#macros are expanded at build time.  There has to be some BUILD TIME
case where it stops expanding.  Just like in C.  

[Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

Here, I've tightened up the example a bit, and made it so #visitChildren 
does not call itself directly - to deflect accusations regarding the 
inability of a #macro to build itself given that it calls itself and hence 
cannot hope to complete. Still it doesn't work, but I suspect still for 
this reason, due to the way macros are built internally.

We definitely need this fixed IMO. At the very least recursion, if it 
cannot be supported, should be trapped and an exception thrown before your 
servlet log becomes 500KB in one second!

However, I can't believe that we can't support macro recursion. Without it, 
any kind of DOM/XML based processing in WM templates is pretty much 
impossible / extremely ugly (using recursive #includes).

## ********************************************************
## This is all just to set up a trivial tree structure
## ********************************************************
 
#bean $rootNode = "java.util.Hashtable"
#set $rootNode.Text="root"
 
#bean $wmdocNode = "java.util.Hashtable"
#set $wmdocNode.Text="wmdoc"
 
#bean $titleNode1 = "java.util.Hashtable"
#set $titleNode1.Text="title1"
 
#bean $paraNode1 = "java.util.Hashtable"
#set $paraNode1.Text="para1"
 
#bean $titleNode2 = "java.util.Hashtable"
#set $titleNode2.Text="title2"
#bean $paraNode2 = "java.util.Hashtable"
#set $paraNode2.Text="para2"
 
#bean $paraNode3 = "java.util.Hashtable"
#set $paraNode3.Text="para3"
 
#set $rootNode.Children = [ $wmdocNode]
#set $paraNode1.Children = [$titleNode2, $paraNode2]
#set $wmdocNode.Children = [$titleNode1, $paraNode1, $paraNode3]
 
## ********************************************************
## Here is the code
## ********************************************************
 
#macro visitChildren( $node)

  #if ($node.Children) 
       #foreach $n in $node.Children
          $n.Text<br>
          #doIt( $n)
       #end
  #end
 
#end
 
## Added this so visitChildren does not -directly-call itself

#macro doIt( $node)
  #visitChildren($node)
#end

## ********************************************************
## Call the code
## ********************************************************
 
#visitChildren( $rootNode)


-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants