Re: [Webmacro-devel] Revised #macro stack overflow test code

[Marc P. <ma...@an...>]

On Sat, 19 Apr 2003 19:28:06 -0700, Brian Goetz <br...@qu...> wrote:

> That's probably true.  I agree 100% it could be documented better.  I 
> think that just saying "its like C #define" goes a long way towards 
> helping make it clear what's going on, at least to programmers.

...so maybe we would have to add the caveat "If you don't know what C 
#define is... considering using #include" :-)

Thing is I'm a programmer, and I still got bitten by it. Perhaps because I 
am a programmer too :)

> You make some good arguments about how subtle #macro is.  Its true that 
> if you don't understand the distinction between compile time and 
> execution time (and few folks do, because it all happens dynamically), it 
> is confusing.  I think these arguments actually supports _removing_ 
> #macro -- because its too complicated and too easy to make mistakes.  Our 
> goal was to keep the language butt-simple.

I was thinking exactly that. In fact for the "WM webapp" I was thinking of 
removing #bean and #macro from the default config (they can always be put 
back on a per-module basis) - for this very reason.

>> I really think that WM -must-have some kind of runtime-evaluated 
>> callable (and recursable) block mechanism otherwise we are really going 
>> to have huge problems with any kind of tree traversal.
>
> You're taking it as a given that WM is designed for traversing dynamic 
> structures like trees, and I'm not sure you'd get 100% agreement from 
> everyone on that.  This is a pretty big step beyond what most people are 
> doing with WM.
>
> Compare it to #foreach.  #foreach was added to allow iteration over 
> linear collections (array, vectors, etc.)  The language didn't have a 
> concept for looping or indexing (by design), so we added a facility for 
> interating over lists.
>
> Tree traversal is, conceptually, like list iteration.  Perhaps a 
> directive for "iterating" over trees would solve the problem?  You could 
> still use macros in the body, just use the directive to do the iteration. 
> I think the problem is that you're using the #macro for both expansion 
> (presentation) and iteration (structure.)

I don't think it would. I also don't think what I'm trying to do is "too 
much" to expect to be able to do in WM.

Taking any kind of simple XML - say an RSS news feed, VERY common on 
websites and handled "out of the box" by PHP distros - and rendering it in 
HTML is purely a display task.

WebMacro must be able to do it (it can, using #include), and preferably 
easily (without #include). Without this, you either have to:

(a) write some really horrible servlet code/helper that will return the XML 
rendered to HTML or to a series of "block" elements in an array that you 
render in the page. This in itself may not be too bad except that you need 
to write new code for every XML format you wish to display. Bad news.

-or-

(b) Not use WebMacro and use XSLT - or perhaps slightly better, write an 
XSLT directive... #xsltransform "http://myserver.com/mydata.xml" with 
"mytransformation.xsl"  - which would be quite elegant in a way but 
requires knowledge and usage of XSL in addition to WM, and all kinds of 
caching horribleness and slow processing.


A modified #foreach or some #tree directive is not desirable I think. Think 
about traversing a trivial XML document - you may not want to visit all 
nodes, and this could be very inefficient.

Take an RSS news feed for example, typically these can contain up to 15 
news items. You may only want to display five of them. As the item nodes 
are usually under a channel node, this would be rather awkward to implement 
with a generic tree traversal directive.

Of course you could just "ignore" the other nodes once you have had 5, but 
you still have the problem of (a) this is a very simple example, and (b) 
you would need some mechanism to tell the directive which property of each 
object in the initial list is the one containing the list of children. 
Sure, it's doable but it's quite ugly. It would look something like:

#foreach $node in $nodeList recursing with children "ChildrenList"
... and then a list of #if ($node.Type == "xxxx")
#end

..and this would probably require addition of a #continue directive to make 
coding the list of #if(s) nicer, so you can break out and continue the loop 
if the node is of no interest for whatever reason.

I don't think we could avoid the "with children 'PropName'" clause unless 
we enforced a certain node interface... which would be bad.

However, I suppose we could make #foreach detect items that have iterator() 
and elements() methods themselves - and using an option "recursing" keyword 
make it automatically present all of these elements too (recursing into 
them) as a flattened list. We would probably need options for top- 
down/left-right and left-right/top-down traversal.

Again I think this would be problematic because with XML nodes for example, 
you typically have different ways of getting children.

For example I might want to write code that iterates over the tree 
represented by $node.getChildren() or by $node.getChildrenByName( "mytag"). 
So again we are back to the "method name for child list" requirement.

Recursion-aware template inlining (AKA #include as template where script 
comes from an inline block) still looks like the most attractive option to 
me.

> You could even adapt #foreach to handle tree traversal relatively easily 
> by defining an iterator to traverse the tree in the desired order, and 
> write a context tool to take a tree and return such an interator, so you 
> could traverse it with #foreach.  Since your example simply flattens the 
> tree and visits each node, that would work, with no language changes, 
> just a new "tree flattening" context tool.

Yes, see above.

>> I really think webdesigners (not programmers) will have trouble 
>> understanding the "limitations" of #macro so we should find a way to 
>> make it abundantly clear what it cannot be used for. However, I would 
>> wager that most web designers would still think "Well why the hell is 
>> that, that's a real pain".
>
> I think it takes a pretty big leap before this would occur to a web 
> designer.  I think that most web designers would not even think to try 
> and build recursive macros to traverse tree structures.  Most web 
> designers would thing "cool, I don't have to type the same thing over and 
> over."  That's really what it was for anyway -- a simple substitution 
> mechanism, not a programming language construct.

You're probably right there. However I think to many people recursion is 
natural. If you don't really understand what it means to recurse, you don't 
see why there could be problems doing it, and it seems rather simple :)

> I wouldn't call that better -- look how many people balked at having to 
> distinguish between #include as (text|template|macro).
>
> I think that what's needed is to step back and see what is the underlying 
> problem here, if any -- and then decide if that's something that WM 
> should do.  The downside of trying to make WM all things to all people is 
> that you end up with JSP :(

...and we definitely don't need that. However the world is becoming 
increasingly XML aware, and competing technologies have this "out of the 
box". WebMacro needs it too. We can do it already in a clumsy way (#foreach 
loop in an included template) but I think we all agree we can do better.

Try finding a portal/blog site these days that does not use RSS or other 
XML formats in some way.

I will be writing the webapp online docs (served by the webapp) using a 
simple XML grammar. I thought of using the Wiki code to do render it, but 
that it looks like that would require a fair amount of Java coding (vs. 
none for the XML version, have XML helper written already) and it also 
makes the webapp dependent on a large amount of non-core code purely for 
documentation purposes.

The docs will be written like this:

<wmdoc>
     <title>Introduction to template writing</title>

     <para>
          WebMacro templates are written using WMScript. This is a simple 
language
          that uses directives, commands prefixed with
          <code>#</code>, and variables which
          are prefixed with <code>$</code>.
     </para>

     <para>
          <title>Using the most common directives</title>

          <para>
               The most commond directives used are:
               <bulletlist>
                    <listitem>
                         <directive>#foreach</directive>
                    </listitem>
                    <listitem>
                         <directive>#include</directive>
                    </listitem>
                    <listitem>
                         <directive>#set</directive>
                    </listitem>
               </bulletlist>
          </para>
     </para>
</wmdoc>

This can be rendered very simply using WebMacro script alone, especially if 
we later add some tree-supporting mechanism as discussed.

> Still, there are some good ideas that have been batted around here -- 
> definable functions, inline templates, representing hashmaps -- all of 
> which could help.

Yes the inline "map" syntax would be interesting.

I have written the #newobject directive by the way. Haven't quite got the 
"inherits $xxx" clause working, some problem with my complete lack of 
understanding of the macro build/eval process.

I'm reading the wiki docs on the pluggable directive mechanism. A couple of 
comments:

1. The HowToWriteADirective pages is blank, and should probably be obsolete 
given PluggableDirectiveMechanism exits.

2. The PDM page shows an example using PunctArg which is not listed in the 
list of primitive arg types, and I believe doesn't exist in our code.

See http://www.webmacro.org/PluggableDirectiveMechanism

Thanks


-- 
Marc Palmer (Wangjammer5)
http://www.wangjammers.org
Java Consultants