Re: [Wheat-developer] comments on template attributes syntax

[Donovan P. <wh...@ul...>]

On Apr 6, 2005, at 9:10 AM, Mark Lentczner wrote:

> Folks -
>
> I'm looking for some comments on
> 	http://www.wheatfarm.org/wiki/TinyTemplateAttributes
> I'm leaning toward the alternate proposal (the one modeled on Nevow), 
> with both short-cuts.

I have already explained why I favor the verbose form (uniformity) and 
I like the proposed shortcuts even though they are not uniform. So this 
looks good.

> I think it is time to change the attribute naming syntax in Tiny 
> Template: I've implemented some of the ideas about testing templates 
> (see: http://www.wheatfarm.org/wiki/TinyTemplateUnit), and it isn't 
> worth the complications to make them support the current method of 
> handling attributes.
>
> By the way, I'm proceeding implementing the CardFile application and 
> using it as a driving customer story for implementing changes to the 
> system.  Right now I'm thwarted because writing test cases for 
> generated pages is such a tedious and error prone affair.  I really 
> think getting this template test unit stuff correct will make writing 
> applications with proper tests much easier both now and in the future.
>
> Also, having surveyed what's written out there about testing HTML/HTTP 
> applications, I have to say that I'm generally underwhelmed.  Everyone 
> seems to say that of course you should do it, then throws their hands 
> up and says "oh well, the best we can do is lots of string matches".  
> At best some people do lots of XPath or DOM based tests.  They all 
> acknowledge how brittle it all is, and simply shrug.  Or those suggest 
> that the final template layer be very thin and not tested.  I think we 
> have the opportunity to do something much better here.

I have implemented 4 functional testing frameworks for woven/nevow from 
the ground up:

- One which used twisted.web.client and twisted.web.microdom (for 
woven, unreleased (probably included in very old quotient))
- One which used newclient (never finished) and twisted.web.microdom 
(for nevow, unreleased (probably included in old quotient))
- One which used mechanize_ (quotient.test.webtest for quotient)
- One based on ideas from selenium_ which uses livepage and a bunch of 
javascript (nevow.livetest for nevow 0.4)

All use some sort of DOM structure to locate subsections of a page, and 
all forced you to tag nodes for test with unique ids. The API I have 
settled on as lowest common denominator required functionality is:

assert(nodeId, nodeContents)
follow(nodeId) -- follows a link
post(formId, {fieldName: fieldValue, fieldName2: fieldValue2}) -- posts 
a form

Selenium allows you to locate nodes using XPath or javascript 
expressions (document.form.field)

The first three frameworks used some sort of HTTP client library and 
parsed the resulting HTML. The selenium/livetest approach is to instead 
tell the browser to fetch a URL and perform some assertion for you. 
There are big advantages and some disadvantages to this approach, but I 
believe it is the only sane way to perform web functional testing.

Disadvantages:
- Requires a real browser present to run
- Requires writing JavaScript which may fail
- Requires writing cross-browser compatibility code

Advantages:
- Can actually assert whether the app works in multiple browsers
- Can execute JavaScript
- Can extract information from dynamically generated or mutated 
portions of a page
- An advantage of livetest but not selenium is the ability for the 
client and server to coordinate test continuation, for example the 
server can pause running tests until some event occurs in the browser

There may be another solution, writing a server-side testing framework 
which incorporates spidermonkey_ and possibly the dom-construction code 
from Mozilla (no idea where this is or if it is feasible to extract 
it?) to give the javascript-execution and dom examination capabilities 
of the client side approach without requiring a browser to run the 
tests. But this sounds like a lot of work for little benefit, and 
indeed the loss of the ability to run the same test suite in multiple 
browsers to test cross-browser compatibility.

An advantage of my current livetest module is that it is very short. I 
believe I could produce a livetest clone for wheat with very little 
work, and would be very interested in doing this. I am down in Palo 
Alto again this week, however I have a very full schedule. Perhaps I 
could take time off on Friday afternoon again to help design or 
implement whatever approach is decided upon.

This is a very difficult problem but it is one which is essential to 
solve in order to have a sane development environment.

..mechanize: http://wwwsearch.sourceforge.net/mechanize/
..selenium: http://selenium.thoughtworks.com/index.html
..spidermonkey: http://www.mozilla.org/js/spidermonkey/

Donovan