Re: [Wheat-developer] comments on template attributes syntax

[Donovan P. <wh...@ul...>]

On Apr 6, 2005, at 2:54 PM, Mark Lentczner wrote:

> Donovan:
>> All use some sort of DOM structure to locate subsections of a page, 
>> and all forced you to tag nodes for test with unique ids.
> Kragen:
>> My experience ... was that the "invisible" details of web
>> pages changed a lot more than the "visible" parts.  Divs would become
>> paragraphs; single pages would split into frames; links would become
>> form submit buttons;
>
> So, these seem at odds with each other: We test by comparing against 
> the XHTML structure, but that structure is the thing that keeps 
> changing.  For the person coding the model objects, and how those 
> objects render parts of themselves, this is awful:  Every time the 
> designer does one of those "invisible" detail changes, the tests for 
> the model object fail, even though nothing is really wrong:  The test 
> just tests the wrong level.

This is why I was advocating applying ids to the nodes you want to test 
and testing that the contents are what you expect. Any other mechanism 
for locating nodes to test depends on the structure of the DOM, which 
is bad.

For example:

<span id="firstName">Donovan</span>

Applying ids by hand is tedious, though, which is why I was suggesting 
it would be nice to have uniform automatic node id generation. (May not 
be possible, but it would be nice.)

> Now, I still believe that tests at the XHTML level are important - but 
> those are more tests of the templates themselves than how the model 
> objects render.

I see them more as a test of whether the app works at all. If you try 
to render a page or an object in a page and get an exception or get 
totally the wrong output, these tests should at least be able to catch 
that. The model objects can't render without a template, so I don't 
really see why you are making a distinction there.

> Donovan:
>> The API I have settled on as lowest common denominator required 
>> functionality is:
>> assert(nodeId, nodeContents)
>> follow(nodeId) -- follows a link
>> post(formId, {fieldName: fieldValue, fieldName2: fieldValue2}) -- 
>> posts a form
>
> I noticed this about your in-browser tests.  These very much have the 
> feel of acceptance tests, not unit tests.  Mostly because rather than 
> explore the cases of how individual objects render, it follows a 
> narrative of a user experience.

Yes. I never, ever call these unit tests. Unit tests should test that 
your units of functionality fit together as expected, which is what you 
are proposing about running a template and making sure the right 
expanders ran. Unit tests are important, but nothing which renders 
pages as a whole and makes assertions about the pages as a whole should 
be called a unit test -- you're not testing a unit, you are testing 
whether the system functions.

Thus, I call these functional tests. They test whether the system 
functions as a whole when put together, from a user interface 
perspective, rather than whether the units of code fulfill the 
contracts they expose at the boundaries of abstraction.

Acceptance tests in my mind are a big long list of things that people 
check by hand. If some of the acceptance tests can be automated, that's 
cool, but the distinction with acceptance tests for me is that the list 
is really long and exhaustive.

> Indeed, when I looked at the examples for things like XMLUnit, 
> HTMLUnit, JWebTest and Canoo Webtest, all of them have the same taste: 
>  In fact, often the examples only superficially tested that the 
> content of the page contained what was expected (testing only the 
> title or a string or two.)

Seems pretty useless to me, although more useful than nothing, which is 
why people write them, I guess.

> Now, I realize there is a fuzzy line between acceptance test and unit 
> test in this case, but the differences in terms of who writes the 
> tests and what sort of things they cover are still there.
>
> So, I think I still want to do the "template trace" style of tests so 
> that the model object author can write server side unit tests that 
> prove that:
> 	a) the object implements all the tt- names the way they are expected 
> (replacing with strings, skipping, keeping, or expanding in a loop 
> properly).
> 	b) the template and the object agree on the set of tt- names 
> (template doesn't ask for things the object doesn't expand, and the 
> template does ask for everything the object expects it to)  There is 
> no need for this kind of test to involve an end-user browser, since 
> the browser doesn't influence this level of rendering.

Absolutely. Unit tests.

> In addition, we also want to ensure the application as a whole works, 
> and this kind of test is written by the template designer and customer 
> to ensure that the links work as expected, and that the pages 
> represent the correct things.  This seems to be the kind of test that 
> I saw in nevow, and I agree that it make lots of sense to have this 
> sort of thing actually driven through/by a live browser.  I love that 
> it gives you a great way to test cross-browser compatibility of the 
> application.

Yep. Functional tests :-)

> I wonder if perhaps this splitting into two layers of tests is what's 
> missing in HTTP/HTML application frameworks.  As I said, everything I 
> found had people throwing their hands up.  Perhaps, by introducing a 
> layer here, we can have what we need in each piece, rather than trying 
> to come up with one piece that does it all.

Totally agree, and I'm sorry I wasn't clear that this was my stance 
earlier. It's been clear to me for a few years that the distinction is 
necessary, and that both kinds are necessary.

> Looking at Selenium, why did nevow need its own version?  Why couldn't 
> Wheat just use Selenium?

Selenium has some architectural flaws that I initially thought I could 
work around, but then decided were insurmountable. I may come back to 
Selenium after I have more experience in the area and attempt to 
provide patches for some of the more egregious flaws, but for my 
purposes livetest was easier to write from scratch. livetest has more 
features that I needed in terms of browser-client coordination, to 
allow the server to wait for things in the client and the tests to 
assert things about the server state.

If you want, we can try to use Selenium with Wheat but I have a hunch 
it'll actually be easier to write something specific for Wheat, which 
is why I proposed it.

Should I plan on coming down Friday? I would like to work on this, but 
I don't want to be pushy, and besides I have lots of other stuff I 
could be doing instead :-)

Donovan