Re: [aXSL-user] Testing suites?

[Victor M. <vi...@ou...>]

Hi Tony:

You wrote: 

> The page at http://www.axsl.org/ includes:
> 
>    The following are possible areas where XSL-FO processing might
>    benefit from standards:
>    ...
>    * Testing suites.
> 
> What do you mean?
> 
> The XSL FO SG has a workable DTD for defining tests and test 
> results [1], and NIST has over 2,700 tests "covering most 
> basic features of the XSL Formatting Objects." [2]

You may be right. However, with regard to the DTD:
1. There is certainly a difference between a standardized format for a test
and a set of standardized tests. In other words, one of the potential
benefits of a standardized API is that JUnit tests can be written to that
API and shared amongst the various implementations that use that API. The
aXSL Hyphenation contains one such test class right now, and I hope we can
expand this concept in future releases. That test should perhaps be
expressed in a more general way (e.g. an XML file) instead of as a JUnit
test.
2. aXSL has 13 modules, and I think the DTD would only be useful in testing
one of them (the FO Tree module). However, I confess I am not aware of
testing schemes that use the DTD you mentioned, and perhaps I am ignorant of
its true scope.

With regard to the NIST suite, I have spent quite a bit of time trying to
figure out how to integrate it into product testing. In my experience with
an aXSL implementation and another XSL-FO implementation, it has not been
very useful, for two reasons:
1. It jumps directly to PDF output. To set up an automated testing scheme,
one would need to parse the PDF files provided as output and logically (not
literally) compare them to PDF output from the product being tested. If
those tests included an abstraction of the area tree instead of (or in
addition to) the end-result PDF, they would be EXTREMELY useful.
2. I am under the impression that the PDFs in the NIST suite were generated
by some XSL-FO implementation, but I could never find any certification that
they were correct. Did someone at NIST measure fractions of points in the
PDF itself or on the screen to ensure that all objects are positioned and
sized correctly? Are we really testing for conformity with some unnamed
XS-FO implementation by using those tests? I was never comfortable that
those tests were actually useful. This may just be a documentation issue.

On both of these issues, I may just be ill-informed, but, at any rate, that
is my current thinking.

Thanks for your inquiry. I am interested to know whether the above
explanation makes sense to you.

Victor Mote