[xmlroff] xmlroff testsuite [Was: Features required for xmlroff 0.4.0?]

[Tony G. <Ton...@Su...>]

Stefan Seefeld <se...@sy...> writes:
...
> I'v been meaning to look at the test suite for quite a while now. I'm still
> not sure I understand everything. How are results validated ? Is there some
> program that checks that the generated pdf is correct or not ?

Validation is by wetware, not software.

You get to look at the output and decide if it is correct.

After the first time that you run the tests, you can put the resulting .pdf
and .png files in the testsuite/ref directory, and you will then get an
indication when any test output changes, but it's still up to you to decide
what's correct and what isn't.

Back at the XSL 1.0 CR stage, Norm Walsh had to print out and eyeball hundreds
of samples from multiple vendors' test results.  Since different vendors had
used different fonts when running each other's test suites, there was never
any guarantee that two outputs of running one test case were going to be in
any way similar.

I had to set up a new instance of the testing module recently, and even I
found it clunky and frustrating.  It does need to be improved -- and since
it's just XSLT, Perl, and Make, it could be improved by nearly anybody.  It
especially needs improvment in configuration/reuse since we're now back to
having two xmlroff backends that can produce output in multiple formats.

Regards,


Tony.