xmlroff-list — xmlroff XSL formatter discussion list

Re: [xmlroff] What changes to FoXmlDoc?

[Tony G. <Ton...@Su...>]

Stefan Seefeld <se...@sy...> writes:
> Tony Graham wrote:
...
>> The short form of my question is, is it better to:
>>  - Make separate 'constructors':
>>       fo_xml_doc_new_from_filename (...)
>>       fo_xml_doc_new_from_memory (...)
>
> Yes !

Done.

You'll also appreciate, I hope, the changes done to
libfo/fo-xslt-transformer.[ch].

Regards,


Tony.

Re: [xmlroff] Remove '--catalogs' option?

[Tony G. <Ton...@Su...>]

Tony Graham <Ton...@Su...> writes:
...
> If it really is unused, then I will remove it (unless someone else volunteers
> to spend the few minutes needed to do it and check it).

Done.

Regards,


Tony.

Re: [xmlroff] Restructure xmlroff source code?

[Tony G. <Ton...@Su...>]

Stefan Seefeld <se...@sy...> writes:
> Tony Graham wrote:
>> Oliver Kiddle made the suggestion a while ago to restructure the xmlroff
>> source code to make it easier to compile just libfo.
>
> By the way, did you consider switching to subversion ? Moving files (and

Somehow I knew you'd mention subversion.

I was considering changing to subversion after this release.

The restructuring could wait until then, too.

> in particular, directories) around is impossible in CVS without loosing
> precious information, but with subversion is as easy as a normal modification.
>
> Sourceforge is now officially supporting subversion...

Yes, I know.  And SourceForge CVS has a major outage a couple of weeks ago
that was a real inconvenience for my day job project.

Regards,


Tony.

Re: [xmlroff] Restructure xmlroff source code?

[Stefan S. <se...@sy...>]

Tony Graham wrote:
> Oliver Kiddle made the suggestion a while ago to restructure the xmlroff
> source code to make it easier to compile just libfo.

By the way, did you consider switching to subversion ? Moving files (and
in particular, directories) around is impossible in CVS without loosing
precious information, but with subversion is as easy as a normal modification.

Sourceforge is now officially supporting subversion...

Regards,
		Stefan

[xmlroff] Restructure xmlroff source code?

[Tony G. <Ton...@Su...>]

Oliver Kiddle made the suggestion a while ago to restructure the xmlroff
source code to make it easier to compile just libfo.

Being able to compile just libfo would cut very little time from compiling
libfo and xmlroff, but restructuring the source might make it easier to
understand what's what.

To get an idea of what I propose, execute this on an 'xmlroff' directory that
you're not otherwise using:

mkdir tmp
mv area datatype expr fo libfo property util tmp
mv tmp libfo
mv fo*.[ch] area-to-pdf.[ch] libfo
mkdir xmlroff
mv xmlroff.c xmlroff
mv libfo/fo-all-area.h libfo/area
mv libfo/fo-all-datatype.h libfo/datatype
mv libfo/fo-all-fo.h libfo/fo


This would require a new libfo/Makefile.am and quite a few other changes, I'm
sure, before it compiled successfully and the docs could be rebuilt, but you
get the idea.

Regards,


Tony.

[xmlroff] Remove '--catalogs' option?

[Tony G. <Ton...@Su...>]

From the use-it-or-lose-it department, is anybody using the '--catalogs'
option of xmlroff for using SGML catalogs when running xmlroff?

I used to use it a long time ago, but haven't for literally years.

If it really is unused, then I will remove it (unless someone else volunteers
to spend the few minutes needed to do it and check it).

Regards,


Tony.

Re: [xmlroff] What changes to FoXmlDoc?

[Stefan S. <se...@sy...>]

Tony Graham wrote:

> The present process for creating an FoXmlDoc from a filename is long-winded:
> 
>       xml_doc = fo_xml_doc_new ();
>       fo_xml_doc_set_filename (xml_doc,
> 			       xml_file);
> 
>       fo_xml_doc_parse (xml_doc,
> 			libfo_context,
> 			&error);

I think it would be cleaner to have a single function 'fo_xml_doc_from_filename'
that subsumes the three functions above.

>       if (error != NULL)
> 	{
> 	  g_critical ("%s:: %s",
> 		      g_quark_to_string (error->domain),
> 		      error->message);
> 	  g_error_free (error);
> 	  exit (1);
> 	}
> 
> That's partly because in the GObject world, everything has a _new() function,
> so the filename is set after the FoXmlDoc is created.

One reason why I strongly dislike GObject. Having a 'default constructor'
seems error prone, since it requires multi-stage construction. If you have
to call three functions to create the document there are three points of
potential failure, which all need to be checked, instead of just one.

> Creating an FoXmlDoc from a buffer would really be a wrapper on xmlReadMemory:
> 
>    xmlDocPtr	xmlReadMemory		(const char * buffer, 
> 					 int size, 
> 					 const char * URL, 
> 					 const char * encoding, 
> 					 int options)
> 
> Where 'options' would be extracted from settings in an FoLibfoContext and the
> rest would be set explicitly.

For symmetry, I'd add a 'fo_xml_doc_from_memory' function for this.

> The short form of my question is, is it better to:
> 
>  - Make separate 'constructors':
> 
>       fo_xml_doc_new_from_filename (...)
> 
>       fo_xml_doc_new_from_memory (...)

Yes !

>  - Make separate subtypes:
> 
>      fo_xml_doc_memory_new ()
> 
>      fo_xml_doc_file_new ()

I'm not sure I understand what 'subtypes' is referring to here. What
is the return type of these two functions ? Wouldn't it be the same ?
Why would the generated type care for whether it was created from
file content or memory ?

>    that each have their own _set() functions for their disparate parameters
> 
>  - Parse on 'construction', such that the fo_xml_doc_parse() function is no
>    longer required

Yes ! The more compact the API becomes the less it is error-prone, since
there are less check-points to watch.

Regards,
		Stefan

[xmlroff] What changes to FoXmlDoc?

[Tony G. <Ton...@Su...>]

I am looking at modifying FoXmlDoc (in libfo/fo-xml-doc.[ch]) so it's also
possible to create an FoXmlDoc from a string.

The question then becomes how to change it.

The present process for creating an FoXmlDoc from a filename is long-winded:

      xml_doc = fo_xml_doc_new ();
      fo_xml_doc_set_filename (xml_doc,
			       xml_file);

      fo_xml_doc_parse (xml_doc,
			libfo_context,
			&error);

      if (error != NULL)
	{
	  g_critical ("%s:: %s",
		      g_quark_to_string (error->domain),
		      error->message);
	  g_error_free (error);
	  exit (1);
	}

That's partly because in the GObject world, everything has a _new() function,
so the filename is set after the FoXmlDoc is created.

Creating an FoXmlDoc from a buffer would really be a wrapper on xmlReadMemory:

   xmlDocPtr	xmlReadMemory		(const char * buffer, 
					 int size, 
					 const char * URL, 
					 const char * encoding, 
					 int options)

Where 'options' would be extracted from settings in an FoLibfoContext and the
rest would be set explicitly.

The short form of my question is, is it better to:

 - Make separate 'constructors':

      fo_xml_doc_new_from_filename (...)

      fo_xml_doc_new_from_memory (...)

 - Make separate subtypes:

     fo_xml_doc_memory_new ()

     fo_xml_doc_file_new ()

   that each have their own _set() functions for their disparate parameters

 - Parse on 'construction', such that the fo_xml_doc_parse() function is no
   longer required

 - Do something completely different?

Regards,


Tony.

[xmlroff] Help wanted: simple patch task

[Tony G. <Ton...@Su...>]

In the process of fixing SourceForge bug "[ 1222524 ] indexterm-related
crash", I added some rudimentary support for fo:page-number-citation.

One part of the implementation was to add a hash of FO's Ids so they can be
looked up later.

A second part was to insert an FO's Id in the hash.  I have so far only done
this for fo:wrapper (in fo/fo-wrapper.c).

The third and most difficult part will be getting from an Id reference to an
FO to an area and then to a page number.  I have also so far only done this
for empty fo:wrapper.

In support of the second part, I have added the XSLT to the 'spec-dump' module
to generate the fo_fo_register_id() call in the right place for those FOs that
have an 'id' property.

The simple patch task is to add the two lines of the fo_fo_register_id() call
to the .c files for the other 31 FOs that don't already have the call.  This
could be done by a selective merge with the generated file in the 'spec-dump'
module or by copying and modifying the one in fo-wrapper.c.  How it's done
isn't all that important.

This is not on the critical path for xmlroff 0.4.0, but it is necessary if
fo:page-number-citation is to be implemented properly.  Is there anybody who
wants a useful but mundane first step in modifying xmlroff source code?

Is there anybody who wants to do a useful second step of implementing getting
from an Id reference to an FO to an area and then to a page number?

Regards,


Tony.

Re: [xmlroff] Syntax for controlling warnings?

[Stefan S. <se...@sy...>]

Tony Graham wrote:

>>For the actual messages please keep in mind that these may be used for
>>regression tests, so it would be good to avoid any runtime data such as
>>process-ids, dates, etc.
> 
> 
> glib puts in the process id.
> 
> See the end of the generated 'testsuite/test.sh' in the 'testing' module for a
> regex for stripping the process ids.

I'm aware of that. That's why I ask. :-)
I don't think the process id is particularly helpful here (and in most other
contexts, for that matter). What is the advantage of using glib for that,
as opposed to your own logger ?


>>In fact, if these messages have some substructure (e.g. start with the
>>line in the input at which the problem occured, it might be good to
>>formalize the message structure to allow third party tools to develop
>>parsers so xmlroff can be wrapped in a gui. (I'm thinking in particular
>>of gcc's errors / warnings, and how these can be transformed into
>>links in an emacs or eclipse console...)
> 
> 
> The input line number has been lost (I think), especially if you started with
> an XML file and an XSLT stylesheet.

Right, I forgot about that. IIRC, libxml2 has some way to store line numbers
in dom nodes, I'm not sure whether that would help to trace the context back
though.

> It's not that difficult to make xmlroff give an error message if you want to
> see one for yourself.

Ok. I'm aware that I'm talking about nice-to-have stuff here. Feel free to
ignore it if it would be too hard to implement. It's just a usability issue
that may come up later again.


Regards,
		Stefan

Re: [xmlroff] Syntax for controlling warnings?

[Tony G. <Ton...@Su...>]

Stefan Seefeld <se...@sy...> writes:
> Tony Graham wrote:
>> As threatened, I have added "warning mode" to FoLibfoContext so you can select
>> to suppress certain types of warnings.
>> xmlroff now defaults to setting "warning mode" so it suppresses "unsupported
>> property" messages (which makes redundant a large part of libfo-compat.xsl).
>
> Awesome !

I'm glad you think so.

>> You can currently set the warning mode on the xmlroff command line using '-w
>> <integer>', similarly to how you can set the debug level.
>> Magic numbers for debug level is fine (or, at least, not too bad) since if
>> you're debugging, you're going to be exposed to all sorts of gory details
>> anyway.
>> Magic numbers for warning mode seems less fine, since even casual users might
>> want to set the warning mode.
>
> I think this is fine, even for casual users, as long as the various levels
> are well documented (for example in the 'xmlroff --help' message).

Not there yet.

> That said, I think textual mode attributes are fine, too. In particular,
> if they can be accumulate, such as in 'xmlroff -w fo -w property ...'.

I thought someone would ask for that.  popt could be made to do that, but I
didn't have the patience on the weekend to rework xmlroff.c to make it do it:
right now, all xmlroff arguments are parsed by popt straight into variable
values, and repeating '-w' would require either looping over the popt context
or implementing a callback function.

> For the actual messages please keep in mind that these may be used for
> regression tests, so it would be good to avoid any runtime data such as
> process-ids, dates, etc.

glib puts in the process id.

See the end of the generated 'testsuite/test.sh' in the 'testing' module for a
regex for stripping the process ids.

> In fact, if these messages have some substructure (e.g. start with the
> line in the input at which the problem occured, it might be good to
> formalize the message structure to allow third party tools to develop
> parsers so xmlroff can be wrapped in a gui. (I'm thinking in particular
> of gcc's errors / warnings, and how these can be transformed into
> links in an emacs or eclipse console...)

The input line number has been lost (I think), especially if you started with
an XML file and an XSLT stylesheet.

It's not that difficult to make xmlroff give an error message if you want to
see one for yourself.

Regards,


Tony.

Re: [xmlroff] xmlroff source code

[Tony G. <Ton...@Su...>]

"Mauro C." <inc...@ya...> writes:
> I would to contribute to xmlroff project not only dev
> a php ext.
>
> I just start to study source, too much source :)
>
> Tony if you could give me some hint so I can
> contribute also to implements xsl-fo.
> Simly give me .c file to study fist, expecially
> fo-area and fo-fo.

How to add a new property:

1. Checkout the 'spec-dump' and 'xmlroff' modules.

2. cd to the 'spec-dump' module.

3. Add an unimplemented property to 'dump-info.xml'

4. 'make'

5. Copy the property's .c and .h files to xmlroff/property

6. For 'fo-context*.[ch]', 'property/fo-all-property.h',
   'property/fo-property-eval.h' and the .c, .h, and -private.h files for any
   FO that implements the property, merge the additions for the new property
   into the existing files in the 'xmlroff' module.

   Ignore evidence of indecision about how to handle properties with multiple
   components and of the 'spec-dump' stylesheets not munging other property
   names correctly (although improvements to stylesheets are always welcome).

6. Add the new property's .c and .h files to xmlroff/property/Makefile.am

7. Build xmlroff.

8. Work on making xmlroff actually use the property that you just added.

   It helps to add test files to 'testsuite' so you can verify that you are
   doing the right thing.  We can cover that, too, if anybody wants.

Regards,


Tony.

Re: [xmlroff] Syntax for controlling warnings?

[Stefan S. <se...@sy...>]

Tony Graham wrote:
> As threatened, I have added "warning mode" to FoLibfoContext so you can select
> to suppress certain types of warnings.
> 
> xmlroff now defaults to setting "warning mode" so it suppresses "unsupported
> property" messages (which makes redundant a large part of libfo-compat.xsl).

Awesome !

> You can currently set the warning mode on the xmlroff command line using '-w
> <integer>', similarly to how you can set the debug level.
> 
> Magic numbers for debug level is fine (or, at least, not too bad) since if
> you're debugging, you're going to be exposed to all sorts of gory details
> anyway.
> 
> Magic numbers for warning mode seems less fine, since even casual users might
> want to set the warning mode.

I think this is fine, even for casual users, as long as the various levels
are well documented (for example in the 'xmlroff --help' message).

That said, I think textual mode attributes are fine, too. In particular,
if they can be accumulate, such as in 'xmlroff -w fo -w property ...'.

For the actual messages please keep in mind that these may be used for
regression tests, so it would be good to avoid any runtime data such as
process-ids, dates, etc.
In fact, if these messages have some substructure (e.g. start with the
line in the input at which the problem occured, it might be good to
formalize the message structure to allow third party tools to develop
parsers so xmlroff can be wrapped in a gui. (I'm thinking in particular
of gcc's errors / warnings, and how these can be transformed into
links in an emacs or eclipse console...)

Regards,
		Stefan

[xmlroff] Syntax for controlling warnings?

[Tony G. <Ton...@Su...>]

As threatened, I have added "warning mode" to FoLibfoContext so you can select
to suppress certain types of warnings.

xmlroff now defaults to setting "warning mode" so it suppresses "unsupported
property" messages (which makes redundant a large part of libfo-compat.xsl).

You can currently set the warning mode on the xmlroff command line using '-w
<integer>', similarly to how you can set the debug level.

Magic numbers for debug level is fine (or, at least, not too bad) since if
you're debugging, you're going to be exposed to all sorts of gory details
anyway.

Magic numbers for warning mode seems less fine, since even casual users might
want to set the warning mode.

xmlroff currently has three separate warning mode flags: 'fo', 'property', and
'unsupported property' (athough only 'unsupported property' currently modifies
any behaviour).

How should the warning mode be specified on the xmlroff command line?

Regards,


Tony.

[xmlroff] xmlroff 0.4.0 in May, not April

[Tony G. <Ton...@su...>]

There will not be an xmlroff release on or even about 1st April.

While there are some improvements in the code in CVS, I have not started on
either incorporating libfo-compat.xsl into the xmlroff executable or building
on Windows (still haven't reformatted that laptop).  I also want to get the
libfo-examples module updated for xmlroff 0.4.0 before releasing xmlroff
0.4.0.

I can only work on xmlroff in my spare time, and I have not had enough spare
time to do everything that needs to be done to present xmlroff 0.4.0 as living
up to its description:

   xmlroff is a fast, free, high-quality, multi-platform XSL formatter that
   aims to excel at DocBook formatting and that integrates easily with other
   programs and with scripting languages.

Regards,


Tony.

[xmlroff] [ xmlroff-Bugs-1456530 ] xmlroff doesn't seem to honor break-after attribute.

[SourceForge.net <no...@so...>]

Bugs item #1456530, was opened at 2006-03-22 18:29
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=536897&aid=1456530&group_id=73148

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Stefan Seefeld (stefan)
Assigned to: Nobody/Anonymous (nobody)
Summary: xmlroff doesn't seem to honor break-after attribute.

Initial Comment:
The attached fo file gets rendered on a single page,
even though between the two versions of the title
is a 'break-after="page"' attribute.
This is xmlroff pre-0.4 CVS (as of 2006-03-22).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=536897&aid=1456530&group_id=73148

[xmlroff] #xmlroff on irc.oftc.net

[Tony G. <Ton...@Su...>]

Thanks to Stefan's gentle persuasion, I have registered the '#xmlroff' channel
on irc.oftc.net for xmlroff discussion.

Stefan and I already had some useful discussion on #xmlroff yesterday about
testing.

I will monitor the channel, but won't always be 'on'.

Regards,


Tony.

Re: [xmlroff] Segmentation fault, XML -> PS

[Tony G. <Ton...@Su...>]

James Kibblewhite <Jam...@gl...> writes:
> When you say 'run that through libfo-compat.xsl' I presume you mean that
> I produce the FO file using this command:
> `xsltproc --output simple.fo simple.xsl simple.xml`
> And then run:
> `xsltproc --output simple2.fo libfo-compat.xsl simple.fo`
>
> And then use:
> `xmlroff -o simple.ps --backend=gp simple2.fo `
>
> If so then I still have the 'Segmentation fault' error.
> If I have done this right then I will create the bog report as
> recommended.

I have not seen a bug report for this.  Did you file one?

Regards,


Tony.

Re: [xmlroff] xmlroff source code

[Tony G. <Ton...@Su...>]

Tony Graham <Ton...@Su...> writes:
> "Mauro C." <inc...@ya...> writes:
> ...
>> I would to contribute to xmlroff project not only dev
>> a php ext.
>
> That would be great.  Any help you or anyone else can give is greatly
> appreciated.
>
>> I just start to study source, too much source :)
>
> The good thing, and only sometimes the bad thing, about the source is that
> most of it is autogenerated by the spec-dump module.  The code for the FO and
> properties is generated by the spec-dump module and then copied over to
> xmlroff.
>
>> Tony if you could give me some hint so I can
>> contribute also to implements xsl-fo.
>> Simly give me .c file to study fist, expecially
>> fo-area and fo-fo.

There's a couple of things that need doing that don't require intimate
knowledge of GObjects, if you're willing: for example, there's building
libfo-compat.xsl into xmlroff.

> The simplest file for any FO is probably fo-root.c, and the simplest property
> is probably fo-property-media-usage.c.  They're also the two files that I
> tweak by hand first before modifying spec-dump to produce similar code.
>
> If you enable building the documentation with './autogen.sh --enable-gtk-doc'
> (which is prebuilt in the xmlroff distributions), you'll find some minimal
> documentation, including a discussion of fo-root.c.  There used to be more on
> the xmlroff wiki, back when the wiki worked, but I didn't keep it up because
> it didn't seem to be being used, and now it's gone because the database
> containing the wiki data was lost when SourceForge changed their database
> setup.

Thanks to the Way Back Machine (http://web.archive.org/), I was able to
resurrect some of the old Wiki pages and add their content to the docs in the
xmlroff CVS.

There's also the start of a discussion of backends and FoDoc.

After you've read the available docs, please tell me what are the biggest
holes in the documentation.

Regards,


Tony.

[xmlroff] Cairo backend: too much of a good thing

[Tony G. <Ton...@Su...>]

The code in CVS can now produce PostScript and PDF output using the Cairo
backend, but no graphics yet.

However, the text in both PostScript and PDF output is about 50% too large, so
pagination is completely different between GNOME Print and Cairo backends.

Regards,


Tony.

Re: [xmlroff] Features required for xmlroff 0.4.0?

[Tony G. <Ton...@Su...>]

Stefan Seefeld <se...@sy...> writes:
> Tony Graham wrote:
>>  - fast
>>    We're still relying on the fact that xmlroff is written in C.
>>    Recent changes to, for example, avoid using xmlDocPtr in public libfo
>>    header files may mean that xmlroff is fractionally slower than previously.
>
> I'm not worried about this at all, but I'm curious: why do you want to hide
> the libxml2 dependency ? Do you expect to replace that, or at least make it
> a true 'implementation detail' ?

It should make it easier to interface to libfo, but my main motivation is to
get away from have seemingly all of xmlroff depend on seemingly all of the
third party libraries that xmlroff uses.

The changes that I've already made to headers and to Makefiles to add
convenience libraries means that compiling xmlroff is 25% faster now than it
was for xmlroff 0.3.9.  That makes a difference to me, at least.

>>  - free
>>    Yup, still free.
>
> Good ! :-)
>
>>  - high-quality
>>    Quality hasn't gone down AFAICT, but then 'quality' is a vague term.
>>    There are at least two recent reports of segfaults that should be fixed so
>>    xmlroff better exhibits the quality of not falling over when you use it.
>
> I think for usability is is important to be as transparent as possible, i.e.

The focus of this thread is what needs to be done before we can release an
xmlroff 0.4.0.

I would hope that xmlroff is already highly transparent: the current
conformance to the spec is on the website and in the xmlroff manual, and
libfo-compat.xsl has a 'verbose' parameter that, when true(), means
libfo-compat.xsl will output a message for nearly every change it makes.

When we incorporate libfo-compat.xsl into the xmlroff executable, we also need
to continue to support the 'verbose' parameter and support an xmlroff option
that will print out the libfo-compat.xsl stylesheet so people can see for
themselves what's being modified and how.

You are welcome to propose additional features that would be required for
xmlroff 0.4.0.

> not only avoid segfaults (obviously), but also report to the user whenever
> some input was encountered that couldn't be dealt with as mandated by the specs.
> This may be in part documentation that explains how to work around the unimplemented
> feature, or just a notice "Warning: this node is skipped !".
> Again, users will probably be unfamiliar with fo, as that is only an intermediate
> step between the initial xml (docbook, say), so giving meaningful messages may be
> out of scope for xmlroff. May be some docbook people may want to collaborate,
> though, for example Norm may want to modify the docbook-xsl stylesheets to
> accomodate xmlroff better (there are already special cases for saxon et al, so
> why not xmlroff ?)
>
>>  - multi-platform
>>    This is supposed to mean "also runs on Windows".  I was hoping I could say
>>    this would be nice to have for xmlroff 0.4.0, but I think it is essential
>>    that xmlroff 0.4.0 runs on Windows (even if it requires cygwin to do it).
>>    I need to reinstall Windows on a laptop before I'll be able to start on
>>    this.
>>  - XSL formatter
>>    Unfortunately, I don't see myself being able to implement any new FOs or
>>    properties for xmlroff 0.4.0 given the number of other changes that I'm
>>    working on.
>
> Understandably !
>
>>  - that aims to excel at DocBook formatting
>>    Stefan's idea of including libfo-compat.xsl in xmlroff fits in here, since
>>    you would then not need an additional step when processing DocBook files.
>>    The man pages in DocBook markup idea would also fit in here.
>
> I think you may consider announcing xmlroff on the docbook-app list, and in particular,
> ask for help to make the combination of docbook-xsl and xmlroff as usable as possible.

I (or someone else) still need to announce xmlroff 0.3.9 on all the usual
lists.

I (or someone else) still need to create a FreshMeat entry for xmlroff.

Regards,


Tony.

Re: [xmlroff] xmlroff testsuite [Was: Features required for xmlroff 0.4.0?]

[Stefan S. <se...@sy...>]

Tony Graham wrote:
> Stefan Seefeld <se...@sy...> writes:
> ...
> 
>>I'v been meaning to look at the test suite for quite a while now. I'm still
>>not sure I understand everything. How are results validated ? Is there some
>>program that checks that the generated pdf is correct or not ?
> 
> 
> Validation is by wetware, not software.
> 
> You get to look at the output and decide if it is correct.

Oh well, that's what I was afraid of you would say.

> After the first time that you run the tests, you can put the resulting .pdf
> and .png files in the testsuite/ref directory, and you will then get an
> indication when any test output changes, but it's still up to you to decide
> what's correct and what isn't.

Ok.

> Back at the XSL 1.0 CR stage, Norm Walsh had to print out and eyeball hundreds
> of samples from multiple vendors' test results.  Since different vendors had
> used different fonts when running each other's test suites, there was never
> any guarantee that two outputs of running one test case were going to be in
> any way similar.
> 
> I had to set up a new instance of the testing module recently, and even I
> found it clunky and frustrating.  It does need to be improved -- and since
> it's just XSLT, Perl, and Make, it could be improved by nearly anybody.  It
> especially needs improvment in configuration/reuse since we're now back to
> having two xmlroff backends that can produce output in multiple formats.

For the purpose of measuring xmlroff's own state, would it be possible to
somehow fix at least some parameters that could impact the output without
impacting the pass/fail result (such as fonts etc.), and then automate the
validation ?

I'm developing QMTest (http://www.codesourcery.com/qmtest), and I'm considering
to generate a test suite for xsl-fo from the existing code. The xsl-fo processor
would just be a parameter that could be specified in a config file (e.g. 'xmlroff').
However, this would be only partly useful if the validation part would always
generate 'pass', i.e. it would be good if at least some of the validation could
be automated, too.

The way I imagine this to be done is to run the test suite once, generating
output (pdf or whatever) without validation. Then you visually validate all
tests, putting valid output into the tests suite as reference for subsequent
runs.
Once this is done subsequent test runs can validate by comparing (diffing)
actual output with the reference output.

Of course, this only works if you can control the free parameters (fonts,
for example) reasonably well so it makes sense to generate reference output.
(Such reference output could be generated per renderer backend, if the output
is too different).

What do you think about such a strategy ?

Regards,
		Stefan

[xmlroff] xmlroff testsuite [Was: Features required for xmlroff 0.4.0?]

[Tony G. <Ton...@Su...>]

Stefan Seefeld <se...@sy...> writes:
...
> I'v been meaning to look at the test suite for quite a while now. I'm still
> not sure I understand everything. How are results validated ? Is there some
> program that checks that the generated pdf is correct or not ?

Validation is by wetware, not software.

You get to look at the output and decide if it is correct.

After the first time that you run the tests, you can put the resulting .pdf
and .png files in the testsuite/ref directory, and you will then get an
indication when any test output changes, but it's still up to you to decide
what's correct and what isn't.

Back at the XSL 1.0 CR stage, Norm Walsh had to print out and eyeball hundreds
of samples from multiple vendors' test results.  Since different vendors had
used different fonts when running each other's test suites, there was never
any guarantee that two outputs of running one test case were going to be in
any way similar.

I had to set up a new instance of the testing module recently, and even I
found it clunky and frustrating.  It does need to be improved -- and since
it's just XSLT, Perl, and Make, it could be improved by nearly anybody.  It
especially needs improvment in configuration/reuse since we're now back to
having two xmlroff backends that can produce output in multiple formats.

Regards,


Tony.

Re: [xmlroff] Features required for xmlroff 0.4.0?

[Stefan S. <se...@sy...>]

Tony Graham wrote:

>  - fast
> 
>    We're still relying on the fact that xmlroff is written in C.
> 
>    Recent changes to, for example, avoid using xmlDocPtr in public libfo
>    header files may mean that xmlroff is fractionally slower than previously.

I'm not worried about this at all, but I'm curious: why do you want to hide
the libxml2 dependency ? Do you expect to replace that, or at least make it
a true 'implementation detail' ?

>  - free
> 
>    Yup, still free.

Good ! :-)

>  - high-quality
> 
>    Quality hasn't gone down AFAICT, but then 'quality' is a vague term.
> 
>    There are at least two recent reports of segfaults that should be fixed so
>    xmlroff better exhibits the quality of not falling over when you use it.

I think for usability is is important to be as transparent as possible, i.e.
not only avoid segfaults (obviously), but also report to the user whenever
some input was encountered that couldn't be dealt with as mandated by the specs.
This may be in part documentation that explains how to work around the unimplemented
feature, or just a notice "Warning: this node is skipped !".
Again, users will probably be unfamiliar with fo, as that is only an intermediate
step between the initial xml (docbook, say), so giving meaningful messages may be
out of scope for xmlroff. May be some docbook people may want to collaborate,
though, for example Norm may want to modify the docbook-xsl stylesheets to
accomodate xmlroff better (there are already special cases for saxon et al, so
why not xmlroff ?)

>  - multi-platform
> 
>    This is supposed to mean "also runs on Windows".  I was hoping I could say
>    this would be nice to have for xmlroff 0.4.0, but I think it is essential
>    that xmlroff 0.4.0 runs on Windows (even if it requires cygwin to do it).
> 
>    I need to reinstall Windows on a laptop before I'll be able to start on
>    this.
> 
>  - XSL formatter
> 
>    Unfortunately, I don't see myself being able to implement any new FOs or
>    properties for xmlroff 0.4.0 given the number of other changes that I'm
>    working on.

Understandably !

>  - that aims to excel at DocBook formatting
> 
>    Stefan's idea of including libfo-compat.xsl in xmlroff fits in here, since
>    you would then not need an additional step when processing DocBook files.
> 
>    The man pages in DocBook markup idea would also fit in here.

I think you may consider announcing xmlroff on the docbook-app list, and in particular,
ask for help to make the combination of docbook-xsl and xmlroff as usable as possible.

>  - integrates easily with other programs
> 
>    I have been working on reducing the references to third-party libraries in
>    libfo headers.
> 
>    Stefans idea of bundling PangoXSL with xmlroff fits in here since, as Mauro
>    also noted recently, it would save downloading and installing another
>    package just to use xmlroff.
> 
>  - and with scripting languages
> 
>    Mauro and I have been discussing API changes (and anyone else can chime
>    in).
> 
>    There has been no action on creating a SWIG interface.  (I did download
>    SWIG, but that hardly counts as action.)
> 
> 
> There's a bunch of other stuff that needs to be done just because this is
> 0.4.0 instead of 0.3.x, including:
> 
>  - Update the libfo-examples module to work with libfo-0.4.
> 
>    I would also like to update the GTK+ examples to use real tree widgets
>    instead of the current shortcut of creating text representations of trees.
>    Any takers?
> 
> 
> What else needs to be done for xmlroff 0.4.0?

I'v been meaning to look at the test suite for quite a while now. I'm still
not sure I understand everything. How are results validated ? Is there some
program that checks that the generated pdf is correct or not ?

Thanks,
		Stefan

[xmlroff] Features required for xmlroff 0.4.0?

[Tony G. <Ton...@Su...>]

As you know, in a moment of great optimism, I changed the xmlroff description
to:

   xmlroff is a fast, free, high-quality, multi-platform XSL formatter that
   aims to excel at DocBook formatting and that integrates easily with other
   programs and with scripting languages.

We now need to deliver on that statement, and we should do so for xmlroff
0.4.0.

Looking at each part of that statement in turn:

 - fast

   We're still relying on the fact that xmlroff is written in C.

   Recent changes to, for example, avoid using xmlDocPtr in public libfo
   header files may mean that xmlroff is fractionally slower than previously.

 - free

   Yup, still free.

 - high-quality

   Quality hasn't gone down AFAICT, but then 'quality' is a vague term.

   There are at least two recent reports of segfaults that should be fixed so
   xmlroff better exhibits the quality of not falling over when you use it.

 - multi-platform

   This is supposed to mean "also runs on Windows".  I was hoping I could say
   this would be nice to have for xmlroff 0.4.0, but I think it is essential
   that xmlroff 0.4.0 runs on Windows (even if it requires cygwin to do it).

   I need to reinstall Windows on a laptop before I'll be able to start on
   this.

 - XSL formatter

   Unfortunately, I don't see myself being able to implement any new FOs or
   properties for xmlroff 0.4.0 given the number of other changes that I'm
   working on.

 - that aims to excel at DocBook formatting

   Stefan's idea of including libfo-compat.xsl in xmlroff fits in here, since
   you would then not need an additional step when processing DocBook files.

   The man pages in DocBook markup idea would also fit in here.

 - integrates easily with other programs

   I have been working on reducing the references to third-party libraries in
   libfo headers.

   Stefans idea of bundling PangoXSL with xmlroff fits in here since, as Mauro
   also noted recently, it would save downloading and installing another
   package just to use xmlroff.

 - and with scripting languages

   Mauro and I have been discussing API changes (and anyone else can chime
   in).

   There has been no action on creating a SWIG interface.  (I did download
   SWIG, but that hardly counts as action.)


There's a bunch of other stuff that needs to be done just because this is
0.4.0 instead of 0.3.x, including:

 - Update the libfo-examples module to work with libfo-0.4.

   I would also like to update the GTK+ examples to use real tree widgets
   instead of the current shortcut of creating text representations of trees.
   Any takers?


What else needs to be done for xmlroff 0.4.0?

Regards,


Tony.