From: Orion P. <or...@co...> - 2013-08-08 16:01:58
|
Okay, next issue :) ------------------------------------------------------------------------ r12284 | airwin | 2013-01-07 22:26:34 -0700 (Mon, 07 Jan 2013) | 13 lines Insert \$ENV{DESTDIR} appropriately (with the "$" escaped because you want the environment variable DESTDIR to be read at install time rather than cmake time) for all install(CODE ... signatures usage. This change should let "make DESTDIR=<whatever> install" work correctly at install time whenever the doxygen documentation and the docbook documentation are being installed. Only the doxygen install change has been tested, but the corresponding change to the docbook install should also work since the same CMake rules should apply. ------------------------------------------------------------------------ This ends up adding an extra DESTDIR in the path since cmake automatically handles adding DESTDIR to the paths. This is with -DBUILD_DOC:BOOL=ON. I have not tested with PREBUILT_DOC. Reverting this commit fixes the install for me (unsurprising since 12281 was working fine for me before). -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane or...@nw... Boulder, CO 80301 http://www.nwra.com |
From: Alan W. I. <ir...@be...> - 2013-08-09 15:24:29
|
On 2013-08-08 10:01-0600 Orion Poplawski wrote: > Okay, next issue :) > > ------------------------------------------------------------------------ > r12284 | airwin | 2013-01-07 22:26:34 -0700 (Mon, 07 Jan 2013) | 13 lines > > Insert \$ENV{DESTDIR} appropriately (with the "$" escaped because you > want the environment variable DESTDIR to be read at install time > rather than cmake time) for all install(CODE ... signatures usage. > > This change should let "make DESTDIR=<whatever> install" work > correctly at install time whenever the doxygen documentation and the > docbook documentation are being installed. > > Only the doxygen install change has been tested, but the > corresponding change to the docbook install should also work > since the same CMake rules should apply. > ------------------------------------------------------------------------ > > This ends up adding an extra DESTDIR in the path since cmake automatically > handles adding DESTDIR to the paths. This is with -DBUILD_DOC:BOOL=ON. I > have not tested with PREBUILT_DOC. > > Reverting this commit fixes the install for me (unsurprising since 12281 was > working fine for me before). >From the above commit message I explicitly tested that this worked for the doxygen case. Is that case also fine for you, i.e., is the docbook cases the only ones where DESTDIR is applied twice? I also notice (now) that the docbook case uses the file(INSTALL...) signature inside the install(CODE...) command. Normally, install(CODE...) needs explicit DESTDIR (which my above commit supplied), but one interpretation of the documentation would imply the case where file(INSTALL...) is used inside install(CODE...) is an exception to that rule which would mean DESTDIR would be applied twice but only for the docbook cases and not for the doxgyen case. Anyhow, I will test both the doxygen install and docbook install cases and figure out what to do. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Orion P. <or...@co...> - 2013-08-09 15:55:57
|
On 08/09/2013 09:24 AM, Alan W. Irwin wrote: > On 2013-08-08 10:01-0600 Orion Poplawski wrote: > >> Okay, next issue :) >> >> ------------------------------------------------------------------------ >> r12284 | airwin | 2013-01-07 22:26:34 -0700 (Mon, 07 Jan 2013) | 13 lines >> >> Insert \$ENV{DESTDIR} appropriately (with the "$" escaped because you >> want the environment variable DESTDIR to be read at install time >> rather than cmake time) for all install(CODE ... signatures usage. >> >> This change should let "make DESTDIR=<whatever> install" work >> correctly at install time whenever the doxygen documentation and the >> docbook documentation are being installed. >> >> Only the doxygen install change has been tested, but the >> corresponding change to the docbook install should also work >> since the same CMake rules should apply. >> ------------------------------------------------------------------------ >> >> This ends up adding an extra DESTDIR in the path since cmake automatically >> handles adding DESTDIR to the paths. This is with -DBUILD_DOC:BOOL=ON. I >> have not tested with PREBUILT_DOC. >> >> Reverting this commit fixes the install for me (unsurprising since 12281 was >> working fine for me before). > >> From the above commit message I explicitly tested that this worked > for the doxygen case. Is that case also fine for you, i.e., is > the docbook cases the only ones where DESTDIR is applied twice? > > I also notice (now) that the docbook case uses the file(INSTALL...) > signature inside the install(CODE...) command. Normally, > install(CODE...) needs explicit DESTDIR (which my above commit > supplied), but one interpretation of the documentation would imply the > case where file(INSTALL...) is used inside install(CODE...) is an > exception to that rule which would mean DESTDIR would be applied twice > but only for the docbook cases and not for the doxgyen case. > > Anyhow, I will test both the doxygen install and docbook install > cases and figure out what to do. Specifically, the plplotdoc.info files ended up with the duplicated DESTDIR. Not sure about anything else I'm afraid. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane or...@nw... Boulder, CO 80301 http://www.nwra.com |
From: Alan W. I. <ir...@be...> - 2013-08-10 00:04:46
|
On 2013-08-09 09:55-0600 Orion Poplawski wrote: > On 08/09/2013 09:24 AM, Alan W. Irwin wrote: >> Anyhow, I will test both the doxygen install and docbook install >> cases and figure out what to do. > > Specifically, the plplotdoc.info files ended up with the duplicated DESTDIR. > Not sure about anything else I'm afraid. To Orion and Andrew: Please try revision 12479 with cmake options -DBUILD_DOX_DOC=ON -DBUILD_DOC=ON (which should build both the doxygen and docbook documentation). That case works without errors for me when I run "make DESTDIR=<whatever> install" on Debian wheezy. I would also advise using -DBUILD_DOX_DOC=ON when generating Fedora rpm's or Debian packages. Of course, both our doxygen-generated and docbook-generated documentation needs work (see my recent comments about some of the bit rot that is showing up for the doxygen documentation, and Lord knows it is long past time to move our docbook generation completely over to docbook XML tools rather than partially relying on historical docbook SGML tools which are no longer maintained). Nevertheless, despite these known issues, the results we now have for both doxygen and docbook are well worth distributing. Therefore, since both of you already distribute the docbook results, I think you will probably both want to distribute the doxygen results as well. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Orion P. <or...@co...> - 2013-08-12 16:58:01
|
On 08/09/2013 06:04 PM, Alan W. Irwin wrote: > On 2013-08-09 09:55-0600 Orion Poplawski wrote: > >> On 08/09/2013 09:24 AM, Alan W. Irwin wrote: >>> Anyhow, I will test both the doxygen install and docbook install >>> cases and figure out what to do. >> >> Specifically, the plplotdoc.info files ended up with the duplicated DESTDIR. >> Not sure about anything else I'm afraid. > > To Orion and Andrew: > > Please try revision 12479 with cmake options -DBUILD_DOX_DOC=ON > -DBUILD_DOC=ON (which should build both the doxygen and docbook > documentation). That case works without errors for me when I run > "make DESTDIR=<whatever> install" on Debian wheezy. > > I would also advise using -DBUILD_DOX_DOC=ON when generating Fedora > rpm's or Debian packages. > > Of course, both our doxygen-generated and docbook-generated > documentation needs work (see my recent comments about some of the bit > rot that is showing up for the doxygen documentation, and Lord knows > it is long past time to move our docbook generation completely over to > docbook XML tools rather than partially relying on historical docbook > SGML tools which are no longer maintained). Nevertheless, despite > these known issues, the results we now have for both doxygen and > docbook are well worth distributing. Therefore, since both of you > already distribute the docbook results, I think you will probably both > want to distribute the doxygen results as well. - 12479 installs fine. - The doxygen documentation is huge ~88MB - I don't think the docbook documentation builds anymore on Fedora: -- DSSSL Style Sheet DTD found -- DocBook HTML Stylesheet found -- DocBook Print Stylesheet found -- WARNING: DocBook DTD not found -- WARNING: Not building print documentation - dtd files / style sheets are missing -- WARNING: Not building html documentation - dtd files / style sheets are missing jadeout.log.x: /usr/bin/openjade:/usr/share/sgml/docbook/xml-dtd-4.5/ent/isogrk4.ent:42:30:E: "1D6C2" is not a character number in the document character set /usr/bin/openjade:/usr/share/sgml/docbook/xml-dtd-4.5/ent/isogrk4.ent:43:30:E: "1D6C3" is not a character number in the document character set /usr/bin/openjade:/usr/share/sgml/docbook/xml-dtd-4.5/ent/isogrk4.ent:44:30:E: "1D6D8" is not a character number in the document character set ..... This may be a Fedora issue... -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane or...@nw... Boulder, CO 80301 http://www.nwra.com |
From: Orion P. <or...@co...> - 2013-08-12 16:59:58
|
On 08/12/2013 10:57 AM, Orion Poplawski wrote: > - I don't think the docbook documentation builds anymore on Fedora: > > -- DSSSL Style Sheet DTD found > -- DocBook HTML Stylesheet found > -- DocBook Print Stylesheet found > -- WARNING: DocBook DTD not found > -- WARNING: Not building print documentation - dtd files / style sheets are > missing > -- WARNING: Not building html documentation - dtd files / style sheets are > missing > > jadeout.log.x: > /usr/bin/openjade:/usr/share/sgml/docbook/xml-dtd-4.5/ent/isogrk4.ent:42:30:E: > "1D6C2" is not a character number in the document character set > /usr/bin/openjade:/usr/share/sgml/docbook/xml-dtd-4.5/ent/isogrk4.ent:43:30:E: > "1D6C3" is not a character number in the document character set > /usr/bin/openjade:/usr/share/sgml/docbook/xml-dtd-4.5/ent/isogrk4.ent:44:30:E: > "1D6D8" is not a character number in the document character set > ..... > > This may be a Fedora issue... > > > https://bugzilla.redhat.com/show_bug.cgi?id=703472 seems to point to the fact of sgml being dead and needing to use xmlto. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane or...@nw... Boulder, CO 80301 http://www.nwra.com |
From: Alan W. I. <ir...@be...> - 2013-08-12 19:22:41
|
On 2013-08-12 10:59-0600 Orion Poplawski wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=703472 seems to point to the fact > of sgml being dead and needing to use xmlto. Hi Orion: Thanks for mentioning xmlto which I was unaware of. It appears to be quite promising as a modern XML tool for converting our DocBook XML source to a large number of documentation formats (it advertises itself as XML to anything). Our build system (for -DBUILD_DOC=ON) currently does that task using a variety of SGML DocBook backend tools which are all extremely dated and therefore not maintained. Since it appears you are beginning to run into errors for those old SGML tools, I have decided to try xmlto (starting later today) to see whether we can use it instead of those SGML tools. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-14 00:27:44
|
On 2013-08-12 12:22-0700 Alan W. Irwin wrote: > [Because SGML is dead] I have decided to try xmlto (starting later today) to see whether we can > use it instead of those SGML [DocBook backend] tools. To Andrew and Orion: Sorry this is long, but I have achieved quite a few results since I wrote the above just yesterday so this e-mail is packed with goodness. :-) I discovered a documentation issue with the SGML backend tools which is that Table 3.4 (which is designed to show how #g<normal character> maps to Greek characters) currently gives gibberish (see http://plplot.sourceforge.net/docbook-manual/plplot-html-5.9.9/characters.html#greek). This problem obviously occurred for our last release (done by Hazen with whatever Debian release he was using at that time) and also occurs now with my Debian wheezy platform. (This is some sort of regression in the SGML html backend since we did not have this problem for older releases.) This regression and also the errors that Orion is experiencing with generating our DocBook-based documentation for a cutting-edge Linux distribution are all symptoms of the lack of maintenance for the SGML backend tools for many years now. Therefore, I think it is long past time to move from the SGML backend tools to XML backend tools to put generation of documentation from our DocBook source back on solid footing again. I changed the subject line of this e-mail to something appropriate to that project, and the following concerns my initial results with that project. 1. xmlto initially bombed because it uses xmllint for validation, and that validator showed there were issues with our DocBook source code. I fixed those issues (as of revision 12482) so this is an immediate benefit of looking into xmlto. Specifically, xmllint --noout --nonet --xinclude --postvalid --noent plplotdoc.xml works perfectly for revision 12482. However, I am not going to replace our current validator onsgmls (which is less careful than xmllint since it did not detect the need for changes and validated our DocBook source code both several revisions before 12482 and also for that revision) because when there are validation errors I find that xmllint is not robust, i.e., it tends to segfault. 2. HTML results generated from our DocBook source. There are promising html results from xmlto but with some caveats. a. Extremely promising.... After revision 12482 and after running make validate in the build tree to take care of dependencies, the command xmlto -o html-dir html plplotdoc.xml succeeded and also rendered table 3.4 without issues (a big improvement on results generated with the html SGML backend tool). b. Caveats.... Just before that table 3.4 on the webpage the overline-underline example ends up empty. Furthermore, the colour-coded API examples in the API chapter are now a much more bland looking format with no colour coding, and the filenames for the html bits and pieces are arbitrarily numerical (e.g., html-dir/ch19s133.html) rather than the logical names you get from the SGML backend (e.g., plplot-html-5.9.9/plssym.html which refers to the same area of the documentation as that numerical file name generated by the above xmlto command). All these html style issues are currently controlled for the SGML HTML backend by a configurable DSSSL stylesheet (plplotdoc-html.dsl.in in doc/docbook/src) supplemented by the CSS stylesheet, stylesheet.css in that same directory. Norman Walsh's on-line "DocBook: The Definitive Guide" <http://docbook.org/tdg/en/html/docbook.html> (TDG, copyright 2003, last updated in 2006) covers DSSSL stylesheets in some detail in Chapter 4, but remarks (a) few tools honour DSSSL and (b) DSSSL stylesheets are actually SGML documents (which means the paucity of open-source tools that can deal with SGML makes life difficult for the DSSSL approach). Furthermore, from that book it was clear that XSL stylesheets were rapidly gaining acceptance as an alternative to DSSSL (probably because there are so many XML tools out there in the open-source world) and Bob Stayton wrote a chapter in that book concerning XSL which has since expanded into its own independent book, DocBook XSL: The Complete Guide 4th edition (TCG, Copyright 2007) <http://www.sagehill.net/docbookxsl/>. Furthermore interest in DSSSL has waned since TDG was written ~10 years ago. Norman Walsh has published <http://docbook.org/tdg51/en/html/docbook.html> (copyright 2013) which is the DocBook 5.1 variant of TDG. Chapter 4 in that latest variant doesn't even mention DSSSL as a publishing tool (i.e., a backend language)! Also, I am pretty sure that the tools actually invoked by the xmlto script don't understand DSSSL stylesheets. Thus, my conclusion is that our current DSSSL style sheets (and probably the CSS stylesheet, stylesheet.css as well) must be replaced by XSL styling sheets following the methods that are documented in TCG. And until we do that the style of our xmlto results is going to be quite bland. 3. Print (PDF) results generated from our DocBook source. Here too, there are promising results but also some caveats. a. Extremely promising.... After revision 12482 and after running make validate in the build tree to take care of dependencies, the command xmlto --with-fop pdf plplotdoc.xml succeeded and also rendered table 3.4 without issues. (For example, the small number of missing glyphs in the SGML PDF backend results are present here.) b. Caveats.... xmlto pdf plplotdoc.xml errors out (see https://bugzilla.redhat.com/show_bug.cgi?id=949087 where there doesn't seem to be any quick solution for this default pdf issue) and xmlto --with-dblatex pdf plplotdoc.xml succeeds but does not give good Table 3.4 results. So avoid these variants of the xmlto command for pdf (which is easy to do, but I thought I had better remark on it here). Another much more important caveat is all the style issues that occurred for html with xmlto also occur for pdf. So the same remarks about moving to XSL style sheets apply here as well. 4. Print (PostScript) results generated from our DocBook source. Here too, there are promising results but also some caveats. a. Extremely promising.... After revision 12482 and after running make validate in the build tree to take care of dependencies, the command xmlto --with-fop ps plplotdoc.xml succeeded and also rendered table 3.4 without issues (as does the PostScript SGML backend). b. Caveats.... Both xmlto ps plplotdoc.xml and xmlto --with-dblatex ps plplotdoc.xml error out so avoid these variants for ps (which is easy to do, but I thought I had better remark on it here). Another much more important caveat is all the style issues that occurred for html and pdf also occur for ps. So the same remarks about moving to XSL style sheets apply here as well. 5. Print (dvi) results generated from our DocBook source. Here there are slightly promising results but also some strong caveats. a. Slightly promising.... After revision 12482 and after running make validate in the build tree to take care of dependencies, the command xmlto --with-dblatex dvi plplotdoc.xml succeeded without obvious error messages if and only if I locally replaced the Greek entities in math.ent by their equivalent Math symbol unicode values, e.g., unicode x391 changed to unicode x1D6A8. (Note, that probably anything that was unrecognizable would have worked for these entities.) b. Caveats.... Both xmlto dvi plplotdoc.xml and xmlto --with-fop dvi plplotdoc.xml error out (with or without the changed math.ent) so avoid these variants for dvi (which is easy to do, but I thought I had better remark on it here). Another caveat is the dvi result produced by the one (--with-dblatex) variant of xmlto that works for dvi above is the entities (mostly Math symbols for the Greek letters) defined by the locally replaced math.ent were all meaningless to the tools invoked by --with-dblatex (probably because of the large numerical unicode index for the Math symbol variants of the Greek letters). So the resulting Table-3.4 results printed out the entities verbatim e.g., 𝚨 rather than the Greek letter, capital alpha). (This issue also occurred for --with-dblatex for pdf which is why that variant should be avoided in the pdf case, see above.) The SGML backend dvi results do not have this issue. I presume there is some sort of bug in dblatex concerning propagating entities to dvi that is avoided if you use large unrecognizable (at least for this XML dvi backend) unicode indices. So this is a pretty crummy dvi result which depends on internal details to avoid other bugs in the XML dvi backend. An additional less serious caveat is all the style issues that occurred for html also occur for dvi. So the same remarks about moving to XSL style sheets apply here as well. Other remarks: I have not looked yet at man and info results with xmlto, but apparently they are possible (which would complete the backend set of tools that we need). Also, according to documentation available on the web, xml is almost completely (with just a few necessary exceptions) utf8 aware so I have tried the experiment of inserting the utf8 code for a gamma (e.g., "γ" if your mailer is utf8 aware) right into math.xml and xmlto --with-fop pdf plplotdoc.xml filled out the appropriate bit of Table-3.4 in the resulting plplotdoc.pdf with no issues. So this constitutes a proof-of-concept that numerical entities such as "γ" (or the equivalent decimal equivalent "γ") that define the "γ" entity in math.ent could be replaced by the utf8 code for gamma, "γ" and so on for all the other Greek letters. In sum, xmlto is looking pretty good right now (except for problematic dvi results and DSSSL stylesheet replacements by XSL stylesheets that would have to be made in the future) so assuming I can get man and info to work with the xmlto approach, I would likely deprecate the SGML backend tools. So by default (unless -DDEPRECATED_SGML_BACKEND=ON was specified) those building our documentation would get the xmlto backend results. That would give us a chance to work on XSL stylesheets (using the TCG reference above) to improve the style of our xmlto backend results to the equivalent or better than the style of our current SGML backend results. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-14 17:03:25
|
On 2013-08-13 17:27-0700 Alan W. Irwin wrote: > Also, according to documentation available on the web, xml is almost > completely (with just a few necessary exceptions) utf8 aware so I have tried the > experiment of inserting the utf8 code for a gamma (e.g., "γ" if your > mailer is utf8 aware) right into math.xml and > > xmlto --with-fop pdf plplotdoc.xml > > filled out the appropriate bit of Table-3.4 in the resulting > plplotdoc.pdf with no issues. So this constitutes a proof-of-concept > that numerical entities such as "γ" (or the equivalent decimal > equivalent "γ") that define the "γ" entity in math.ent > could be replaced by the utf8 code for gamma, "γ" and so on for > all the other Greek letters. Replacing all numerical entities in math.ent with their utf8 equivalents made no difference to the xmlto approach. pdf and ps worked fine, and dvi errored out just as before for --with-dblatex, --with-fop, or with neither. (I presume dvi would have worked (lamely) if I had used Math symbols since those are out of the range which is recognized and therefore avoid the entity issues that seem to bedevil dvi right now.) xmllint validation continued to work. This change did make a major difference to the SGML approach; onsgmls blew its little mind trying to figure out utf8. The complaints concerned invalid SGML characters. The other drawback of the utf8 approach is it is hard to visually distinguish between say the Greek alpha and the equivalent math symbol (which have very different unicode indices). But the math symbol turns into a missing glyph for PDF and PostScript (remember the severe limitation in the range of glyphs that are available for Type 1 fonts) while the Greek letter does not. So it is best from the point of view of being as definite as possible to continue using the numerical entities in math.ent, but nevertheless it was an interesting experiment. My next step is to look at generating our info documentation using the xmlto approach. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-14 21:35:22
|
On 2013-08-14 10:03-0700 Alan W. Irwin wrote: > My next step is to look at generating our info documentation using the > xmlto approach. That remark was due to two misconceptions on my part. In fact, xmlto has no ability to generate info format, but it doesn't matter because our present docbook2x method of generating the documentation in info format is based on XSLT style sheets, i.e., it is a modern XML/XSLT method that does not need replacing. However, as a result of reviewing how we generate the info form I noticed a utf8 encoding option for one of the docbook2x commands that we use. So I tried that and it worked! The result is Table 3-4 now (revision 12483) contains good-looking Greek characters (rather than written out alpha, beta, etc.) in the info form of our documentation which is a nice improvement. So here is where this project stands now: The man and info parts of our documentation generation are fine as they are since they both use modern XML/XSLT rather than dead/old-fashioned SGML/DSSSL. The xmlto approach (which uses XML/XSLT) gives good results for the HTML, PDF, and PostScript parts of our documentation generation. However, the results use a default style which is quite bland looking. The xmlto script also claims to generate DVI, but there is a bug in that at the moment (at least for version 0.0.25 of xmlto that is installed on Debian wheezy). My next step is to deprecate the SGML backend tools to generate HTML, PDF, PostScript, and DVI in our build system. Instead of those I will introduce as default into our build system the xmlto alternatives for producing all of those backend formats (with DVI as an option that is turned off by default until the upstream bug in xmlto concerning dvi generation can be fixed). The resulting HTML, PDF, PostScript, and (eventually) DVI results will have an extremly bland style. That style is probably okay for the near term, but eventually we should learn to update that style to something much nicer using standard and well-maintained XSLT capabilities that are described in TCG <http://www.sagehill.net/docbookxsl/>. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-20 03:24:19
|
On 2013-08-14 14:35-0700 Alan W. Irwin wrote: > My next step is to deprecate the SGML backend tools to generate HTML, > PDF, PostScript, and DVI in our build system. Instead of those I will > introduce as default into our build system the xmlto alternatives for > producing all of those backend formats (with DVI as an option that is > turned off by default until the upstream bug in xmlto concerning dvi > generation can be fixed). The resulting HTML, PDF, PostScript, and > (eventually) DVI results will have an extremly bland style. To Andrew and Orion: I have updated (as of revision 12490) the old documentation build logic to make it much easier to understand for both the DOCBOOK_XML_BACKEND=OFF case (which now gets you the old SGML/DSSSL backend results) and the newly implemented default DOCBOOK_XML_BACKEND=ON case (as described above). Please give DOCBOOK_XML_BACKEND=OFF and the default DOCBOOK_XML_BACKEND=ON cases a try (using the -DBUILD_DOC=ON cmake option and the install target in doc/docbook/src) and let me know what you think of the results. Both those options currently work for me. DOCBOOK_XML_BACKEND=OFF should pretty much give you what was available before. For the default DOCBOOK_XML_BACKEND=ON case, pay attention to the messages from xmlto _at run time_. It will warn you if you need some additional packages installed such as fop. Unless there is something you guys dislike about these results that can be fixed in the short term, I am pretty much finished with it although there are some obvious issues mentioned in the commit message for revision 12490 that will need to be addressed in the long term. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-20 17:41:48
|
Hi Andrew: On 2013-08-19 20:24-0700 Alan W. Irwin wrote: > Unless there is something you guys dislike about these results that > can be fixed in the short term, I am pretty much finished with it > although there are some obvious issues mentioned in the commit message > for revision 12490 that will need to be addressed in the long term. Well, I am always curious about new stuff so I did take a quick look at http://www.sagehill.net/docbookxsl to see what was possible for customizing the style of the results. And at least the first step in that process (generate meaningful names for the html chunk filenames) turned out (revision 12491) to be trivial. So based on that extremely encouraging result you may see some additional revisions in the next day or so from me as I look at some other customization possibilities. But I plan to test all such revisions before I commit them so please do not wait for customization perfection from me to evaluate what I have done. It's also important that more than one of us understands the customizations so I encourage you to get involved in XSL customizaton yourself (at least by looking up the detailed web references I give in comments included in the new plplotdoc-html.xsl.in file which controls html customization for the default -DDOCBOOK_XML_BACKEND=ON case.) Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-21 23:25:02
|
On 2013-08-20 10:41-0700 Alan W. Irwin wrote: > Hi Andrew: > > On 2013-08-19 20:24-0700 Alan W. Irwin wrote: > >> Unless there is something you guys dislike about these results that >> can be fixed in the short term, I am pretty much finished with it >> although there are some obvious issues mentioned in the commit message >> for revision 12490 that will need to be addressed in the long term. > > Well, I am always curious about new stuff so I did take a quick look > at http://www.sagehill.net/docbookxsl to see what was possible for > customizing the style of the results. And at least the first step in > that process (generate meaningful names for the html chunk filenames) > turned out (revision 12491) to be trivial. > > So based on that extremely encouraging result you may see some > additional revisions in the next day or so from me as I look at some > other customization possibilities. But I plan to test all such > revisions before I commit them so please do not wait for customization > perfection from me to evaluate what I have done. It's also important > that more than one of us understands the customizations so I encourage > you to get involved in XSL customizaton yourself (at least by looking > up the detailed web references I give in comments included in the new > plplotdoc-html.xsl.in file which controls html customization for the > default -DDOCBOOK_XML_BACKEND=ON case.) With revision 12494, I think the style for -DDOCBOOK_XML_BACKEND=ON is now looking pretty good both for the html and print (pdf and ps) results. The only issue I am aware of is the ability to represent "S̅(f̲r̲e̲q̲)" in the html and print results. For the html case, the old DSSSL method converted the special xml fragment <!ENTITY over-under '<anchor id="over-under"/>'> in inline-html.ent to the html fragment <span class="overline">S</span>(<span class="underline">freq</span>) and css rules in stylesheet.css took care of the rest. For the print case, the old method converted the special xml fragment <!ENTITY over-under "Ё"> in inline-print.ent using jadetex (an old SGML backend tool) configuration contained in jadetex.cfg. It is possible something similar to these special techniques to render overlining and underlining for the html and print cases could also be used for the new XML/XSL backend. (In fact, I am fairly close to a special solution for the html case, see comments in plplotdoc-html.xsl.in.) But I think this is a rather low priority since overlining and underlining actually works badly or not at all for modern PLplot devices, see my other post today). So I am going to put off working further on this until the long term (if at all). In any case, in the long term the general utf-8 solution <!ENTITY over-under "S̅(f̲r̲e̲q̲)"> should just work instead. But PDF generation on Linux currently seems to be limited to just type 1 fonts so those unicode glyphs are not recognized as indicated by the warning messages generated by "xmlto --with-fop" Glyph "̅" (0x305, overlinecmb) not available in font "Times-Roman". Glyph "̲" (0x332, lowlinecmb) not available in font "Times-Roman". The generated PDF also had a "#" sign in place of each of the missing overlinecmb (just after the "S") and lowlinecmb (just after each of the letters in "freq") glyphs. So this is not a good solution in the short term. In the next day or so I plan to play a bit more with dblatex to see if I can generate dvi results with it, and I am also ready to deal with any issues you find as well, but otherwise I believe I have completed this project. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-24 23:50:56
|
On 2013-08-21 16:24-0700 Alan W. Irwin wrote: > In the next day or so I plan to play a bit more with dblatex to see if > I can generate dvi results with it, and I am also ready to deal with > any issues you find as well, but otherwise I believe I have completed > this project. Hi Andrew: It turned out the source of the dvi issue was a dblatex bug (see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=720624 for a description of the problem and a patch to fix it). Once I applied that patch, the dblatex command finally started working properly to generate dvi results both when dblatex was invoked directly or indirectly via "xmlto --with-dblatex". Please give the updated documentation build a spin following the (newly updated as of revision 12497) directions in doc/docbook/README.developers. Also, please feel free to update that file if you feel more details about the packages required by xmlto are needed. Revision 12497 is the last commit I plan to do in the near term concerning the DocBook backend tools that we use unless you find some easily fixed issue in the documentation build. In the medium term (i.e., after the next release when we will be using these new backend tools to help generate our website) I plan two more DocBook backend changes. The first of those is to disable use of the deprecated SGML/DSSSL backend tools completely which will allow me to do the second change which is to use UTF-8 directly in our DocBook source. The big advantage of UTF-8 is it allows us to include a potentially enormous range of glyphs in our DocBook source including all of the mathematical and human-language glyphs that appear in our examples. Part of that range allows overlining and underlining so UTF-8 provides a fundamental solution for representing the UTF-8 string "S̅(f̲r̲e̲q̲)" in our documentation (which is the last outstanding issue of the present approach as far as I know). (I hope your mailer is UTF-8 aware so you can read that UTF-8 string as intended!) The only disadvantage (which I consider to be minor) of UTF-8 is that the dvi format is fundamentally incompatible with it outside the very narrow range of glyphs currently represented in our documentation. The inescapable conclusion is that the dvi format will have to be dropped from our documentation once we extend the range of glyphs in our documentation in the slightest. (In fact, UTF-8 glyphs for overlining and underling are already outside the valid glyph range of the dvi format.) For further comments about how UTF-8 can be implemented for PDF, PostScript, and HTML and the excellent results I have already had with a proof of concept for that implementation please see the updated discussion in doc/docbook/README.developers. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-08-31 06:44:05
|
On 2013-08-24 16:50-0700 Alan W. Irwin wrote: > On 2013-08-21 16:24-0700 Alan W. Irwin wrote: > >> In the next day or so I plan to play a bit more with dblatex to see if >> I can generate dvi results with it, and I am also ready to deal with >> any issues you find as well, but otherwise I believe I have completed >> this project. > > Hi Andrew: > > It turned out the source of the dvi issue was a dblatex bug (see > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=720624 for a > description of the problem and a patch to fix it). Once I applied > that patch, the dblatex command finally started working properly to > generate dvi results both when dblatex was invoked directly > or indirectly via "xmlto --with-dblatex". > > Please give the updated documentation build a spin following the > (newly updated as of revision 12497) directions in > doc/docbook/README.developers. Also, please feel free to update that > file if you feel more details about the packages required by xmlto are > needed. > > Revision 12497 is the last commit I plan to do in the near term > concerning the DocBook backend tools that we use unless you find some > easily fixed issue in the documentation build. Hi Andrew: One additional piece of news for this thread is that Orion kindly ran a number of tests, and as a result I made some additional fixups (as of revision 12501) of the Docbook documentation build. One of those fixups was to empty DESTDIR before each call to xmlto. That works around a bug in xmlto or one of its dependencies (see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=721366) for the case when DESTDIR is set for the overall build. As a result of these fixups, I get good results for the documentation build and install both with and without DESTDIR set for the overall build. Furthermore, Orion now reports complete success with a build and install (using DESTDIR) of our DocBook documentation on Fedora. (His test was without dvi which will probably be removed, in any case, right after the upcoming release when I hope to move to the pure UTF-8 approach I described in a previous post in this thread.) So I think I am done with refining the documentation build process for this release cycle, but any additional testing by you of the new documentation build on platforms accessible to you would be appreciated so we don't get any last-minute documentation build surprises just before the release. My next step for this release cycle is to finish up the pllegend/plcolorbar doxygen and DocBook documentation as discussed in a different thread. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Andrew R. <and...@us...> - 2013-09-06 09:09:39
|
On Friday 30 Aug 2013 23:43:55 Alan W. Irwin wrote: > On 2013-08-24 16:50-0700 Alan W. Irwin wrote: > > On 2013-08-21 16:24-0700 Alan W. Irwin wrote: > >> In the next day or so I plan to play a bit more with dblatex to see if > >> I can generate dvi results with it, and I am also ready to deal with > >> any issues you find as well, but otherwise I believe I have completed > >> this project. > > > > Hi Andrew: > > > > It turned out the source of the dvi issue was a dblatex bug (see > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=720624 for a > > description of the problem and a patch to fix it). Once I applied > > that patch, the dblatex command finally started working properly to > > generate dvi results both when dblatex was invoked directly > > or indirectly via "xmlto --with-dblatex". > > > > Please give the updated documentation build a spin following the > > (newly updated as of revision 12497) directions in > > doc/docbook/README.developers. Also, please feel free to update that > > file if you feel more details about the packages required by xmlto are > > needed. > > > > Revision 12497 is the last commit I plan to do in the near term > > concerning the DocBook backend tools that we use unless you find some > > easily fixed issue in the documentation build. > > Hi Andrew: > > One additional piece of news for this thread is that Orion kindly ran > a number of tests, and as a result I made some additional fixups (as > of revision 12501) of the Docbook documentation build. One of those > fixups was to empty DESTDIR before each call to xmlto. That works > around a bug in xmlto or one of its dependencies (see > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=721366) for the case > when DESTDIR is set for the overall build. As a result of these > fixups, I get good results for the documentation build and install > both with and without DESTDIR set for the overall build. Furthermore, > Orion now reports complete success with a build and install (using > DESTDIR) of our DocBook documentation on Fedora. (His test was > without dvi which will probably be removed, in any case, right after > the upcoming release when I hope to move to the pure UTF-8 approach I > described in a previous post in this thread.) > > So I think I am done with refining the documentation build process for > this release cycle, but any additional testing by you of the new > documentation build on platforms accessible to you would be > appreciated so we don't get any last-minute documentation build > surprises just before the release. > Alan, Thanks for this. After a fair bit of playing around I've got this working on by Ubuntu system. Since this wasn't entirely trivial and involved installing a number of new packages I think it is probably worth documenting to help others. When I initially tried, the doc build was mostly disabled until I installed xmlto (not surprising!). Having done that the build failed on the dvi with some undefined latex sequences (e.g. \Epsilon, \Tau,\ texttheta). I didn't probe too far into this, but disabled the dvi build. Is this the same problem you encountered? The build then failed on the pdf docs with an error about passivetex being missing. A quick package search suggested I should install xmltex (which I did). This got me further, but still failed with a warning about fop not being found. It seems xmlto then used some alternative which didn't work. I installed fop and I can now build all the documentation except for the dvi. I still got a warning about "unable to locate servlet-api in /usr/share/java". Installing libservlet2.5-java fixes this, but seems to make no discernable difference to the results. I'm not too worried about dvi to be honest. I don't see we need this AND pdf built by default, though it would be nice to fix it. The conclusion is that the new pdf output looks good (no problems I've noticed with special characters including greek). The tool chain looks like it is better supported going forward. Thanks for this work! Some comments: The cmake support is not too robust at the moment as my testing shows. We probably need to test for e.g. fop if that is required so that it fails gracefully (and with a helpful error message!) Fop seems to be required at the moment. This is quite a hefty requirement as it pulls in java and quite a number of added packages. This is probably ok since we're not building the documentation all the time. Regards Andrew |
From: Alan W. I. <ir...@be...> - 2013-09-06 16:57:23
|
On 2013-09-06 10:09+0100 Andrew Ross wrote: > On Friday 30 Aug 2013 23:43:55 Alan W. Irwin wrote: >> On 2013-08-24 16:50-0700 Alan W. Irwin wrote: >>> On 2013-08-21 16:24-0700 Alan W. Irwin wrote: >>>> In the next day or so I plan to play a bit more with dblatex to see if >>>> I can generate dvi results with it, and I am also ready to deal with >>>> any issues you find as well, but otherwise I believe I have completed >>>> this project. >>> >>> Hi Andrew: >>> >>> It turned out the source of the dvi issue was a dblatex bug (see >>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=720624 for a >>> description of the problem and a patch to fix it). Once I applied >>> that patch, the dblatex command finally started working properly to >>> generate dvi results both when dblatex was invoked directly >>> or indirectly via "xmlto --with-dblatex". >>> >>> Please give the updated documentation build a spin following the >>> (newly updated as of revision 12497) directions in >>> doc/docbook/README.developers. Also, please feel free to update that >>> file if you feel more details about the packages required by xmlto are >>> needed. >>> >>> Revision 12497 is the last commit I plan to do in the near term >>> concerning the DocBook backend tools that we use unless you find some >>> easily fixed issue in the documentation build. >> >> Hi Andrew: >> >> One additional piece of news for this thread is that Orion kindly ran >> a number of tests, and as a result I made some additional fixups (as >> of revision 12501) of the Docbook documentation build. One of those >> fixups was to empty DESTDIR before each call to xmlto. That works >> around a bug in xmlto or one of its dependencies (see >> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=721366) for the case >> when DESTDIR is set for the overall build. As a result of these >> fixups, I get good results for the documentation build and install >> both with and without DESTDIR set for the overall build. Furthermore, >> Orion now reports complete success with a build and install (using >> DESTDIR) of our DocBook documentation on Fedora. (His test was >> without dvi which will probably be removed, in any case, right after >> the upcoming release when I hope to move to the pure UTF-8 approach I >> described in a previous post in this thread.) >> >> So I think I am done with refining the documentation build process for >> this release cycle, but any additional testing by you of the new >> documentation build on platforms accessible to you would be >> appreciated so we don't get any last-minute documentation build >> surprises just before the release. >> > > Alan, > > Thanks for this. After a fair bit of playing around I've got this working on > by Ubuntu system. Since this wasn't entirely trivial and involved installing a > number of new packages I think it is probably worth documenting to help > others. Currently this is covered in doc/docbook/README.developer by saying essentially pay attention to the run-time errors from xmlto. I don't think we should bother with anything more than that since a further large change (the pure UTF-8 approach see below) is planned right after the release. > > When I initially tried, the doc build was mostly disabled until I installed > xmlto (not surprising!). Having done that the build failed on the dvi with > some undefined latex sequences (e.g. \Epsilon, \Tau,\ texttheta). I didn't > probe too far into this, but disabled the dvi build. Is this the same problem > you encountered? Yes. The dblatex patch should fix this issue, but I would not bother with it, see below. > > The build then failed on the pdf docs with an error about passivetex being > missing. A quick package search suggested I should install xmltex (which I > did). This got me further, but still failed with a warning about fop not being > found. It seems xmlto then used some alternative which didn't work. I > installed fop and I can now build all the documentation except for the dvi. > > I still got a warning about "unable to locate servlet-api in /usr/share/java". > Installing libservlet2.5-java fixes this, but seems to make no discernable > difference to the results. > > I'm not too worried about dvi to be honest. I don't see we need this AND pdf > built by default, though it would be nice to fix it. The dblatex patch I mentioned fixed it for me, but I think the proper thing to do is simply to abandon DVI now since DVI is badly supported (presumably because few are interested in that format any more since it is not UTF-8 aware) and the next change I have in mind (right after the release) to move to a pure UTF-8 approach will force us to drop DVI in any case. So unless I hear differently from you in the next few days I plan to go ahead and completely disable DVI production, and drop it from our website for the current release. > > The conclusion is that the new pdf output looks good (no problems I've noticed > with special characters including greek). The tool chain looks like it is > better supported going forward. Thanks for this work! Thanks for that evaluation, and I am glad the new backend tools are giving good results on your platform. > > Some comments: > > The cmake support is not too robust at the moment as my testing shows. We > probably need to test for e.g. fop if that is required so that it fails > gracefully (and with a helpful error message!) > > Fop seems to be required at the moment. This is quite a hefty requirement as > it pulls in java and quite a number of added packages. This is probably ok > since we're not building the documentation all the time. The next change I have planned (moving to a pure UTF-8 approach just after the release) is going to be pretty big. xmlto will be used just for html results. dblatex with xetex backend will be used to generated PDF and PostScript. DVI will be dropped. There should be no more need for passivetex (which is deprecated in any case since it is a package with many bugs and no upstream development for many years) or fop. Those dependencies will get replaced by the xetex dependencies which is most of texlive, but presumably that was a dependency in any case. I suggest for now we should stick with the current non-robust cmake support until we make this further planned change to a pure UTF-8 approach. I am very much looking forward to that further change since it will give us access to a huge range of mathematical and human-language glyphs in our DocBook source documentation. It also finally makes our documentation build rely on UTF-8 clean backend tools which I am pretty confident (because there is a lot of interest in UTF-8) will be well supported for many years to come. I am actually tempted to move ahead with the pure UTF-8 approach now, but that is clearly too much change just before a release so I will stick with the plan of doing this change just after the release which will give us a full release cycle to evaluate it. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-09-06 17:38:22
|
On 2013-09-06 10:18+0100 Andrew Ross wrote: > One further comment - in the old version plplpot functions were highlighted in > red which was nice. Now they are in typewriter font. > I quite liked the red > font, but there are advantages to being black and white and I don't feel too > strongly. Maybe others have a preference? Hi Andrew: I also like that "red font" style for cross references in the PDF and PostScript documentation. However, let's put off dealing with this style issue until after the transition to the pure UTF-8 approach is made just after the current release. If the default style for xetex results produced by dblatex produces cross references that are not visually obvious (e.g., not something like a red font), then we will have to learn how to style such results to make that happen. However, such style changes should be straightforward since the whole point of a pure XML/XSL approach is to use XSL stylesheets to derive the exact style that you want in the DocBook backend results that are generated. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-10-05 23:55:28
|
On 2013-09-06 09:57-0700 Alan W. Irwin wrote: > On 2013-09-06 10:09+0100 Andrew Ross wrote: >> Some comments: >> >> The cmake support is not too robust at the moment as my testing shows. We >> probably need to test for e.g. fop if that is required so that it fails >> gracefully (and with a helpful error message!) >> >> Fop seems to be required at the moment. This is quite a hefty requirement as >> it pulls in java and quite a number of added packages. This is probably ok >> since we're not building the documentation all the time. > > The next change I have planned (moving to a pure UTF-8 approach just > after the release) is going to be pretty big. xmlto will be used just > for html results. dblatex with xetex backend will be used to generated > PDF and PostScript. DVI will be dropped. There should be no more need > for passivetex (which is deprecated in any case since it is a package > with many bugs and no upstream development for many years) or fop. > Those dependencies will get replaced by the xetex dependencies which > is most of texlive, but presumably that was a dependency in any case. > I suggest for now we should stick with the current non-robust cmake > support until we make this further planned change to a pure UTF-8 > approach. > > I am very much looking forward to that further change since it will > give us access to a huge range of mathematical and human-language > glyphs in our DocBook source documentation. It also finally makes our > documentation build rely on UTF-8 clean backend tools which I am > pretty confident (because there is a lot of interest in UTF-8) will be > well supported for many years to come. I am actually tempted to move > ahead with the pure UTF-8 approach now, but that is clearly too much > change just before a release so I will stick with the plan of doing > this change just after the release which will give us a full release > cycle to evaluate it. As of revision 12582, I have implemented this change which I hope is the last fundamental change in how we generate our documentation for years to come. I have also substantially updated doc/docbook/README.developers and disabled the SGML/DSSL backend tools we used for 5.9.9 and previous releases and which were deprecated for 5.9.10. The summary is the results for UTF-8 documentation are much improved compared to previously, and we are probably doing the best that is possible. Nevertheless, there are still some limitations due to system and tool issues. See the updated doc/docbook/README.developers for discussion of those UTF-8 details. There may also be some style issues in the present results that need to be addressed since I used default style virtually everywhere except the part of the HTML styling that was done with a cascaded stylesheet. One known limitation of the present method of using dblatex with the xetex backend is the only way to generate PostScript results is to use pdftops to convert the generated PDF results to PostScript. I implemented that method in the CMake code, but after one look at the results (which were 50 (!) times larger than the corresponding pdf results, slow to generate, slow to execute, and ugly as well) I disabled it. This disabling of PostScript results means our next PLplot release would not have documentation in PostScript form. I think that is probably OK since PDF readers are widely available to read the PDF form of our documentation, but I welcome additional comments on dropping PostScript from anybody here who might still care about that form of our documentation. @Andrew and Orion: I would appreciate your comments on this change in the way we generate the pdf form of our documentation. In particular does it build well and give good looking results on the various Linux platforms accessible to you? For example, Andrew, I think you expressed some concern before about cross-references not being in red for the pdf version? I am happy to report that red cross-reference links are present (as a result of the default style) with this new method of generating the pdf form of our documentation. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-10-06 08:03:21
|
Hi Alan, > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@be...] > Sent: Sunday, October 06, 2013 1:55 AM > To: Andrew Ross; Orion Poplawski > Cc: PLplot development list > Subject: Re: [Plplot-devel] Project to replace DocBook backend SGML tools with > XML tools > > On 2013-09-06 09:57-0700 Alan W. Irwin wrote: > > > One known limitation of the present method of using dblatex with the xetex backend > is the only way to generate PostScript results is to use pdftops to convert the > generated PDF results to PostScript. I implemented that method in the CMake code, > but after one look at the results (which were 50 (!) times larger than the > corresponding pdf results, slow to generate, slow to execute, and ugly as well) I > disabled it. > > This disabling of PostScript results means our next PLplot release would not have > documentation in PostScript form. I think that is probably OK since PDF readers are > widely available to read the PDF form of our documentation, but I welcome additional > comments on dropping PostScript from anybody here who might still care about that > form of our documentation. > I have used PostScript quite a lot in the past, but it has always been a bit troublesome on Windows - no direct support for printing such files, you always needed ghostview/ghostscript to view or print them. In my opinion PDF is far more important nowadays as a portable document format than PostScript, most definitely on Windows. Regards, Arjen DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |
From: Alan W. I. <ir...@be...> - 2013-10-06 20:11:23
|
This morning I found the unifont font which completely eliminates missing glyphs from the pdf form of our documentation. To try this for yourself locally, apply the following patch: Index: dblatex_stylesheet.xsl =================================================================== --- dblatex_stylesheet.xsl (revision 12579) +++ dblatex_stylesheet.xsl (working copy) @@ -1,11 +1,11 @@ <?xml version='1.0' encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version='1.0'> <xsl:param name="xetex.font"> - <xsl:text>\setmainfont{FreeSerif} + <xsl:text>\setmainfont{unifont} </xsl:text> - <xsl:text>\setsansfont{FreeSans} + <xsl:text>\setsansfont{unifont} </xsl:text> - <xsl:text>\setmonofont{FreeMono} + <xsl:text>\setmonofont{unifont} </xsl:text> </xsl:param> </xsl:stylesheet> However, I decided not to commit this change because although there are no missing glyphs in the pdf result, that result is quite ugly because unifont is a scaled, low-resolution, monospaced, bitmapped font. So for now, I will continue with FreeSerif, FreeSans, and FreeMono to obtain reasonable quality in our pdf results at the price of missing CJK glyphs (i.e., empty boxes instead of the Korean and Mandarin forms of the "peace" words) in those results. I have rewritten (revision 12584) the relevant discussion in doc/docbook/README.developers in light of this latest experience with an extremely comprehensive but low-quality font. To summarize that discussion, I think we have made the best possible choice of tools to deal with UTF-8 in our DocBook documentation. Nevertheless, for the pdf case alone we will be forced to make this compromise between comprehensiveness and quality of specific fonts until the xelatex/fontspec tools we indirectly use to build the pdf form of our documentation are modified to allow generic fonts to be specified (i.e., where fontconfig is allowed to do its job of automatically choosing the best specific serif, sans, or monotype system font for any unicode glyph that is encountered). Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2014-01-26 02:56:26
|
On 2013-10-05 16:55-0700 Alan W. Irwin wrote: > As of revision 12582, I have implemented this change [to pure UTF-8 approach] which I hope is > the last fundamental change in how we generate our documentation for > years to come. I have also substantially updated > doc/docbook/README.developers[...] > > The summary is the results for UTF-8 documentation are much improved > compared to previously, and we are probably doing the best that is > possible. Nevertheless, there are still some limitations due to > system and tool issues. See the updated doc/docbook/README.developers > for discussion of those UTF-8 details[....] To resurrect this 3-month-old thread, today for the first time I introduced UTF-8 symbols into api.xml. As expected from previous tests which included an extensive UTF-8 table in advanced.xml, those results rendered properly for the html, pdf, and info backends for our DocBook documentation. However, this was the first opportunity I had to test what happens with UTF-8 symbols with the man backend since that backend ignores all our DocBook files except for api.xml. I horsed around with nroff -man options, but could never get a good rendering of those symbols so I was a bit discouraged by that initial result, but finally I tried the actual man viewer and not "nroff -man" (by copying the relevant man pages to a temporary directory man/man3 and using "man -M./man <command name>"), and that method of testing our man pages rendered the UTF-8 symbols without issues. So I am happy with our generated man pages which include UTF-8 symbols, and (as of revision 12959) I have updated doc/docbook/README.developers accordingly. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |