Menu
▾
▴
[Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3
|
From: Dan M. <d-...@uc...> - 2000-11-13 04:22:34
|
Hi everybody,
I've been sitting on this for a few days, hoping I will have time to
polish it up and fill in a few missing parts. Things have gotten very
busy for me though and the mailing list has gotten more active lately, so
I think it is best I just send this so everybody can see where we are at.
The main difference between this proposal and proposal #2 is the
installation method. We originally passed an argument to the installation
script which would announce where the new OMF files are at. The benefit
was that the OMF file could be installed anywhere (potentially even on a
different computer) and that ScrollKeeper would know exactly what to
install/uninstall without wasting any time finding out by other means.
The downside was that it made things somewhat complicated for developers
and packagers, since they had to prepare their metadata before
installation and pass more variables around. After talking with several
people about this, we were "strongly encouraged" to abort our previous
plan and have all of the OMF files installed in a central location and
then have the install script detect changes in this directory. The speed
penalty does not seem significant according to our tests, so I think the
benefit in terms of simplicity to developers and packagers as well as in
robustness against inevitable errors made by packagers outways the
slowdown.
Feedback is welcome :)
Dan
-----------
--------------------------------------
| ScrollKeeper 0.1 - Proposal #3 |
--------------------------------------
This document is arranged as follows:
I. Definitions and Overview
II. Building, Installing, and Uninstalling
III. Other Tools for Admin and Packaging
IV . ScrollKeeper Internals
V. Example File Snippits
I. Definitions and Overview
Contents List - This is a list of documents on the system (or more
generally, including docs which are not local to the single machine),
sorted into a tree. Generally the tree will be sorted by subject, probably
using a controlled list of subjects.
TOC (Table of Contents) - This is a tree representing the sections and
subsections of a document. The TOC can be extracted from an SGML/XML file.
Extended Contents List - This is the Contents List with the TOC for each
document merged into it. Thus, one can not only navigate through the
the docs on the system, but also the sections within each doc, all in
one tree.
Index - This is a list of terms/concepts with links to parts of a document
discussing each of the terms. This is often found at the back of printed
books. In DocBook, indexing information is manually inserted into the
document by an indexer using indexing markup. Thus, an Index can be
extracted from an SGML/XML file.
Generally a help browser will want to present all of the above information
to the user. The Contents List is determined from OMF metadata, either
from a standalone file or from a DocBook doc. The TOC and Index are generated
directly from the doc. Since all of this is metadata and must be accessed
rather frequently, and to avoid potential performance problems, this
metadata will be processed only at install and uninstall time. The
data will be stored in XML files for easy access by the help browser as
necessary.
II. Building, Installing, and Uninstalling
i. Building
The OMF documents are manipulated during the application
build process, generally as part of 'make all'. The purpose of this
is to (1) substitute the correct URL of the document where it will
actually be installed, which is not generally known until build time,
and to (2) extract the OMF from DocBook documents into a seperate file.
(NOTE: #2 may not be supported in ScrollKeeper 0.1)
This processing is done by 'scrollkeeper-preinstall'. This generally
takes three parameters: the URL for the installed document, the
original OMF metadata file in XML, and the name that the generated
OMF file should be written to. For example:
all:
<stuff>
scrollkeeper-preinstall file:$(DOCDIR)/foo-manual.sgml foo-manual-fr.omf omf-install/foo-manual-fr.omf
If the path where the generated OMF file does not exist, it is created.
So in this example, the IDENTIFIER in foo-manual-fr.omf is substituted with
a file URL which will point to the installed document and the
subsequent OMF file is written in a new directory called "omf-install".
Note that the OMF file is named <application>-<document_title>-<locale>.omf.
This is important to avoid collisions between OMF files when they are
installed.
ii. Installing
Installation is done by first copying the document and OMF file into
place and then having ScrollKeeper update its database by calling
'scrollkeeper-update'.
install:
install foo-manual.sgml $(DOCDIR)
install omf-install/foo-manual-fr.omf $(OMFDIR)
scrollkeeper-update -p $(SCROLLKEEPER_DB_DIR)
$(SCROLLKEEPER_DB_DIR) is the directory to use as the ScrollKeeper
database directory. If $(localstatedir) is set during the configure
stage, $(SCROLLKEEPER_DB_DIR) should be set to
$(localstatedir)/lib/scrollkeeper. If not, $(SCROLLKEEPER_DB_DIR)
should be set to `scrollkeeper-config --pkglocalstatedir`.
The -p option is important for building and testing purposes and should
always be used when packaging applications. When building RPMs, for
example, it guarantees that instead of actually modifying the primary
database on the builder's machine, it generates and modifies a new
database under BUILDROOT. It is clearly useful for testing purposes
as well.
scrollkeeper-update identifies the presence of a new OMF file in
OMFDIR and registers new metadata.
Uninstalling is done using 'scrollkeeper-update' as well.
uninstall:
rm $(DOCDIR)/foo-manual.sgml
rm $(OMFDIR)/foo-manual-fr.omf
scrollkeeper-update -p $(SCROLLKEEPER_DB_DIR)
scrollkeeper-update identifies that the OMF file was removed from
OMFDIR and unregisters the old metadata.
III. Other Tools for Admin and Packaging
In addition to the functions described above (scrollkeeper-preinstall
and scrollkeeper-update), ScrollKeeper provides the following additional
utilities:
scrollkeeper-rebuilddb:
This is used to rebuild the ScrollKeeper database completely using only
the original OMF files in OMFDIR. It also has a "-p" option which
allows one to specify the output directory of the newly created
database files.
scrollkeeper-config:
This is used to return various configuration paths for ScrollKeeper.
Options:
--help Prints scrollkeeper-config usage.
--version Prints Scrollkeeper version.
--prefix Scrollkeeper configure prefix. (eg. /usr)
--localstatedir Scrollkeeper configure localstatedir. (eg. /var)
--pkglocalstatedir Scrollkeeper data directory. (eg. /var/lib/scrollkeeper)
--pkgdatadir Scrollkeeper home directory. (eg. /usr/share/scrollkeeper)
--omfdir Directory where ScrollKeeper looks for OMF files.
(eg. /usr/doc/omf)
IV. ScrollKeeper Internals
This section describes some details about scrollkeeper functions.
$SCROLLKEEPER_DB_DIR is the directory where ScrollKeeper keeps its data
files. (It is typically under /var.)
$SCROLLKEEPER_DIR is the directory where ScrollKeeper installs its
static non-database files, such as the README and license files. (It is
typically under /usr or /share.)
$OMFDIR is the directory where packages must install their OMF files and
where ScrollKeeper looks to find OMF files. (This is typically
$prefix/doc/omf.)
scrollkeeper-update -p <scrollkeeper_db_dir>:
(Note: the -p flag is optional. If omitted, the path is determined
using `scrollkeeper-config --pkglocalstatedir`. However this flag
should not be omitted when scrollkeeper-update is used in Makefiles
since building often uses BUILDROOT's.)
It looks for added, removed, or updated files in OMFDIR and then calls
scrollkeeper-install and scrollkeeper-uninstall as necessary.
scrollkeeper-install -p <scrollkeeper_db_dir> <omfname>
1) Validate <omfname>
2) For each entry in <omfname> do the following:
A) Verify that the URL in the OMF entry exists. (eg. the file
it points to on the drive actually exists.) If it fails this
test, abort all further action on this OMF entry.
B) Add the entry to <scrollkeeper_db_dir>/scrollkeeper-docs.
(This file is created if it doesn't exist.) (see V. for format)
C) Update Contents List:
a) Determine title, language, and position(s) in Contents List from
TITLE, LANGUAGE, and SUBJECT.toc respectively.
b) Verify that the SUBJECT.toc lies within our controlled list
specified by `scrollkeeper-config --pkgdatadir`/contents-tree.xml
(Do we really want ScrollKeeper to rigorously enforce the
controlled list? Perhaps this should be left up to the help browser?)
c) Add this document to the Content List file in the appropriate
language's directory:
<scrollkeeper_db_dir>/<LANGUAGE>/scrollkeeper-contents-list.xml
(See IV. for format.) (Notice the doc ID is used as an attribute
to make it uninstallation easy.)
(This file is created if it doesn't exist.)
D) (for SGML and XML docs only) Extract TOC from doc, placing it
into a file: <scrollkeeper_db_dir>/TOC/<IDNUMBER>
(format to be determined)
(This file is created if it doesn't exist, or overwritten if it does.)
E) (for SGML and XML docs only) Insert TOC into an
"extended contents list". This is the Contents List with TOC listed
as a sub-tree for each doc. As above, the doc ID is used to make
uninstallation easy. This file is at
<scrollkeeper_db_dir>/<LANGUAGE>/scrollkeeper-extended-contents-list.xml
(This file is created if it doesn't exist.)
scrollkeeper-uninstall -p <scrollkeeper_db_dir> <omfname>
[ Fill in later. Should be trivial since the contents-lists and
toc are all identified by doc id.]
scrollkeeper-rebuilddb -p <scrollkeeper_db_dir>:
(Note: The option '-p' is optional. If omitted, the path is determined
using `scrollkeeper-config --pkglocalstatedir`.)
This will cause scrollkeeper to:
a) Move all of its files under <scrollkeeper_db_dir> into some
temporary location. (or maybe just delete them.)
b) Run scrollkeeper-update -p <scrollkeeper_db_dir> to enter
all OMF files into a new database in <scrollkeeper_db_dir>.
This can be used whenever the database is corrupted or becomes out-of-
date. It is also an easy (brute-force) way to allow for scrollkeeper
to update its databases when a new version is installed.
-- For use by help browsers --
scrollkeeper-get-contents-list <language>:
1) Returns the path:
${pkglocalstatedir}/<language>/scrollkeeper-contents-list.xml
scrollkeeper-get-extended-contents-list <language>:
1) Returns the path:
${pkglocalstatedir}/<language>/scrollkeeper-extended-contents-list.xml
scrollkeeper-get-toc-from-docpath <docpath>:
1) Looks in ${pkglocalstatedir}/scrollkeeper-docs to find doc id for <docpath>
2) Returns the path:
${pkglocalstatedir}/TOC/<docid>
[ Note that extending this all to include the Index should be very
similar. ]
V. Example File Snippits
scrollkeeper-docs:
This is used to keep track of which docs are on the system
(and possibly some remote docs), listing the location of
their OMF metadata, and identifying each doc by a code which
can be used to simplify uninstallation and some other tasks.
It also has a timestamp from when the OMF file was last written
so that scrollkeeper-update can detect when an OMF file has
been updated.
/usr/doc/omf/foo-manual-fr.omf 000001 /usr/doc/foo/foo-manual.sgml Sun Oct 29 13:48:50
/usr/doc/omf/bar-manual-en_US.omf 000002 /usr/doc/bar/bar-manual.sgml Tue Oct 24 02:47:28
/usr/doc/omf/foob-manual-da.omf 000003 /usr/doc/foob/foob-manual.sgml Tue Oct 24 02:47:28
contents-tree.xml:
This is the controlled list for valid Contents List sections.
scrollkeeper-contents-list.xml:
This is a simple XML file which reveals what the Contents List
looks like and where the OMF files and doc are at. This is used
by the help browser to display the Contents List.
[We will need to write up a simple DTD.]
<sect>
<title>Applications</title>
<sect>
<title>Games</title>
<doc docid="000050">
<doctitle>FreeCell</doctitle>
<docomf>/usr/local/games/freecell/freecell-omf.xml</docomf>
<docsource>/usr/local/games/freecell/freecell.sgml</docsource>
<docformat>SGML</docformat>
</doc>
</sect>
</sect>
TOC/<IDNUMBER>:
Contains a description of the sectioning layout of the doc.
It also must provide URI's for each doc section. If every
section has an "id", then this is straightforward. If not,
then .... (?? FILL THIS IN ??) (How should we store this?
XML or a flat file?)
scrollkeeper-extended-contents-list.xml:
Similar to scrollkeeper-contents-list.xml, except has TOC info in
it too. This is used by the help browser to display the
"extended Contents List".
<sect>
<title>Applications</title>
<sect>
<title>Games</title>
<doc docid="000050">
<doctitle>FreeCell</doctitle>
<docomf>/usr/local/games/freecell/freecell-omf.xml</docomf>
<docsource>/usr/local/games/freecell/freecell.sgml</docsource>
<docformat>SGML</docformat>
<toc>
<tocsect1 linkid="introduction">Introduction to FreeCell</tocsect1>
<tocsect1 linkid="playing">Playing FreeCell
<tocsect2 linkid="winning">Winning FreeCell</tocsect2>
</tocsect1>
</toc>
</doc>
</sect>
</sect>
foo-manual.omf:
[FIXME]
|
