scrollkeeper-devel — Developer discussion list for ScrollKeeper

Re: [Scrollkeeper-devel] questions, questions

[Dan M. <d-...@uc...>]

On Thu, 30 Nov 2000, Laszlo Kovacs wrote:

> Bob Stayton wrote:
> > Also, there seems to be a running assumption that an
> > XML-aware browser is available that understands a
> > Contents List and can present it to the user.
> > Is this based on the Gnome and KDE browsers under development?
> > Are other applications expected to use the Contents Lists?
>
> I think Ali answered this mostly. Any browser can use Scrollkeeper as
> long as they understand XML.

Right now we are focussing on the internal workings of ScrollKeeper and
how documents are installed. Note that eventually only scrollkeeper itself
will care how it sorts and stores its data.  We can always change to
another method to store the internal data if we find a better way.

Once we have things working better and the design has stabalized, we can
start putting in a real library to export to help browsers.  For right
now, ScrollKeeper actually points the help browser to its ScrollKeeper's
internal database files which are in XML format.  This is enough to get it
working with GNOME's help browser and any others which are willing to
handle XML files.  We can use this for testing and development. (Remember,
we don't even have a 0.1 version done yet.)  As things mature, we will
eventually hide the internal database from the help browser and provide a
real exported API which potentially will support not only XML-aware help
browsers but perhaps others as well (if there is a demand for it).

As for just supporting GNOME and KDE, this is definitely *not* the
intention.  There are many people who do not use GNOME or KDE, and that
will always be the case.  I would be happy to see another help browser
which is independent of these two desktops and which can utilize
scrollkeeper.  If there was a compelling reason to use some format other
than XML, then scrollkeeper's API could be extended.

Dan

[Scrollkeeper-devel] Re: Scrollkeeper backend

[Ali A. <ali...@au...>]

* Bob Stayton (bo...@sc...) wrote at 09:24 on 01/12/00:
> > From: Ali Abdin <ali...@au...>
> 
> > > You just contradicted yourself and confused me. If the XML is a purely
> > > internal data structure, why is Nautilus reading it?
> > 
> > Yes, I did contradict myself :) 
> > 
> > Its not really an internal data structure or file format.
> > 
> > It is an external one, and instead of devising our own, we used an easily
> > parsable existing one...XML. I mean if you really want we could use a fast
> > binary database, but I doubt people would appreciate that ;)
> 
> I completely agree that XML is a good standard format to export
> data to applications like a help browser.  And of course
> the incoming data is XML (the OMF files).  
> 
> But I think there is room in the current design for a
> backend using a fast binary database to house the metadata.
> Whether it will be appreciated depends entirely on the
> applications that might use it.  Scrollkeeper could be used
> to keep track of metadata for thousands of documents.
> Those documents might be served upon request by an HTTP
> server, after a query engine processes a metadata request.
> You won't want to scan a big XML metadata document for each
> such request.
> 
> I'm thinking that Scrollkeeper could maintain an internal
> lightweight database of the OMF data.  A database is easy
> to update when one OMF file added, changed, or deleted. And you
> can extract an XML Content List in various forms as
> queries and reports from the database.  The source of the
> data will always be the original OMF files, so you can
> still rebuild the database from scratch.
> But a database would give sk greater flexibility and
> performance in making the metadata available to different
> applications.
> 
> I'm not volunteering anyone else's copious free time
> to do this.  8^)
> If the group is interested, I'd be happy to 
> pursue this area a little further.  There are several
> available technologies that might fit the bill.

I think the only really viable solution is to use Berkley DB ;) That is the
only cross-platform Free Software database backend that I know of.

Regards,
Ali

Re: [Scrollkeeper-devel] questions, questions

[Ali A. <ali...@au...>]

* Greg Ferguson (gf...@ho...) wrote at 02:57 on 01/12/00:
> On Nov 30, 10:08am, Laszlo Kovacs wrote:
> > Subject: Re: [Scrollkeeper-devel] questions, questions
> > > On Wed, Nov 29, 2000 at 09:59:55PM -0200, Ali Abdin wrote:
> > > ...
> > > You just contradicted yourself and confused me. If the XML is a purely
> > > internal data structure, why is Nautilus reading it?
> >
> > The way how it works is that XML files are exported from Scrollkeeper
> > to the browser. The main Content List (where every entry is a different
> > doc), the TOCs of each doc and the combination of the two (called the
> > Extended Content List) are all XML files. Apart from that everything Ali
> > wrote is valid (about the availability of the XML parsers on various
> > platforms, Gnome, Nautilus etc).
> 
> Does the browser receive a ptr to the contents of the file,
> ptr to the filename, ptr to an internal XML-based data struct,
> or something else?
> 
> Or put a different way - is there a spec available for the public API?

Right now there is no library at all. You have to manually locate and parse
the file ;) The project is still at 0.0.1 for a reason.

Perhaps when there is a library we can have functions to get the ptr to the
filename and a function to return the "internal XML-based data struct" (a.k.a
xmlDocPtr)

You could perhaps also recieve document "fragments" (xmlNodePtr?)

Regards,
Ali

[Scrollkeeper-devel] Scrollkeeper backend

[Bob S. <bo...@sc...>]

> From: Ali Abdin <ali...@au...>

> > You just contradicted yourself and confused me. If the XML is a purely
> > internal data structure, why is Nautilus reading it?
> 
> Yes, I did contradict myself :) 
> 
> Its not really an internal data structure or file format.
> 
> It is an external one, and instead of devising our own, we used an easily
> parsable existing one...XML. I mean if you really want we could use a fast
> binary database, but I doubt people would appreciate that ;)

I completely agree that XML is a good standard format to export
data to applications like a help browser.  And of course
the incoming data is XML (the OMF files).  

But I think there is room in the current design for a
backend using a fast binary database to house the metadata.
Whether it will be appreciated depends entirely on the
applications that might use it.  Scrollkeeper could be used
to keep track of metadata for thousands of documents.
Those documents might be served upon request by an HTTP
server, after a query engine processes a metadata request.
You won't want to scan a big XML metadata document for each
such request.

I'm thinking that Scrollkeeper could maintain an internal
lightweight database of the OMF data.  A database is easy
to update when one OMF file added, changed, or deleted. And you
can extract an XML Content List in various forms as
queries and reports from the database.  The source of the
data will always be the original OMF files, so you can
still rebuild the database from scratch.
But a database would give sk greater flexibility and
performance in making the metadata available to different
applications.

I'm not volunteering anyone else's copious free time
to do this.  8^)
If the group is interested, I'd be happy to 
pursue this area a little further.  There are several
available technologies that might fit the bill.

bobs
Bob Stayton                                 400 Encinal Street
Publications Architect                      Santa Cruz, CA  95060
Technical Publications                      voice: (831) 427-7796
The Santa Cruz Operation, Inc.              fax:   (831) 429-1887
                                            email: bo...@sc...

Re: [Scrollkeeper-devel] questions, questions

[Ali A. <ali...@au...>]

* David Merrill (dme...@lu...) wrote at 18:09 on 30/11/00:
> On Wed, Nov 29, 2000 at 09:59:55PM -0200, Ali Abdin wrote:
> > XML is just the file format that ScrollKeeper uses to store its own "data".
> > Wether GNOME and KDE have an XML browser is irrelevant. The fact is,
> > ScrollKeeper could have used its own non-standard file format, or it could
> > have used the easily parsable XML file format (there are tons of XML parses on
> > almost all platforms, and it beats creatign your own parser for the format).
> > ScrollKeeper uses libxml to do its own parsing, so that does limit your
> > choices (but libxml is cross-platform, doesn't depend on any other libraries
> > (although it is part of the GNOME platform), and it is written in C (meaning
> > you can create wrapper APIs in other languages)).
> > 
> > GNOME has a file-manager called Nautilus (in development), in this file
> > manager there is a 'Help Sidebar'. This sidebar presents the documents
> > installed on the system to the user. Sun submitted a patch to make it utilize
> > the scrollkeeper XML file.
> > 
> > When you select an item in this Help Sidebar, /then/ the Help Browser starts
> > up and displays the document (based on the document URI).
> 
> You just contradicted yourself and confused me. If the XML is a purely
> internal data structure, why is Nautilus reading it?

Yes, I did contradict myself :) 

Its not really an internal data structure or file format.

It is an external one, and instead of devising our own, we used an easily
parsable existing one...XML. I mean if you really want we could use a fast
binary database, but I doubt people would appreciate that ;)
 

The real reason though is: To be buzzword compliant :)

Regards,
Ali

Re: [Scrollkeeper-devel] questions, questions

[Greg F. <gf...@ho...>]

On Nov 30, 10:08am, Laszlo Kovacs wrote:
> Subject: Re: [Scrollkeeper-devel] questions, questions
> > On Wed, Nov 29, 2000 at 09:59:55PM -0200, Ali Abdin wrote:
> > ...
> > You just contradicted yourself and confused me. If the XML is a purely
> > internal data structure, why is Nautilus reading it?
>
> The way how it works is that XML files are exported from Scrollkeeper
> to the browser. The main Content List (where every entry is a different
> doc), the TOCs of each doc and the combination of the two (called the
> Extended Content List) are all XML files. Apart from that everything Ali
> wrote is valid (about the availability of the XML parsers on various
> platforms, Gnome, Nautilus etc).

Does the browser receive a ptr to the contents of the file,
ptr to the filename, ptr to an internal XML-based data struct,
or something else?

Or put a different way - is there a spec available for the public API?

thx.

-- 
Greg Ferguson     - s/w engr / mtlhd         | gferg at sgi.com
SGI Tech Pubs     - http://techpubs.sgi.com/ |
Linux Doc Project - http://www.linuxdoc.org/ | gferg at metalab.unc.edu

Re: [Scrollkeeper-devel] questions, questions

[Laszlo K. <las...@su...>]

Bob Stayton wrote:
> 
> [Pardon me if you receive a duplicate of this message.
> The first copy didn't seem to make it to the list.]
> 
> I've been studying the current design (proposal #3) of
> scrollkeeper and catching up on the recent sk mail.  Thanks
> for sorting out the issues into smaller mails.
> I would like to present scrollkeeper to some colleagues, but
> there are some areas that I need to ask you questions about.
> I'll try to keep this short. :)
> 
> I'm a little confused about the database, the Contents List,
> and TOCs.  From the design document, it appears that the
> database is a flat file database named scrollkeeper-docs
> that consists of records with four fields:
> 
> <URI to OMF file>  <uniqueID>  <URI to document>   <timestamp>
> 
> This file has one record for each document registered
> with sk, right?

yes.

> 
> Also, this database does not actually contain the OMF data
> itself, right?

yes.

> 
> So the "master" source for each document's OMF
> data resides in its OMF file.  In effect, the database
> of OMF data is a combination of the scrollkeeper-docs
> file and the collection of OMF files its records point to.

yes, but I would probably add the Content List files for each locale
also.

> 
> A Contents List is a selection of data from this extended
> database that is serialized into an XML file.  Each Contents
> List contains information on documents in one language,
> since the function "scrollkeeper-get-contents-list <language>"
> returns the path:
> ${pkglocalstatedir>/<language>/scrollkeeper-contents-list.xml

yes.

> 
> Within a Contents List file, the documents are sorted into
> the category hierarchy within nested <sect> elements.
> Each document's data is listed within a <doc> element.
> An alternative proposal from Nik has the documents in a
> flat list with the categories as subelements for each
> document to reduce data redundancy.

yes.

> 
> The data in a <doc> element is a selection of the
> OMF data for the document, right?  In the example,
> I saw title, source, format, and <docomf>.  I presume
> the <docomf> is there so an application can track down
> the OMF file if they need more of its data?

yes. However we do not use this at all at the moment.

> 
> Proposal #3 also mentions that for each SGML and XML document,
> a document TOC is extracted and kept in the TOC directory
> under an IDNUMBER.  This is to be inserted into the
> extended Contents List.  Its format is to be determined.

Yes, but this is actually implemented according to proposal #3. You can
find how the TOC and the extended content list looks like at the end of
the proposal, I think.

> But then some later mails described generating a ToC from
> a Contents List:
> 
>   Date: Wed, 22 Nov 2000 21:55:45 -0600 (CST)
>   From: Dan Mueth <d-...@uc...>
> 
>   > It's my understanding that there's a 1:1 mapping between a Contents List
>   > and a ToC.  That is, you can't have a ToC that includes information
>   > about documents that aren't in the Contents List.
> 
>   Yes.
Yes this is correct. I think what the above meant is that when you
install a doc with Scrollkeeper, a TOC is generated for it, when you
uninstall it, this is deleted. So there will not be TOCs hanging around
that don't have their doc installed with Scrollkeeper.

> 
> This was one of those long mails (8^) and I kind of lost track
> of the context, so I wasn't sure what it was talking about.
> It is my understanding that the help browser was supposed
> to read the Contents List and present that to the user.
> What ToC has a 1:1 mapping to the Contents List?  Is that
> an HTML file generated from the Contents List?
I think I answered this above. The TOC is XML file, generated from the
doc and offered to the browser. So you have a content list (one entry
for each doc) and each doc from this has a TOC. This is all offered to
the browser. Also The content list file and the TOCs combined form the
so called extended content list. We are not sure how useful it is to
have an extended content list, we will see this later.


> 
> Also, there seems to be a running assumption that an
> XML-aware browser is available that understands a
> Contents List and can present it to the user.
> Is this based on the Gnome and KDE browsers under development?
> Are other applications expected to use the Contents Lists?
I think Ali answered this mostly. Any browser can use Scrollkeeper as
long as they understand XML.

Laszlo

Re: [Scrollkeeper-devel] questions, questions

[Laszlo K. <las...@su...>]

> On Wed, Nov 29, 2000 at 09:59:55PM -0200, Ali Abdin wrote:
> > XML is just the file format that ScrollKeeper uses to store its own "data".
> > Wether GNOME and KDE have an XML browser is irrelevant. The fact is,
> > ScrollKeeper could have used its own non-standard file format, or it could
> > have used the easily parsable XML file format (there are tons of XML parses on
> > almost all platforms, and it beats creatign your own parser for the format).
> > ScrollKeeper uses libxml to do its own parsing, so that does limit your
> > choices (but libxml is cross-platform, doesn't depend on any other libraries
> > (although it is part of the GNOME platform), and it is written in C (meaning
> > you can create wrapper APIs in other languages)).
> >
> > GNOME has a file-manager called Nautilus (in development), in this file
> > manager there is a 'Help Sidebar'. This sidebar presents the documents
> > installed on the system to the user. Sun submitted a patch to make it utilize
> > the scrollkeeper XML file.
> >
> > When you select an item in this Help Sidebar, /then/ the Help Browser starts
> > up and displays the document (based on the document URI).
> 
> You just contradicted yourself and confused me. If the XML is a purely
> internal data structure, why is Nautilus reading it?

The way how it works is that XML files are exported from Scrollkeeper to
the browser. The main Content List (where every entry is a different
doc), the TOCs of each doc and the combination of the two (called the
Extended Content List) are all XML files. Apart from that everything Ali
wrote is valid (about the availability of the XML parsers on various
platforms, Gnome, Nautilus etc).

Laszlo

Re: [Scrollkeeper-devel] questions, questions

[David M. <dme...@lu...>]

On Wed, Nov 29, 2000 at 09:59:55PM -0200, Ali Abdin wrote:
> XML is just the file format that ScrollKeeper uses to store its own "data".
> Wether GNOME and KDE have an XML browser is irrelevant. The fact is,
> ScrollKeeper could have used its own non-standard file format, or it could
> have used the easily parsable XML file format (there are tons of XML parses on
> almost all platforms, and it beats creatign your own parser for the format).
> ScrollKeeper uses libxml to do its own parsing, so that does limit your
> choices (but libxml is cross-platform, doesn't depend on any other libraries
> (although it is part of the GNOME platform), and it is written in C (meaning
> you can create wrapper APIs in other languages)).
> 
> GNOME has a file-manager called Nautilus (in development), in this file
> manager there is a 'Help Sidebar'. This sidebar presents the documents
> installed on the system to the user. Sun submitted a patch to make it utilize
> the scrollkeeper XML file.
> 
> When you select an item in this Help Sidebar, /then/ the Help Browser starts
> up and displays the document (based on the document URI).

You just contradicted yourself and confused me. If the XML is a purely
internal data structure, why is Nautilus reading it?

-- 
Dr. David C. Merrill                     http://www.lupercalia.net
Linux Documentation Project                dme...@lu...
Collection Editor & Coordinator            http://www.linuxdoc.org

Boucher's Observation:
	He who blows his own horn always plays the music
	several octaves higher than originally written.

Re: [Scrollkeeper-devel] Simple URI's

[Ali A. <ali...@au...>]

* Laszlo Kovacs (las...@su...) wrote at 21:43 on 29/11/00:
> > > 2) Mapping the names.  The OMF file should specify the name by which the
> > > document should be referred.  This seems to require yet another OMF
> > > attribute to be created, perhaps IDENTIFIER.URINAME?  We will not have
> > > complete uniqueness here.  However ideally we will have a 1-1
> > > correspondence between NUID's and URINAME's.  We could keep a database on
> > > the web so people can register names and find out if a name is taken.
> > > This creates even more administrative overhead, but I think it is well
> > > worth the trouble it would take to set up a simple php/mysql site on
> > > SourceForge.
> > 
> > Not sure this is a good idea...you will have to have a 'URINAME' if you
> > are typing the URI manually.
> > 
> > BUT:
> > 
> > I think that applications should give scrollkeeper a "title", locale
> > (optional), and version # (optional), format (default would be Docbook).
> > Scrollkeeper would then search its database and return the unique IDs
> > (possible there will be more than one doc matching the criteria, so the
> > client would either pick the first one, or try to do some other
> > processing (e.g: See Also)).
> > 
> > You could also have another function that would return the "first found
> > match" of the document directly (so you don't have to search for UUID,
> > and then tell scrollkeeper to return this document - you just tell it
> > "return the first document that matches this criteria").
> > 
> > Anyway, this approach would only work if you have a list of documents to
> > choose from.
> > 
> > So, maybe URINAME's are the way to go.
> 
> Are you suggesting to try to partially match the string from the
> Scrollkeeper URI with the "title" that you described above? 

Well, you could use the title - but you would have to implement proper URI
escaping/handling (i.e. use %20 instead of a space, etc.)

This would suck though, because people are not going to make links to
sk-help:PPP%20HowTo (note: it would be case sensitive so HowTo may not work
(it might be HOWTO, or Howto - or it even could be 'How-to' or 'HOW-TO', etc.)
The title could be 'Gnumeric Manual' and people will not want to type all of
that out (and they may forget its name, was it 'Gnumeric manual?' or 'Gnumeric
Help', or 'Gnumeric Tutorial', etc.)

So I don't think title is a good idea.

We have the option of a new field called 'URINAME'. This means that there
needs to be a 'unique' URINAME for each help file, so the line between UUIDs
and URINAMEs become blurred (the only diff. is a doc in another locale has a
diff. UUID, but the same URINAME)...

I suggest you just go ahead and add an extra URINAME field (note you will
still need to handle propery escaping, but I doubt people will make the
URINAME "complex" (otherwise doc. authors just wont' link to it because they
can't remember the proper URINAME to use).

Regards,
Ali

Re: [Scrollkeeper-devel] questions, questions

[Ali A. <ali...@au...>]

* Bob Stayton (bo...@sc...) wrote at 21:52 on 29/11/00:
> [Pardon me if you receive a duplicate of this message.
> The first copy didn't seem to make it to the list.]
> 
> I've been studying the current design (proposal #3) of
> scrollkeeper and catching up on the recent sk mail.  Thanks
> for sorting out the issues into smaller mails.
> I would like to present scrollkeeper to some colleagues, but
> there are some areas that I need to ask you questions about.
> I'll try to keep this short. :)
> 
> I'm a little confused about the database, the Contents List,
> and TOCs.  From the design document, it appears that the
> database is a flat file database named scrollkeeper-docs
> that consists of records with four fields:
> 
> <URI to OMF file>  <uniqueID>  <URI to document>   <timestamp>
> 
> This file has one record for each document registered
> with sk, right?
> 
> Also, this database does not actually contain the OMF data
> itself, right?
> 
> So the "master" source for each document's OMF
> data resides in its OMF file.  In effect, the database
> of OMF data is a combination of the scrollkeeper-docs
> file and the collection of OMF files its records point to.
> 
> A Contents List is a selection of data from this extended
> database that is serialized into an XML file.  Each Contents
> List contains information on documents in one language,
> since the function "scrollkeeper-get-contents-list <language>"
> returns the path:
> ${pkglocalstatedir>/<language>/scrollkeeper-contents-list.xml
> 
> Within a Contents List file, the documents are sorted into
> the category hierarchy within nested <sect> elements. 
> Each document's data is listed within a <doc> element.
> An alternative proposal from Nik has the documents in a
> flat list with the categories as subelements for each
> document to reduce data redundancy.
> 
> The data in a <doc> element is a selection of the
> OMF data for the document, right?  In the example,
> I saw title, source, format, and <docomf>.  I presume
> the <docomf> is there so an application can track down
> the OMF file if they need more of its data? 
> 
> Proposal #3 also mentions that for each SGML and XML document,
> a document TOC is extracted and kept in the TOC directory
> under an IDNUMBER.  This is to be inserted into the
> extended Contents List.  Its format is to be determined.
> 
> But then some later mails described generating a ToC from
> a Contents List: 
> 
>   Date: Wed, 22 Nov 2000 21:55:45 -0600 (CST)
>   From: Dan Mueth <d-...@uc...>
> 
>   > It's my understanding that there's a 1:1 mapping between a Contents List
>   > and a ToC.  That is, you can't have a ToC that includes information
>   > about documents that aren't in the Contents List.
> 
>   Yes.
> 
> This was one of those long mails (8^) and I kind of lost track
> of the context, so I wasn't sure what it was talking about.
> It is my understanding that the help browser was supposed
> to read the Contents List and present that to the user.
> What ToC has a 1:1 mapping to the Contents List?  Is that
> an HTML file generated from the Contents List?
> 
> Also, there seems to be a running assumption that an
> XML-aware browser is available that understands a
> Contents List and can present it to the user.
> Is this based on the Gnome and KDE browsers under development?
> Are other applications expected to use the Contents Lists?

XML is just the file format that ScrollKeeper uses to store its own "data".
Wether GNOME and KDE have an XML browser is irrelevant. The fact is,
ScrollKeeper could have used its own non-standard file format, or it could
have used the easily parsable XML file format (there are tons of XML parses on
almost all platforms, and it beats creatign your own parser for the format).
ScrollKeeper uses libxml to do its own parsing, so that does limit your
choices (but libxml is cross-platform, doesn't depend on any other libraries
(although it is part of the GNOME platform), and it is written in C (meaning
you can create wrapper APIs in other languages)).

GNOME has a file-manager called Nautilus (in development), in this file
manager there is a 'Help Sidebar'. This sidebar presents the documents
installed on the system to the user. Sun submitted a patch to make it utilize
the scrollkeeper XML file.

When you select an item in this Help Sidebar, /then/ the Help Browser starts
up and displays the document (based on the document URI).

Regards,
Ali 

[Scrollkeeper-devel] questions, questions

[Bob S. <bo...@sc...>]

[Pardon me if you receive a duplicate of this message.
The first copy didn't seem to make it to the list.]

I've been studying the current design (proposal #3) of
scrollkeeper and catching up on the recent sk mail.  Thanks
for sorting out the issues into smaller mails.
I would like to present scrollkeeper to some colleagues, but
there are some areas that I need to ask you questions about.
I'll try to keep this short. :)

I'm a little confused about the database, the Contents List,
and TOCs.  From the design document, it appears that the
database is a flat file database named scrollkeeper-docs
that consists of records with four fields:

<URI to OMF file>  <uniqueID>  <URI to document>   <timestamp>

This file has one record for each document registered
with sk, right?

Also, this database does not actually contain the OMF data
itself, right?

So the "master" source for each document's OMF
data resides in its OMF file.  In effect, the database
of OMF data is a combination of the scrollkeeper-docs
file and the collection of OMF files its records point to.

A Contents List is a selection of data from this extended
database that is serialized into an XML file.  Each Contents
List contains information on documents in one language,
since the function "scrollkeeper-get-contents-list <language>"
returns the path:
${pkglocalstatedir>/<language>/scrollkeeper-contents-list.xml

Within a Contents List file, the documents are sorted into
the category hierarchy within nested <sect> elements. 
Each document's data is listed within a <doc> element.
An alternative proposal from Nik has the documents in a
flat list with the categories as subelements for each
document to reduce data redundancy.

The data in a <doc> element is a selection of the
OMF data for the document, right?  In the example,
I saw title, source, format, and <docomf>.  I presume
the <docomf> is there so an application can track down
the OMF file if they need more of its data? 

Proposal #3 also mentions that for each SGML and XML document,
a document TOC is extracted and kept in the TOC directory
under an IDNUMBER.  This is to be inserted into the
extended Contents List.  Its format is to be determined.

But then some later mails described generating a ToC from
a Contents List: 

  Date: Wed, 22 Nov 2000 21:55:45 -0600 (CST)
  From: Dan Mueth <d-...@uc...>

  > It's my understanding that there's a 1:1 mapping between a Contents List
  > and a ToC.  That is, you can't have a ToC that includes information
  > about documents that aren't in the Contents List.

  Yes.

This was one of those long mails (8^) and I kind of lost track
of the context, so I wasn't sure what it was talking about.
It is my understanding that the help browser was supposed
to read the Contents List and present that to the user.
What ToC has a 1:1 mapping to the Contents List?  Is that
an HTML file generated from the Contents List?

Also, there seems to be a running assumption that an
XML-aware browser is available that understands a
Contents List and can present it to the user.
Is this based on the Gnome and KDE browsers under development?
Are other applications expected to use the Contents Lists?

Thanks for taking the time to help me understand this.

bobs
Bob Stayton                                 400 Encinal Street
Publications Architect                      Santa Cruz, CA  95060
Technical Publications                      voice: (831) 427-7796
The Santa Cruz Operation, Inc.              fax:   (831) 429-1887
                                            email: bo...@sc...

Re: [Scrollkeeper-devel] draft article, for review

[Rich M. <rd...@cf...>]

At 11:58 AM +0000 11/29/00, Laszlo Kovacs wrote:
>Cool, I finally managed to read it. Can we get a link to the article
>when it appears in the magazine itself?

Unfortunately, SunWorld uses some sort of arcane naming system, so I
can't predict the exact URL.  The column should show up in December,
however, and it will be listed in the "Views" area of the main page
at http://www.sunworld.com.  Whoever notices the piece there should
feel free to send a link to the list...

-r
-- 
--
Rich Morin:          rd...@cf..., +1 650-873-7841, http://www.cfcl.com/rdm
Prime Time Freeware: in...@pt..., +1 408-433-9662, http://www.ptf.com
MacPerl: http://www.macperl.com, http://www.ptf.com/ptf/products/MPPE
MkLinux: http://www.mklinux.org, http://www.ptf.com/ptf/products/MKLP

[Scrollkeeper-devel] Re: draft article, for review

[Nik C. <ni...@fr...>]

Rich,

On Mon, Nov 27, 2000 at 11:09:54AM -0800, Rich Morin wrote:
> I have written up some recent experiences with FreeBSD documentation
> for my column (Silicon Carny) in SunWorld (http://www.sunworld.com).
> The column also attempts to summarize the aims of the ScrollKeeper
> and Meta Projects.
> 
> If you would like to look at the draft I'm submitting, take a look at
> http://www.cfcl.com/rdm/SC.200012.1.  Please send me email (in the
> next few days) about any problems you notice.  I should be able to
> squeeze in a few changes during final edit...

I'm keeping the cc's in place for this, because this is something I want
to reiterate to the commentators and lurkers on the FreeBSD-doc mailing
list (and the various Linux doc people on the ScrollKeeper list will
probably agree).

We're in complete agreement about the need for better documentation, and
the requirement for better searching and indexing tools.  Of this latter
requirement, I have high hopes for ScrollKeeper (once we've finished
batting around some of the design decisions).

Your choice of make.conf is perhaps a little unfortunate, as in the past
few weeks someone else has had exactly the same problem you had.  They
also realised that a make.conf manual page would be a good thing to have
for the reasons you specify in your article.

So, they wrote one, put together a PR (Problem Report, or Patch Report,
depending on who you listen to), visible at

    http://www.FreeBSD.org/cgi/query-pr.cgi?pr=22676

As you can see, this was then kicked back and forth a bit on the mailing
list, changes were made, and then it was committed to FreeBSD.  So
FreeBSD, thanks to the actions of one of its users, now has a
make.conf(5) manual page.

I don't doubt that there are other areas of the system that need further
documentation and explanation.  It's an unfortunate fact of life for all
of us that there are this >           < many things to do, and this > <
much time in which to do them.  This is why the FreeBSD project is very
open to receiving PRs, and I do my level best to bring people in to the
doc project as soon as they demonstrate that they can generate good PRs
and won't need an excessive amount of handholding.  In the recent past
this includes people like

    Bruce Mah
    Ben Smithurst
    Mark Ovens
    Daniel Harris
    Kazuo Horikawa
    Hiroki Sato
    Alexander Langer

and many others (Neil, Sheldon, Jeroen, the list is long, and an
omission just means I haven't got time to work through CVSROOT/access).

This is how the process works -- people find problems, fix them, and
submit patches.  Sometimes the people finding the problem are already
committers, but a lot of the time it's FreeBSD users.  And with the best
will in the world, a PR with a patch is a hell of a lot easier for a
committer to deal with than a message saying that something needs
fixing.

[ For example, the fact that sysinstall doesn't know about the
  new documentation packages yet -- I know this is a problem, as do
  people following this mailing list.  But unless someone else steps up
  to the plate (anyone!) with patches, the problem's going to have to
  wait until I've got the requisite 12 clear hours to sit down with the
  sysinstall code and write the glue. ]

N
-- 
Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
Telephone line, $24.95 a month.  Software, free.  USENET transmission,
hundreds if not thousands of dollars.  Thinking before posting, priceless.
Somethings in life you can't buy.  For everything else, there's MasterCard.
  -- Graham Reed, in the Scary Devil Monastery

Re: [Scrollkeeper-devel] Simple URI's

[Laszlo K. <las...@su...>]

> > 2) Mapping the names.  The OMF file should specify the name by which the
> > document should be referred.  This seems to require yet another OMF
> > attribute to be created, perhaps IDENTIFIER.URINAME?  We will not have
> > complete uniqueness here.  However ideally we will have a 1-1
> > correspondence between NUID's and URINAME's.  We could keep a database on
> > the web so people can register names and find out if a name is taken.
> > This creates even more administrative overhead, but I think it is well
> > worth the trouble it would take to set up a simple php/mysql site on
> > SourceForge.
> 
> Not sure this is a good idea...you will have to have a 'URINAME' if you
> are typing the URI manually.
> 
> BUT:
> 
> I think that applications should give scrollkeeper a "title", locale
> (optional), and version # (optional), format (default would be Docbook).
> Scrollkeeper would then search its database and return the unique IDs
> (possible there will be more than one doc matching the criteria, so the
> client would either pick the first one, or try to do some other
> processing (e.g: See Also)).
> 
> You could also have another function that would return the "first found
> match" of the document directly (so you don't have to search for UUID,
> and then tell scrollkeeper to return this document - you just tell it
> "return the first document that matches this criteria").
> 
> Anyway, this approach would only work if you have a list of documents to
> choose from.
> 
> So, maybe URINAME's are the way to go.

Are you suggesting to try to partially match the string from the
Scrollkeeper URI with the "title" that you described above? 

Laszlo

Re: [Scrollkeeper-devel] Simple URI's

[Laszlo K. <las...@su...>]

Dan Mueth wrote:
> 
> GNOME uses a "gnome-help" URI scheme.  So if you type
> "gnome-help:gnumeric", it looks in
> $prefix/share/gnome/help/gnumeric/<locale>/ and looks for the doc under
> the name gnumeric.sgml or else index.html.  It is a very convenient way to
> find a manual.  It also allows for cross-referencing of documents very
> simply since any document you write knows that "gnome-help:gnumeric" will
> point to the Gnumeric manual regardless of where things are installed (eg.
> under /usr or /opt or ...).
> 
> The problem with this is:
> 1) other help browsers have to understand how to handle the
> "gnome-help" URI scheme
> 2) presumably other desktops will introduce more URI schemes
> 3) many docs won't be addressable at all this way unless scrollkeeper
> provides a mechanism.
> 
> So probably what we want is to introduce a scheme which passes the
> identifier (what is the right term here?) to scrollkeeper which can return
> the path.  So for example, one would use: "sk-help:modem-HOWTO".  It
> would have to find the doc (and select from any available versions,
> languages, formats) and then display it.  This would be *very* useful IMO,
> both for users as well as for cross-referencing between documents.
> 
> A couple issues which pop up:
> 
> 1) Selecting the right doc.  We could have a default setting for language
> (just use the current locale), formats (perhaps in /etc/sk.conf we list in
> order of preference), and versions (show the biggest version number).
> 
> 2) Mapping the names.  The OMF file should specify the name by which the
> document should be referred.  This seems to require yet another OMF
> attribute to be created, perhaps IDENTIFIER.URINAME?  We will not have
> complete uniqueness here.  However ideally we will have a 1-1
> correspondence between NUID's and URINAME's.  We could keep a database on
> the web so people can register names and find out if a name is taken.
> This creates even more administrative overhead, but I think it is well
> worth the trouble it would take to set up a simple php/mysql site on
> SourceForge.
> 
> Thoughts?
> 
> Dan
> 

I think it is a good idea. The Gnome scheme definitely look simple,
something similar would be useful. We just need to figure out how
feasible it is to implement this with Scrollkeeper.

Laszlo

Re: [Scrollkeeper-devel] draft article, for review

[Laszlo K. <las...@su...>]

> > > Are you using:
> > >
> > > http://www.cfcl.com/rdm/SC.200012.1
> > >
> > > and not including the period at the end?
> > >
> > > It worked for me yesterday and again just now.
> > >
> > > Dan

Cool, I finally managed to read it. Can we get a link to the article
when it appears in the magazine itself?

Thanks,

Laszlo

Re: [Scrollkeeper-devel] Unique identifiers

[Ali A. <ali...@au...>]

* Dan Mueth (d-...@uc...) wrote at 23:54 on 28/11/00:
> 
> On Tue, 28 Nov 2000, Ali Abdin wrote:
> 
> > * Dan Mueth (d-...@uc...) wrote at 01:20 on 28/11/00:
> > > > We talked about this issue previously and we thought about an ISBN like
> > > > ID generated by a server available throughout the internet. This server
> > > > would generate unique IDs on request by document creators.
> > > 
> > > Alan and Daniel were kind enough to pop in on IRC to discuss scrollkeeper
> > > yesterday and the conversation quickly moved to UUID's.  They recommended
> > > (as Ali does) creating a UUID for each unique document.  Here a unique
> > > document is probably a {doc, version, language, format} quadruplet, or
> > > possibly even a superset of this.  It should uniquely specify the
> > > resource.  We will need to decide where this belongs in the OMF, perhaps
> > > an attribute to IDENTIFIER (ie. IDENTIFIER.UUID).  The UUID will be the
> > > database key.  To make things simple, we can ship a script which generates
> > > a UUID with scrollkeeper.  For systems with uuidgen, it will use that.  
> > > For other systems, we may want it to request one from a server which runs
> > > uuidgen (as suggested by Ali).  
> > 
> > It is important to point out that the UUID should not change for each
> > /version/ of a doc. If you have a doc and then update it, it will have the
> > _SAME_ uuid but a diff. version number, in my opinion. (by default, if you
> > just use UUID you will get the "latest" doc.)
> > 
> > Also - different formats should have the same UUID. They are the same
> > document, so they will have the same _METADATA_ (i.e. same author, same
> > version, same category, etc.)
> 
> One of the reasons we want a UUID is as a database key.  So it needs to be
> unique between multiple versions, formats, or languages of a document. 
> 
> We also want to guarantee that any two resources with the same UUID are in
> fact the same.  I'm not sure how we will want to handle multiple
> references to the same resource, but we will want to know if two resources
> are identical or not. This will be particularly important if we are ever
> merging multiple databases because you will have situations where both
> databases will refer to the exact same resource (although they may have
> different IDENTIFIERs.)  This is the reason for modifying the UUID for
> each version.

I think duplicates (i.e. "collisions") in of keys should be handled. I still
think that Documents that are the "same" (but diff. version) should have the
same UUID. If you want "further distinction" then you also use the version
key (and possibly file format).

If version and/or format is not specified then you just use the default
(latest version and some sort of priority-based format).
 
> I don't know if we would ever want to have duplicates of a resource in a
> database. (eg. http://www.foo.org/faq.sgml and
> file:/<stuff>/faq.sgml)  If this were the case, the database would need to
> use a key which combined UUID with a code generated from the IDENTIFIER.

No, just use the same UUID and allow the API to be a bit intelligent (i.e. by
specific version, format, wether to only return files installed locally, etc.)

You are saying that the UUID should be /always/ unique. I am saying there can
be duplicates.

Look at .oafinfo files, they do not changed their ID just because the oafinfo
file changes (I think).

> This is leaning slightly back to my original idea of basing the database
> key on the URL, but Alan and DV insisted that URL was not a good key and
> that we should use UUID's instead.

I agree with them :) and even if I didn't, I would still agree with them cos
they are better experts at this stuff.

Regards,
Ali

Re: [Scrollkeeper-devel] draft article, for review

[Rich M. <rd...@cf...>]

Laszlo Kovacs wrote:
>
>This link does not seem to work.

Sorry, I've been trying to clean up a problem in Apache,
but I seem to have made things worse.  Please try it now.

   http://www.cfcl.com/rdm/SC.200012.1

-r
-- 
--
Rich Morin:          rd...@cf..., +1 650-873-7841, http://www.cfcl.com/rdm
Prime Time Freeware: in...@pt..., +1 408-433-9662, http://www.ptf.com
MacPerl: http://www.macperl.com, http://www.ptf.com/ptf/products/MPPE
MkLinux: http://www.mklinux.org, http://www.ptf.com/ptf/products/MKLP

Re: [Scrollkeeper-devel] draft article, for review

[<jf...@in...>]

On Tue, 28 Nov 2000, Laszlo Kovacs wrote:

> > 
> > Are you using:
> > 
> > http://www.cfcl.com/rdm/SC.200012.1
> > 
> > and not including the period at the end?
> > 
> > It worked for me yesterday and again just now.
> > 
> > Dan
> 
> Yes. And it still doesnt work.
> 

A different address worked for me, which I found by backing up to the
root and then experimenting in the manner of a bored journalist:

http://www.cfcl.com/cfcl/rdm/SC.200012.1

Cheers,

John

Re: [Scrollkeeper-devel] draft article, for review

[Laszlo K. <las...@Su...>]

> 
> Are you using:
> 
> http://www.cfcl.com/rdm/SC.200012.1
> 
> and not including the period at the end?
> 
> It worked for me yesterday and again just now.
> 
> Dan

Yes. And it still doesnt work.

Laszlo

Re: [Scrollkeeper-devel] Unique identifiers

[Dan M. <d-...@uc...>]

On Tue, 28 Nov 2000, Ali Abdin wrote:

> * Dan Mueth (d-...@uc...) wrote at 01:20 on 28/11/00:
> > > We talked about this issue previously and we thought about an ISBN like
> > > ID generated by a server available throughout the internet. This server
> > > would generate unique IDs on request by document creators.
> > 
> > Alan and Daniel were kind enough to pop in on IRC to discuss scrollkeeper
> > yesterday and the conversation quickly moved to UUID's.  They recommended
> > (as Ali does) creating a UUID for each unique document.  Here a unique
> > document is probably a {doc, version, language, format} quadruplet, or
> > possibly even a superset of this.  It should uniquely specify the
> > resource.  We will need to decide where this belongs in the OMF, perhaps
> > an attribute to IDENTIFIER (ie. IDENTIFIER.UUID).  The UUID will be the
> > database key.  To make things simple, we can ship a script which generates
> > a UUID with scrollkeeper.  For systems with uuidgen, it will use that.  
> > For other systems, we may want it to request one from a server which runs
> > uuidgen (as suggested by Ali).  
> 
> It is important to point out that the UUID should not change for each
> /version/ of a doc. If you have a doc and then update it, it will have the
> _SAME_ uuid but a diff. version number, in my opinion. (by default, if you
> just use UUID you will get the "latest" doc.)
> 
> Also - different formats should have the same UUID. They are the same
> document, so they will have the same _METADATA_ (i.e. same author, same
> version, same category, etc.)

One of the reasons we want a UUID is as a database key.  So it needs to be
unique between multiple versions, formats, or languages of a document. 

We also want to guarantee that any two resources with the same UUID are in
fact the same.  I'm not sure how we will want to handle multiple
references to the same resource, but we will want to know if two resources
are identical or not. This will be particularly important if we are ever
merging multiple databases because you will have situations where both
databases will refer to the exact same resource (although they may have
different IDENTIFIERs.)  This is the reason for modifying the UUID for
each version.

I don't know if we would ever want to have duplicates of a resource in a
database. (eg. http://www.foo.org/faq.sgml and
file:/<stuff>/faq.sgml)  If this were the case, the database would need to
use a key which combined UUID with a code generated from the IDENTIFIER.

This is leaning slightly back to my original idea of basing the database
key on the URL, but Alan and DV insisted that URL was not a good key and
that we should use UUID's instead.

Dan

Re: [Scrollkeeper-devel] draft article, for review

[Dan M. <d-...@uc...>]

On Tue, 28 Nov 2000, Laszlo Kovacs wrote:

> Rich Morin wrote:
> > 
> > I have written up some recent experiences with FreeBSD documentation
> > for my column (Silicon Carny) in SunWorld (http://www.sunworld.com).
> > The column also attempts to summarize the aims of the ScrollKeeper
> > and Meta Projects.
> > 
> > If you would like to look at the draft I'm submitting, take a look at
> > http://www.cfcl.com/rdm/SC.200012.1.  Please send me email (in the
> > next few days) about any problems you notice.  I should be able to
> > squeeze in a few changes during final edit...
> > 
> > -r
> 
> This link does not seem to work. 

Are you using:

http://www.cfcl.com/rdm/SC.200012.1

and not including the period at the end?

It worked for me yesterday and again just now.

Dan

Re: [Scrollkeeper-devel] Unique identifiers

[Ali A. <ali...@au...>]

* Dan Mueth (d-...@uc...) wrote at 01:20 on 28/11/00:
> > We talked about this issue previously and we thought about an ISBN like
> > ID generated by a server available throughout the internet. This server
> > would generate unique IDs on request by document creators.
> 
> Alan and Daniel were kind enough to pop in on IRC to discuss scrollkeeper
> yesterday and the conversation quickly moved to UUID's.  They recommended
> (as Ali does) creating a UUID for each unique document.  Here a unique
> document is probably a {doc, version, language, format} quadruplet, or
> possibly even a superset of this.  It should uniquely specify the
> resource.  We will need to decide where this belongs in the OMF, perhaps
> an attribute to IDENTIFIER (ie. IDENTIFIER.UUID).  The UUID will be the
> database key.  To make things simple, we can ship a script which generates
> a UUID with scrollkeeper.  For systems with uuidgen, it will use that.  
> For other systems, we may want it to request one from a server which runs
> uuidgen (as suggested by Ali).  

It is important to point out that the UUID should not change for each
/version/ of a doc. If you have a doc and then update it, it will have the
_SAME_ uuid but a diff. version number, in my opinion. (by default, if you
just use UUID you will get the "latest" doc.)

Also - different formats should have the same UUID. They are the same
document, so they will have the same _METADATA_ (i.e. same author, same
version, same category, etc.)

Translations can have different UUIDs.

This means to get an "instance" of a doc you would need the UUID of the doc,
the version # (by default if no version is provided you get the "latest"
(greatest) version), and the format (theoretically, you could have the server
take a DocBook XML file and convert it on the fly to HTML for the user (sound
familiar?) its the same doc, just different "presentation" of the data)

> Having this UUID system will keep things simpler when we implement server
> catalogs and share docs/metadata between machines.
> 
> I'm worried that trying to generate UUID's by hand using some system as
> suggested by Rich will make things more complicated and less unique.

Rich made his suggestion because he wants to be able to "easily" refer to
UUIDs...This shouldn't be the case. You don't refer to a book's ISBN number,
you refer to its title.

> Just for completeness, I'll mention that we also want another ID number
> which we associate with each document.  We need a name - perhaps "docid"
> or "nuid" (non-universally unique id).  Multiple versions, languages,
> formats, etc. of the same document should have the same NUID.  This allows
> us to do things like look for a given document in another language, look
> for a newer (or older) version of a given document, and look for a
> different format of a document (like PS so we can print it). Perhaps we
> could call this IDENTIFIER.NUID in the OMF.

I disagree. We just need one unique ID (at least for now)...

I have a feeling if you implement multiple IDs it will confused doc authors
and just complicate matters unncessarily (if it is determined later on we need
it, then it can be added).

Regards,
Ali

Re: [Scrollkeeper-devel] draft article, for review

[Laszlo K. <las...@su...>]

Rich Morin wrote:
> 
> I have written up some recent experiences with FreeBSD documentation
> for my column (Silicon Carny) in SunWorld (http://www.sunworld.com).
> The column also attempts to summarize the aims of the ScrollKeeper
> and Meta Projects.
> 
> If you would like to look at the draft I'm submitting, take a look at
> http://www.cfcl.com/rdm/SC.200012.1.  Please send me email (in the
> next few days) about any problems you notice.  I should be able to
> squeeze in a few changes during final edit...
> 
> -r

This link does not seem to work. 

Laszlo