Re: [Scrollkeeper-devel] system similar to ScrollKeeper

[Ali A. <ALI...@au...>]

On Tue, 21 Nov 2000, Bob Stayton wrote:

> This mail describes the ODSSI (Online Documentation System
> Services Interface) system that I mentioned in an earlier
> mail. I'm sending this because ODSSI has goals similar to
> ScrollKeeper.  Let me be clear that this is *not* a
> proposal for ScrollKeeper, nor I'm not expecting any
> reviews of our system (believe me, with three years
> of experience using it, I'm well aware of its
> weaknesses 8^). This mail is simply a description of one
> actual implementation of a similar system.  I'll present
> what we did, and then describe how I think it relates to
> ScrollKeeper.  You can skip down to "How this relates
> to ScrollKeeper" if you don't want all the details.

That was a /long/ mail :)
[snip]

> But I think there were some lessons learned that do apply to
> ScrollKeeper:
> 
> 1.  A predetermined complete categorization of content is
> not useful or even possible.  Rather, an open framework
> of general categories can be established so providers
> have a place to hang their stuff.  But providers need
> the ability to add subcategories.

As far as I know - anyone can put their document in any category they 
like. But there should be a set of "recommended" categories for people to 
use.

This happens in RPM, people can put their category as anything in their 
Spec file, but usually they pick one of the "standard" ones.

Scrollkeeper will not "enforce" a strict set of categories, but I think a 
recommended list of categories should be published
 
> 2.  There's more than one way to organize and
> present the content.  Multiple different views can
> be layered on top of the content collection.  This requires
> looking at the content from the point of view of the multiple
> different users, not just the content provider.

Scrollkeeper is not responsible for 'displaying' the help. It is only for 
keeping track of the docs on your system (and allowing metadata for docs).
 
> 3.  Minimize the requirements on providers.  They should
> just need to provide standard metadata and then call a single
> post-install script already available on the system.
> That script would take various actions based on the
> metadata.  In a corollary, minimize the demands on
> system administrators, since they generally have little
> time to manage documentation.

I agree - make it as simple as possible for packagers, doc authors, 
program authors, etc.

> 4.  The system needs to handle content that has incomplete
> metadata.  Its degree of participation and integration into
> the system may be reduced, but it should not be excluded.
> Our system went even further and processed content with
> no metadata and no calls to the postinstall scripts, but
> that's because we could locate all the content under
> DocumentRoot.

I think this is okay - you just keep "empty" XML tags of the non-exiting 
metadata information

> 5.  Metadata can be versioned, so new metadata can supercede
> old without installing new versions of the doc.  You can
> even mark some doc as obsolete without actually removing it.

I believe this is possible with the OMF. 
 
> 6.  The system needs to take into account native language
> translations, and provide a system of fallback when some
> content is not available in a given language.  

This is also taken care of I believe

> 7.  Clean removal of content is very important, and harder
> than you think.  The removal scripts need to clean up
> databases, views, and search indexes.  We didn't pay enough
> attention to this item, and most of our bug reports came
> after some content was incompletely removed.

Scrollkeeper would only be responsible for cleaning up its own databases 
and various files.

> I think many of these items have already been proposed
> and discussed in the design so far.  
> 
> Conclusions
> -----------------
> Based on my experience to date, and at the risk of stepping
> on someone's toes, I suggest that ScrollKeeper
> divide its efforts into two areas:
> 
> 1.  Inventorying doc content on a system.
> 
> 2.  Presenting that content in various ways.
> 
> The inventorying effort includes defining the OMF metadata,
> creating a database to house the data, and defining an API
> for accessing the data.  This effort could also include
> methods for finding non-registered doc (for example,
> scanning RPMs for documentation sections), and fallback
> processes for discovering and filling in missing metadata
> for various file types.  The goal here is to define standard
> methods for recording and reading the metadata.
> 
> The presentation effort can go in many different directions
> to meet many different needs.  I don't think the
> ScrollKeeper project can anticipate all the uses for a
> documentation database like this, nor can it develop all
> applications that use it.  One site will want to use a
> Verity search engine to integrate doc with their other
> searchable content, while another site will want to create
> a website of documentation and index it with htdig, and a
> third will load all the information into their existing
> object-oriented database.  If there is a standard API for
> extracting information from the ScrollKeeper database, then
> those applications become much easier to do.

ScrollKeeper is in no way responsibly for the presentation of the doc. It 
can and should only provide the URI of the document to the "Help 
Browser" and based on this the browser presents the doc in any way it 
wishes. 

Also - for displaying the Index of Documents, is the browser's 
responsibility. ScrollKeeper can just pass the XML file to the help 
browser so it does its own parsing of the document (i.e. then the browser 
can add various functionality such as List only the documents in 
'Music/Players' or 'List only SGML docs')

Optionally, ScrollKeeper could pass xmlDocPtr's to the applications that 
use it. This alows /ScrollKeeper/ to do the searching and construction of 
the XML tree internally. The Help Browser folks would only need to 
"parse" the returned xmlDocPtr.

> I do think the ScrollKeeper project should develop one
> or more basic presentation applications, to the degree that it
> can.  This will make it useful "out of the box" and
> encourage content providers to add OMF data to their packages.
> It will also test the data and the API, and lead to 
> incremental refinement of both.

I disagree. This is not ScrollKeeper's responsibility. Hopefully, the KDE 
or GNOME team can provide an "example" program to show off ScrollKeeper.

In fact, I believe Sun is working on a 'Help SideBar' for Nautilus on 
this (umm, they just sent me a 500kb patch for the Help SideBar (which I 
will examine when I get home)).

> I think the ScrollKeeper project is already doing most
> of this. I would just like to see a clear distinction
> between forming the database and developing the applications
> from it.  It would make it easier to divide up the work,
> and clarify what needs to be delivered.

ScrollKeeper shouldn't have anything to do with developing the 
application (for example, which widget-set should it use? Gtk+/Qt? How 
would it display the HTML (write our own HTML parser???) use GtkHtml? (uh 
oh, KDE won't be happy?) use Mozilla? (ScrollKeeper requires 
mozilla?!?!)) - Not to mention you have to worry about cross-platform 
stuff, etc.

ScrollKeeper is only a project for indexing the docs on your system. 
It is the responsibility of the various "projects" (KDE, GNOME, etc.) to 
provide Help browsers that integrate with it nicely. Some other group, 
could create a command-line interface to it if they wish (actually - it 
might be nice to have a command-line interface to scrollkeeper...)


Regards,
Ali