Re: [Scrollkeeper-devel] Scrollkeeper backend

[Dan M. <d-...@uc...>]

On Thu, 30 Nov 2000, Bob Stayton wrote:

> > From: Ali Abdin <ali...@au...>
> 
> > > You just contradicted yourself and confused me. If the XML is a purely
> > > internal data structure, why is Nautilus reading it?
> > 
> > Yes, I did contradict myself :) 
> > 
> > Its not really an internal data structure or file format.
> > 
> > It is an external one, and instead of devising our own, we used an easily
> > parsable existing one...XML. I mean if you really want we could use a fast
> > binary database, but I doubt people would appreciate that ;)
> 
> I completely agree that XML is a good standard format to export
> data to applications like a help browser.  And of course
> the incoming data is XML (the OMF files).  
> 
> But I think there is room in the current design for a
> backend using a fast binary database to house the metadata.
> Whether it will be appreciated depends entirely on the
> applications that might use it.  Scrollkeeper could be used
> to keep track of metadata for thousands of documents.
> Those documents might be served upon request by an HTTP
> server, after a query engine processes a metadata request.
> You won't want to scan a big XML metadata document for each
> such request.

There is definitely room in the design for this.  I do not have a good
idea how well the current implementation will scale, and it is entirely
possible that we will reach a point where performance becomes an issue
which drives us to another backend.  Aside from performance, the querying
features of a database may be useful as well.  What else would we gain?

> I'm thinking that Scrollkeeper could maintain an internal
> lightweight database of the OMF data.  A database is easy
> to update when one OMF file added, changed, or deleted. And you
> can extract an XML Content List in various forms as
> queries and reports from the database.  The source of the
> data will always be the original OMF files, so you can
> still rebuild the database from scratch.
> But a database would give sk greater flexibility and
> performance in making the metadata available to different
> applications.
> 
> I'm not volunteering anyone else's copious free time
> to do this.  8^)
> If the group is interested, I'd be happy to 
> pursue this area a little further.  There are several
> available technologies that might fit the bill.

I would be interested to hear more about the options.  I think we
definitely would need it to be Free, lightweight, and in common use.

If we do decide that a db backend makes more sense, I'm not sure we would
want to move over too soon.  It may be a good idea to work entirely in XML
to keep things simple for now, until we have a solid version or two
released and a development roadmap complete.

Thanks for your offer to help :)

Dan