scrollkeeper-devel

Re: [Scrollkeeper-devel] Handling relocatable packages

[Nik C. <ni...@no...>]

On Thu, Nov 23, 2000 at 08:05:16PM -0200, Ali Abdin wrote:
> GNOME is non-relocatable...Nautilus is non-relocatable (I maybe incorrect, 
> somebody told me they were non-relocatable)

Gnome might be non-relocatable on a Linux system.  That's not the case
on a FreeBSD system.  All the world is not Linux.

[...]

> Now, I know there are probably hacks that can be done to get around this. I
> know that this may not be "theoretically" correct, etc. etc. etc. But the fact
> is there /ARE/ non-relocatable packages out there, and scrollkeeper should
> support both types of packages.

SK should not need to do anything different, once the data has been
installed.  We may need to provide tools that make it easy to rewrite an
existing OMF file if a document is being installed elsewhere (see my
description of sk-generate-omf in another message).

<snip>

> So, multiple databases does make sense in my opinion (think of it this way,
> just because YOU don't need it, or can't find a reason to use it, it doesn't
> mean that it is not useful for others)..

Exactly.  Tools, not policy.

> FAQ/Manual/HOWTO are not category types in your opinion...What I mean by
> categories is more like "subject" - i.e. "Audio/MP3 Players' or
> 'Office/Spreadsheet' or 'X11/Internet/FTP Clients' etc.

Maybe, maybe not.  SK shouldn't care.  Categories should be opaque types
to it.  SK should place no special meaning on any category.

> I am saying just a HTML file, or plain text file just LISTING the categories -
> nothing fancy...Scrollkeeper would have no knowledge of this file and will not
> interact with it in anyway.

XML, probably.  

SK would need to know about the file, so that it can do sanity checks,
and say things like

     The document you are installed lists a category that does not exist
     in the master category list.  Do you want to continue?

(or something like that).

> The scrips should NEVER EVER become "interactive". Just use whatever is in the
> category (in the OMF file).

People make typos.  SK should probably have a 'loose' and 'strict'
processing option.  It should allow people to shoot themselves in the
foot if they want to.

> > > >   application will have the responsibility for handling user queries of the
> > > >   SK TOC and Index files ]
> > > 
> > > sk should export searching functionality. 
> > 
> > How?  libsk.so?  By having a program that accepts search parameters and
> > returns XML with the results?  Some other mechanism?
> 
> I prefer a library that returns the xmlDocPtr of the file...

xmlDocPtr is specifc to libxml, isn't it?  What if I (as an application
programmer) want to use another XML library?  Since (part of) the point
behind XML is to reduce reliance on one specific parsing library, it
would be more open-ended if libsk could be used to find a document,
which can then be referred to by URL.  It's then up to the calling
application to open that URL and process it.

> > For example, on a BSD system, the contents of /usr/share/man change once
> > in a blue moon, as third party manual pages generally live under
> > /usr/local/share/man.  So the admin might want to create a Contents List
> > for /usr/share/man once, and keep it separate, because the SA knows its
> > never going to change, so it doesn't need to be included in any
> > processes SK undertakes to ensure that the Contents List is sync with
> > the filesystem.
> 
> I think this "optimization" is unncessary at this time. We have no data _AT
> ALL_ on the efficiency in scrollkeeper. We don't know if it will be slow or
> fast, so splitting up databases for this reason is unacceptable in my opinion.

I wasn't introducing it is an example of an optmisation, but as an
example of where an SA might want to have separate databases.

N
-- 
Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
Telephone line, $24.95 a month.  Software, free.  USENET transmission,
hundreds if not thousands of dollars.  Thinking before posting, priceless.
Somethings in life you can't buy.  For everything else, there's MasterCard.
  -- Graham Reed, in the Scary Devil Monastery

Re: [Scrollkeeper-devel] Handling relocatable packages

[Ali A. <ali...@au...>]

* Nik Clayton (ni...@no...) wrote at 03:49 on 26/11/00:
> On Thu, Nov 23, 2000 at 08:05:16PM -0200, Ali Abdin wrote:
> > GNOME is non-relocatable...Nautilus is non-relocatable (I maybe incorrect, 
> > somebody told me they were non-relocatable)
> 
> Gnome might be non-relocatable on a Linux system.  That's not the case
> on a FreeBSD system.  All the world is not Linux.
> 
> [...]

So we should support FreeBSD and not Linux? (this argument works both ways). I
am just saying we should support both relocatable and non-relocatable packages
(you asked for an example of non-reloctabale package, and I gave you one
(GNOME on linux)).
 
> > Now, I know there are probably hacks that can be done to get around this. I
> > know that this may not be "theoretically" correct, etc. etc. etc. But the fact
> > is there /ARE/ non-relocatable packages out there, and scrollkeeper should
> > support both types of packages.
> 
> SK should not need to do anything different, once the data has been
> installed.  We may need to provide tools that make it easy to rewrite an
> existing OMF file if a document is being installed elsewhere (see my
> description of sk-generate-omf in another message).

[snip]
 
> > I am saying just a HTML file, or plain text file just LISTING the categories -
> > nothing fancy...Scrollkeeper would have no knowledge of this file and will not
> > interact with it in anyway.
> 
> XML, probably.  
> 
> SK would need to know about the file, so that it can do sanity checks,
> and say things like
> 
>      The document you are installed lists a category that does not exist
>      in the master category list.  Do you want to continue?
> 
> (or something like that).

Yeah...although I'd still like a HTML and/or plain text version (just to avoid
the visual clutter of the tags). Wait, actually Mozilla now can decently view
XML files so that may not be a problem ;) but whatever.

> > The scrips should NEVER EVER become "interactive". Just use whatever is in the
> > category (in the OMF file).
> 
> People make typos.  SK should probably have a 'loose' and 'strict'
> processing option.  It should allow people to shoot themselves in the
> foot if they want to.

Well - by default, I recommend non-interative prompts..
 
> > > > >   application will have the responsibility for handling user queries of the
> > > > >   SK TOC and Index files ]
> > > > 
> > > > sk should export searching functionality. 
> > > 
> > > How?  libsk.so?  By having a program that accepts search parameters and
> > > returns XML with the results?  Some other mechanism?
> > 
> > I prefer a library that returns the xmlDocPtr of the file...
> 
> xmlDocPtr is specifc to libxml, isn't it?  What if I (as an application
> programmer) want to use another XML library?  Since (part of) the point
> behind XML is to reduce reliance on one specific parsing library, it
> would be more open-ended if libsk could be used to find a document,
> which can then be referred to by URL.  It's then up to the calling
> application to open that URL and process it.

ScrollKeeper /has/ to pick an XML library to rely on for parsing stuff. I was
just suggesting "publicizing" this in the API...Anyway, you /could/ have
libxml-specific stuff trapped in #ifdef's and just provide "other" people with
the FILE descriptor to do their own parsing.

Or, we can make libsk.so libxml-specific totally and then force people who
wish to use something else to use the scripts and/or command-line tools.

Note: to compile (and run) scrollkeeper you will need to have libxml
installed.

Re: [Scrollkeeper-devel] Handling relocatable packages

[Nik C. <ni...@no...>]

On Mon, Nov 27, 2000 at 03:57:15AM -0200, Ali Abdin wrote:
> * Nik Clayton (ni...@no...) wrote at 03:49 on 26/11/00:
> > > > How?  libsk.so?  By having a program that accepts search parameters and
> > > > returns XML with the results?  Some other mechanism?
> > > 
> > > I prefer a library that returns the xmlDocPtr of the file...
> > 
> > xmlDocPtr is specifc to libxml, isn't it?  What if I (as an application
> > programmer) want to use another XML library?  Since (part of) the point
> > behind XML is to reduce reliance on one specific parsing library, it
> > would be more open-ended if libsk could be used to find a document,
> > which can then be referred to by URL.  It's then up to the calling
> > application to open that URL and process it.
> 
> ScrollKeeper /has/ to pick an XML library to rely on for parsing stuff. 

That's fine.  It just shouldn't force any other application that wants
to use the results from SK to use the same library.  The library might
not even be available on some platforms, or languages.  Suppose I'm
writing a wrapper around SK in Perl.  What do I do with a C pointer in
Perl?

SK is going to be used to identify documents in the catalogue, so it seems
natural to me that the result of any queries should be a URL (or list of
URLs) that point to the matching document(s).  Let the calling
application decide what it wants to do with this URL.

> I was
> just suggesting "publicizing" this in the API...Anyway, you /could/ have
> libxml-specific stuff trapped in #ifdef's and just provide "other" people with
> the FILE descriptor to do their own parsing.

Still too C specific. 

> Note: to compile (and run) scrollkeeper you will need to have libxml
> installed.

Maybe to compile it.  But you can always compile it static, and ship the
binary to a system that doesn't have libxml installed.

N
-- 
Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
Telephone line, $24.95 a month.  Software, free.  USENET transmission,
hundreds if not thousands of dollars.  Thinking before posting, priceless.
Somethings in life you can't buy.  For everything else, there's MasterCard.
  -- Graham Reed, in the Scary Devil Monastery

[Scrollkeeper-devel] Multiple Databases

[Laszlo K. <las...@su...>]

Hi All,

I will try to rip the big email to several smaller emails. Here is one,
the issue of multiple databases. Let's ignore whatever we have designed
and implemented for now and try to gather together why do we think this
would be good.

Some ideas I write up here come from the architecture of Solaris/CDE
Help. Some of the ideas below are very different from the current
design:-)

The idea of accessing various databases across the internet looked very
interesting for everybody. I think this is a good idea.

Whereever a remote database is, there has to be a Scrollkeeper around. I
would expect that remote database to be up-to-date, I don't think my
local Scrollkeeper should be responsible for this, it should take for
granted whatever is offered by the remote database. In this case
database would mean extracted TOC, Index, etc I would expect all of this
to come through the remote connection. How does that sound?

This will obviously need a script/binary/config file that will tell
Scrollkeeper that which remote databases should be considered.

The model would be the same for locally installed docs. More databases
on the local system and a script/binary/config file tell which ones to
take into account.

Now this is where it becomes rough, I suggest to change the current
design a lot:-)

1. We install the OMF files per package in the package directory (like
/usr/share/pkgname/omf).
2. We say Scrollkeeper is needed to build the package.
3. We generate partial content list file for this package at build time
(obviously it has to be very different from the one we have now). Same
for TOC, Index etc.
4. At install time we create symbolic link of this package to a special
directory (/etc/scrollkeeper or similar). These directories can be
specified through a search path (more than one dir).
5. We provide an interface that merges the partial lists into a main
list and stores it in $/home/.scrollkeeper. We use timestamps to monitor
the partial database changes (only the files have to be checked).

This is it. This would have some advantages, but many disadvantages
also. So please go ahead and point out the problems.

As about merging, I don't see that a big problem.

Laszlo

[Scrollkeeper-devel] Category Trees

[Laszlo K. <las...@su...>]

Hi,

here is another issue from our very long email. 

How do we deal with category trees? I see category trees as generic
empty content lists.

These are the possibilities we discussed:

1. Enforce a category tree and ignore all docs that don't fit into it.
2. Enforce a tree and put all docs that don't fit into an Other section
of the Content List.
3. Supply a tree (installed) and OMF files come with new categories,
extend the Categories in the Content List with them (don't modify the
category tree).
4. Recommend a set of categories on the project web site, but don't
supply any tree. Build the content list(s) from whatever comes in the
OMF files.
5. Don't recommend any categorization, but keep support in the OMF files
to specify categories. Extend the Content List with whatever comes in
the OMF files.

1. and 5. are there for completeness only. I suppose nobody thinks they
would be a way to go.

So what do people think? Which is the best way to do it? Any other
ideas?

Laszlo

Re: [Scrollkeeper-devel] Category Trees

[Dan M. <d-...@uc...>]

On Fri, 24 Nov 2000, Laszlo Kovacs wrote:

> How do we deal with category trees? I see category trees as generic
> empty content lists.
> 
> These are the possibilities we discussed:
> 
> 1. Enforce a category tree and ignore all docs that don't fit into it.
> 2. Enforce a tree and put all docs that don't fit into an Other section
> of the Content List.
> 3. Supply a tree (installed) and OMF files come with new categories,
> extend the Categories in the Content List with them (don't modify the
> category tree).
> 4. Recommend a set of categories on the project web site, but don't
> supply any tree. Build the content list(s) from whatever comes in the
> OMF files.
> 5. Don't recommend any categorization, but keep support in the OMF files
> to specify categories. Extend the Content List with whatever comes in
> the OMF files.
> 
> 1. and 5. are there for completeness only. I suppose nobody thinks they
> would be a way to go.
> 
> So what do people think? Which is the best way to do it? Any other
> ideas?

One more:
6) Support more than one of the above and allow the user/SA to decide
which one method is employed in /etc/scrollkeeper.conf or
$HOME/.scrollkeeperrc.

One other issue which should be discussed for options #1-#3 - What do we
do with empty parts of the tree?  Do we show branches that don't have
documents in them?  Or are these branches only created as they get
populated?   I think the assumption right now is that empty branches
exist and are passed to the browser.  (The browser could still potentially
snip them out.)

And another issue: If we have a categories tree, should we allow the user
to specify in the conf file which tree to use?  Presumably one may want to
use an alternate tree, such as a subset of the recommended tree.  For
example, a user might copy the default tree into
$HOME/.scrollkeeper/categories.xml and then chop out the man and info
page sections.

Dan

Re: [Scrollkeeper-devel] Category Trees

[Laszlo K. <las...@su...>]

> One other issue which should be discussed for options #1-#3 - What do we
> do with empty parts of the tree?  Do we show branches that don't have
> documents in them?  Or are these branches only created as they get
> populated?   I think the assumption right now is that empty branches
> exist and are passed to the browser.  (The browser could still potentially
> snip them out.)

I don't think this is important from architectural point of view at the
moment. This is why we chopped the big email to small emails and now we
are growing it again:-)

Laszlo

Re: [Scrollkeeper-devel] Category Trees

[Ali A. <ali...@au...>]

* Laszlo Kovacs (las...@su...) wrote at 17:14 on 27/11/00:
> > One other issue which should be discussed for options #1-#3 - What do we
> > do with empty parts of the tree?  Do we show branches that don't have
> > documents in them?  Or are these branches only created as they get
> > populated?   I think the assumption right now is that empty branches
> > exist and are passed to the browser.  (The browser could still potentially
> > snip them out.)
> 
> I don't think this is important from architectural point of view at the
> moment. This is why we chopped the big email to small emails and now we
> are growing it again:-)

I think the "core hackers" (you and Dan) should be more focused to on taking
decisions (even if they are bad ones) rather than having "endless"
discussions. Almost every point brought up has multiple approaches and each
point has pros and cons.

Maybe its just poor communication, but I haven't seen any decisions made (just
discussed to death).

Maybe that'll help the long emails ;)

Regards,
Ali

Re: [Scrollkeeper-devel] Category Trees

[Rich M. <rd...@cf...>]

The "Tree of Knowledge Fallacy" has to do with the mistaken belief
that you can put all of knowledge into a single hierarchy.  Anyone
who has ever tried to sort miscellaneous items has run into it:
Lessee now; I seem to have a lot of screws here.  Do I put all of
the Phillips-head screws together, sort by length, or what?

The problem clearly manifests itself in the area of Open Source
software packages.  With FileWatcher tracking some 10K packages,
it is totally unreasonable to believe that any categorization
scheme will fit all packages for all needs.  Politics aside
(my tree's better than your tree...), different sorting schemes
will always be better or poorer for different searching needs.

So, I suggest that we concentrate on things we _can_ control:

   *  unique identifiers

        Each package (and each variant and version of same)
        should have a unique identifier, so that we can be
        clear and unambiguous in referring to it.  This is
        really the same issue as the database folks have...

   *  characteristics

        A package can have any number of characteristics (e.g.,
        keywords) that someone might use to look it up.  One
        form of keyword is the package's position in a given
        hierarchy (e.g., freebsd-ports/devel/pmake).

By explicitly allowing for the presence of multiple category
trees, this method is able to incorporate (or even dynamically
add) any number of categorization methods at the same time.

-r

-- 
--
Rich Morin:          rd...@cf..., +1 650-873-7841, http://www.cfcl.com/rdm
Prime Time Freeware: in...@pt..., +1 408-433-9662, http://www.ptf.com
MacPerl: http://www.macperl.com, http://www.ptf.com/ptf/products/MPPE
MkLinux: http://www.mklinux.org, http://www.ptf.com/ptf/products/MKLP

Re: [Scrollkeeper-devel] Category Trees

[Ali A. <ali...@au...>]

* Dan Mueth (d-...@uc...) wrote at 01:48 on 25/11/00:
> 
> On Fri, 24 Nov 2000, Laszlo Kovacs wrote:
> 
> > How do we deal with category trees? I see category trees as generic
> > empty content lists.
> > 
> > These are the possibilities we discussed:
> > 
> > 1. Enforce a category tree and ignore all docs that don't fit into it.
> > 2. Enforce a tree and put all docs that don't fit into an Other section
> > of the Content List.
> > 3. Supply a tree (installed) and OMF files come with new categories,
> > extend the Categories in the Content List with them (don't modify the
> > category tree).
> > 4. Recommend a set of categories on the project web site, but don't
> > supply any tree. Build the content list(s) from whatever comes in the
> > OMF files.
> > 5. Don't recommend any categorization, but keep support in the OMF files
> > to specify categories. Extend the Content List with whatever comes in
> > the OMF files.
> > 
> > 1. and 5. are there for completeness only. I suppose nobody thinks they
> > would be a way to go.
> > 
> > So what do people think? Which is the best way to do it? Any other
> > ideas?
> 
> One more:
> 6) Support more than one of the above and allow the user/SA to decide
> which one method is employed in /etc/scrollkeeper.conf or
> $HOME/.scrollkeeperrc.
> 
> One other issue which should be discussed for options #1-#3 - What do we
> do with empty parts of the tree?  Do we show branches that don't have
> documents in them?  Or are these branches only created as they get
> populated?   I think the assumption right now is that empty branches
> exist and are passed to the browser.  (The browser could still potentially
> snip them out.)

I disagree. I do not think it is worth the complexity to actually let the user
choose.

You will also confuse system administrators, and desktop users who are not
sure of which setting to you use.

I prefer method 4. If somebody has a reason why we should NOT use number 4,
please say why? ;)
 
> And another issue: If we have a categories tree, should we allow the user
> to specify in the conf file which tree to use?  Presumably one may want to
> use an alternate tree, such as a subset of the recommended tree.  For
> example, a user might copy the default tree into
> $HOME/.scrollkeeper/categories.xml and then chop out the man and info
> page sections.

Well, I vote no category tree :P

Re: [Scrollkeeper-devel] Category Trees

[Dan M. <d-...@uc...>]

On Sat, 25 Nov 2000, Ali Abdin wrote:

> * Dan Mueth (d-...@uc...) wrote at 01:48 on 25/11/00:
> > 
> > On Fri, 24 Nov 2000, Laszlo Kovacs wrote:
> > 
> > > How do we deal with category trees? I see category trees as generic
> > > empty content lists.
> > > 
> > > These are the possibilities we discussed:
> > > 
> > > 1. Enforce a category tree and ignore all docs that don't fit into it.
> > > 2. Enforce a tree and put all docs that don't fit into an Other section
> > > of the Content List.
> > > 3. Supply a tree (installed) and OMF files come with new categories,
> > > extend the Categories in the Content List with them (don't modify the
> > > category tree).
> > > 4. Recommend a set of categories on the project web site, but don't
> > > supply any tree. Build the content list(s) from whatever comes in the
> > > OMF files.
> > > 5. Don't recommend any categorization, but keep support in the OMF files
> > > to specify categories. Extend the Content List with whatever comes in
> > > the OMF files.
> > > 
> > > 1. and 5. are there for completeness only. I suppose nobody thinks they
> > > would be a way to go.
> > > 
> > > So what do people think? Which is the best way to do it? Any other
> > > ideas?
> > 
> > One more:
> > 6) Support more than one of the above and allow the user/SA to decide
> > which one method is employed in /etc/scrollkeeper.conf or
> > $HOME/.scrollkeeperrc.
> > 
> > One other issue which should be discussed for options #1-#3 - What do we
> > do with empty parts of the tree?  Do we show branches that don't have
> > documents in them?  Or are these branches only created as they get
> > populated?   I think the assumption right now is that empty branches
> > exist and are passed to the browser.  (The browser could still potentially
> > snip them out.)
> 
> I disagree. I do not think it is worth the complexity to actually let the user
> choose.
> 
> You will also confuse system administrators, and desktop users who are not
> sure of which setting to you use.
> 
> I prefer method 4. If somebody has a reason why we should NOT use number 4,
> please say why? ;)

My concerns were:

1) We want to have some configurability on the part of the SA or
user.  This basically means removing branches or adding branches.  For #4,
adding branches is trivial.  We could potentially provide another
mechanism for removing branches in place of specifying the tree, so #4
could still work here.

2) Things becoming a mess.  If we use #4, we would still need to provide a
really good tree on the web page and strongly encourage people to use it
unless their document really doesn't fit into the existing scheme.  Even
in this case, they should suggest that the tree be extended in addition to
creating their own branch in their OMF file.  If we do provide a really
good tree and most people follow it, then we should be okay.  We will need
to track down people who have blatently duplicate existing sections in
other positions than the existing sections.  It only takes a few people
doing this to produce a very confusing and inconsistant interface for the
user. I really don't want to see 9 different places for the GIMP in the
tree ;)

3) Typos and variations on translations.  Unless we do #2, we will always
get spurious duplicated branches from these.  

Given this, #4 seems like a reasonable solution. The people I've spoken
with about this so far seem pretty much split between #4 and #2.  #2
guarantees that concerns (2) and (3) above do not become real problems.  
Of course it has a couple of its own problems to contribute such as the
need to keep your contents list categories up-to-date and make the central
list well-managed and respected (we have to do that anyway though or even
with #4 we will be screwed.)

The difference between this and #3 is that #3 will have empty branches.  
I haven't decided whether I think this is a good thing or a bad thing.  
If the tree on the web page is huge and has large sections which are
frequently empty, it may be best not to have them. 

Does anybody have an opinion on this?

Dan

Re: [Scrollkeeper-devel] Category Trees

[Laszlo K. <las...@su...>]

[SNIP]
> 3) Typos and variations on translations.  Unless we do #2, we will always
> get spurious duplicated branches from these.
We would probably have to create/recommend categories for every
available locale consistently. I mean we should translate the categories
to every locale and promote these categories for the appropriate
locales.

> The difference between this and #3 is that #3 will have empty branches.
> I haven't decided whether I think this is a good thing or a bad thing.
> If the tree on the web page is huge and has large sections which are
> frequently empty, it may be best not to have them.
In case we go with a predefined category tree we will have empty
branches. Ignoring them in an XML environment should be trivial and left
to the browser.

Laszlo

Re: [Scrollkeeper-devel] Category Trees

[Ali A. <ali...@au...>]

* Dan Mueth (d-...@uc...) wrote at 03:43 on 27/11/00:
> 
> On Sat, 25 Nov 2000, Ali Abdin wrote:
> 
> > * Dan Mueth (d-...@uc...) wrote at 01:48 on 25/11/00:
> > > 
> > > On Fri, 24 Nov 2000, Laszlo Kovacs wrote:
> > > 
> > > > How do we deal with category trees? I see category trees as generic
> > > > empty content lists.
> > > > 
> > > > These are the possibilities we discussed:
> > > > 
> > > > 1. Enforce a category tree and ignore all docs that don't fit into it.
> > > > 2. Enforce a tree and put all docs that don't fit into an Other section
> > > > of the Content List.
> > > > 3. Supply a tree (installed) and OMF files come with new categories,
> > > > extend the Categories in the Content List with them (don't modify the
> > > > category tree).
> > > > 4. Recommend a set of categories on the project web site, but don't
> > > > supply any tree. Build the content list(s) from whatever comes in the
> > > > OMF files.
> > > > 5. Don't recommend any categorization, but keep support in the OMF files
> > > > to specify categories. Extend the Content List with whatever comes in
> > > > the OMF files.
> > > > 
> > > > 1. and 5. are there for completeness only. I suppose nobody thinks they
> > > > would be a way to go.
> > > > 
> > > > So what do people think? Which is the best way to do it? Any other
> > > > ideas?
> > > 
> > > One more:
> > > 6) Support more than one of the above and allow the user/SA to decide
> > > which one method is employed in /etc/scrollkeeper.conf or
> > > $HOME/.scrollkeeperrc.
> > > 
> > > One other issue which should be discussed for options #1-#3 - What do we
> > > do with empty parts of the tree?  Do we show branches that don't have
> > > documents in them?  Or are these branches only created as they get
> > > populated?   I think the assumption right now is that empty branches
> > > exist and are passed to the browser.  (The browser could still potentially
> > > snip them out.)
> > 
> > I disagree. I do not think it is worth the complexity to actually let the user
> > choose.
> > 
> > You will also confuse system administrators, and desktop users who are not
> > sure of which setting to you use.
> > 
> > I prefer method 4. If somebody has a reason why we should NOT use number 4,
> > please say why? ;)
> 
> My concerns were:
> 
> 1) We want to have some configurability on the part of the SA or
> user.  This basically means removing branches or adding branches.  For #4,
> adding branches is trivial.  We could potentially provide another
> mechanism for removing branches in place of specifying the tree, so #4
> could still work here.

You don't need to "remove" or "add" branches within scrollkeeper. It will all
be queried at runtime.
 
> 2) Things becoming a mess.  If we use #4, we would still need to provide a
> really good tree on the web page and strongly encourage people to use it
> unless their document really doesn't fit into the existing scheme.  Even
> in this case, they should suggest that the tree be extended in addition to
> creating their own branch in their OMF file.  If we do provide a really
> good tree and most people follow it, then we should be okay.  We will need
> to track down people who have blatently duplicate existing sections in
> other positions than the existing sections.  It only takes a few people
> doing this to produce a very confusing and inconsistant interface for the
> user. I really don't want to see 9 different places for the GIMP in the
> tree ;)

I think that if people "do their own thing" that proves that the Category tree
provided is somehow ineffective (confusing? too many duplicates? etc.) 

I seriously doubt people will just "do their own thing" because they just feel
like it...Actually, on second thoughts, I recommend we DO distribute a
Category list and scrollkeeper-install should provide a "warning" if it is not
ine one of the "suggested" list of categories! (otherwise people will be too
lazy to go to the website to look it up!)

> 3) Typos and variations on translations.  Unless we do #2, we will always
> get spurious duplicated branches from these.  

This is interesting (translations of categories). My vote: don't translate the
category. (this is the same as saying don't translate the XML tags)
 
> Given this, #4 seems like a reasonable solution. The people I've spoken
> with about this so far seem pretty much split between #4 and #2.  #2
> guarantees that concerns (2) and (3) above do not become real problems.  
> Of course it has a couple of its own problems to contribute such as the
> need to keep your contents list categories up-to-date and make the central
> list well-managed and respected (we have to do that anyway though or even
> with #4 we will be screwed.)
> 
> The difference between this and #3 is that #3 will have empty branches.  
> I haven't decided whether I think this is a good thing or a bad thing.  
> If the tree on the web page is huge and has large sections which are
> frequently empty, it may be best not to have them. 
> 
> Does anybody have an opinion on this?

Empty branches suck. They are wasteful. 

Re: [Scrollkeeper-devel] Category Trees

[Ali A. <ali...@au...>]

* Rich Morin (rd...@cf...) wrote at 01:50 on 25/11/00:
> The "Tree of Knowledge Fallacy" has to do with the mistaken belief
> that you can put all of knowledge into a single hierarchy.  Anyone
> who has ever tried to sort miscellaneous items has run into it:
> Lessee now; I seem to have a lot of screws here.  Do I put all of
> the Phillips-head screws together, sort by length, or what?
> 
> The problem clearly manifests itself in the area of Open Source
> software packages.  With FileWatcher tracking some 10K packages,
> it is totally unreasonable to believe that any categorization
> scheme will fit all packages for all needs.  Politics aside
> (my tree's better than your tree...), different sorting schemes
> will always be better or poorer for different searching needs.

Heh, I think you're right. I think Help files SHOULD be able to have multiple
categories! This really does make sense (although potentially confusing for
the user). 

Should the OMF files though have the ability to have "endless" amounts of
categories for a file, or should we impose a limit (i.e. can specify a max of
3 categories, in order from "most appropriate" to "less appropriate" (e.g. for
Nautilus we would have 1) Desktop/File Manager 2) Internet/Web Browser 3)
Audio/MP3 Player (it does play MP3s!!))
 
> So, I suggest that we concentrate on things we _can_ control:
> 
>    *  unique identifiers
> 
>         Each package (and each variant and version of same)
>         should have a unique identifier, so that we can be
>         clear and unambiguous in referring to it.  This is
>         really the same issue as the database folks have...

Yeah, I agree - I disagree with your other mail specifying how to do it. If we
want truly unique id's then we just use the 'uuid-gen' utility found on most
machines. 

>    *  characteristics
> 
>         A package can have any number of characteristics (e.g.,
>         keywords) that someone might use to look it up.  One
>         form of keyword is the package's position in a given
>         hierarchy (e.g., freebsd-ports/devel/pmake).

I'm not sure about this. I don't think we should just add in any features that
might possibly be useful somehow, so we may as well throw it in now. The OMF
file should already have all the metadata necessary for indexing documents - I
don't see why we need a generic "characteristics" or "keywords" section

> By explicitly allowing for the presence of multiple category
> trees, this method is able to incorporate (or even dynamically
> add) any number of categorization methods at the same time.

I don't think we should have a strict "category" tree. We can just allow
people to give each doc multiple categories, but ScrollKeeper should not
enforce it!

Regards,
Ali

Re: [Scrollkeeper-devel] Category Trees

[Rich M. <rd...@cf...>]

At 1:58 AM -0200 11/25/00, Ali Abdin wrote:
>If we want truly unique id's then we just use the 'uuid-gen'
>utility found on most machines.

Its not on FreeBSD (or even mentioned in /usr/ports/INDEX),
though I'm sure it could be added.

My opinion, however, is that there is no reason to use ugly, long,
and easily forgotten keys, in pursuit of mechanical perfection,
when better ones are available.

More to the point, it is very handy for folks discussing a given
package to be able to refer to it clearly and unambiguously.  I
think my proposal serves here, but that the uuid-gen keys do not.

-t
-- 
--
Rich Morin:          rd...@cf..., +1 650-873-7841, http://www.cfcl.com/rdm
Prime Time Freeware: in...@pt..., +1 408-433-9662, http://www.ptf.com
MacPerl: http://www.macperl.com, http://www.ptf.com/ptf/products/MPPE
MkLinux: http://www.mklinux.org, http://www.ptf.com/ptf/products/MKLP

Re: [Scrollkeeper-devel] Category Trees

[Laszlo K. <las...@su...>]

>    *  unique identifiers
> 
>         Each package (and each variant and version of same)
>         should have a unique identifier, so that we can be
>         clear and unambiguous in referring to it.  This is
>         really the same issue as the database folks have...
I think this is a separate issue I might start up a new thread about it.

Laszlo

[Scrollkeeper-devel] Unique identifiers

[Laszlo K. <las...@su...>]

Hi All,

During the category tree discussion the issue of unique identifiers for
docs was raised. As we want to keep the various discssions as focused as
possible I decided to start this thread.

Here are the main issues raised:

(Rich Morin)
	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4716298/
        Each package (and each variant and version of same)
        should have a unique identifier, so that we can be
        clear and unambiguous in referring to it.  This is
        really the same issue as the database folks have... 

(Rich Morin)
	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4716421/
	Use mnemonic identifiers such as package name and principal developer:
	e.g:	perl-lw

(Ali Abdin)
	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4718805/
	Use uuid-gen.

(Rich Morin)
	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4719633/
	uuid-gen is not available on every platform (notably FreeBSD).

We talked about this issue previously and we thought about an ISBN like
ID generated by a server available throughout the internet. This server
would generate unique IDs on request by document creators.

Anybody has any other ideas?

Laszlo

Re: [Scrollkeeper-devel] Unique identifiers

[Dan M. <d-...@uc...>]

On Mon, 27 Nov 2000, Laszlo Kovacs wrote:

> Hi All,
> 
> During the category tree discussion the issue of unique identifiers for
> docs was raised. As we want to keep the various discssions as focused as
> possible I decided to start this thread.
> 
> Here are the main issues raised:
> 
> (Rich Morin)
> 	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4716298/
>         Each package (and each variant and version of same)
>         should have a unique identifier, so that we can be
>         clear and unambiguous in referring to it.  This is
>         really the same issue as the database folks have... 
> 
> (Rich Morin)
> 	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4716421/
> 	Use mnemonic identifiers such as package name and principal developer:
> 	e.g:	perl-lw
> 
> (Ali Abdin)
> 	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4718805/
> 	Use uuid-gen.
> 
> (Rich Morin)
> 	http://www.geocrawler.com/lists/3/SourceForge/6429/0/4719633/
> 	uuid-gen is not available on every platform (notably FreeBSD).
> 
> We talked about this issue previously and we thought about an ISBN like
> ID generated by a server available throughout the internet. This server
> would generate unique IDs on request by document creators.

Alan and Daniel were kind enough to pop in on IRC to discuss scrollkeeper
yesterday and the conversation quickly moved to UUID's.  They recommended
(as Ali does) creating a UUID for each unique document.  Here a unique
document is probably a {doc, version, language, format} quadruplet, or
possibly even a superset of this.  It should uniquely specify the
resource.  We will need to decide where this belongs in the OMF, perhaps
an attribute to IDENTIFIER (ie. IDENTIFIER.UUID).  The UUID will be the
database key.  To make things simple, we can ship a script which generates
a UUID with scrollkeeper.  For systems with uuidgen, it will use that.  
For other systems, we may want it to request one from a server which runs
uuidgen (as suggested by Ali).  

Having this UUID system will keep things simpler when we implement server
catalogs and share docs/metadata between machines.

I'm worried that trying to generate UUID's by hand using some system as
suggested by Rich will make things more complicated and less unique.

Just for completeness, I'll mention that we also want another ID number
which we associate with each document.  We need a name - perhaps "docid"
or "nuid" (non-universally unique id).  Multiple versions, languages,
formats, etc. of the same document should have the same NUID.  This allows
us to do things like look for a given document in another language, look
for a newer (or older) version of a given document, and look for a
different format of a document (like PS so we can print it). Perhaps we
could call this IDENTIFIER.NUID in the OMF.

Dan

Re: [Scrollkeeper-devel] Unique identifiers

[Ali A. <ali...@au...>]

* Dan Mueth (d-...@uc...) wrote at 01:20 on 28/11/00:
> > We talked about this issue previously and we thought about an ISBN like
> > ID generated by a server available throughout the internet. This server
> > would generate unique IDs on request by document creators.
> 
> Alan and Daniel were kind enough to pop in on IRC to discuss scrollkeeper
> yesterday and the conversation quickly moved to UUID's.  They recommended
> (as Ali does) creating a UUID for each unique document.  Here a unique
> document is probably a {doc, version, language, format} quadruplet, or
> possibly even a superset of this.  It should uniquely specify the
> resource.  We will need to decide where this belongs in the OMF, perhaps
> an attribute to IDENTIFIER (ie. IDENTIFIER.UUID).  The UUID will be the
> database key.  To make things simple, we can ship a script which generates
> a UUID with scrollkeeper.  For systems with uuidgen, it will use that.  
> For other systems, we may want it to request one from a server which runs
> uuidgen (as suggested by Ali).  

It is important to point out that the UUID should not change for each
/version/ of a doc. If you have a doc and then update it, it will have the
_SAME_ uuid but a diff. version number, in my opinion. (by default, if you
just use UUID you will get the "latest" doc.)

Also - different formats should have the same UUID. They are the same
document, so they will have the same _METADATA_ (i.e. same author, same
version, same category, etc.)

Translations can have different UUIDs.

This means to get an "instance" of a doc you would need the UUID of the doc,
the version # (by default if no version is provided you get the "latest"
(greatest) version), and the format (theoretically, you could have the server
take a DocBook XML file and convert it on the fly to HTML for the user (sound
familiar?) its the same doc, just different "presentation" of the data)

> Having this UUID system will keep things simpler when we implement server
> catalogs and share docs/metadata between machines.
> 
> I'm worried that trying to generate UUID's by hand using some system as
> suggested by Rich will make things more complicated and less unique.

Rich made his suggestion because he wants to be able to "easily" refer to
UUIDs...This shouldn't be the case. You don't refer to a book's ISBN number,
you refer to its title.

> Just for completeness, I'll mention that we also want another ID number
> which we associate with each document.  We need a name - perhaps "docid"
> or "nuid" (non-universally unique id).  Multiple versions, languages,
> formats, etc. of the same document should have the same NUID.  This allows
> us to do things like look for a given document in another language, look
> for a newer (or older) version of a given document, and look for a
> different format of a document (like PS so we can print it). Perhaps we
> could call this IDENTIFIER.NUID in the OMF.

I disagree. We just need one unique ID (at least for now)...

I have a feeling if you implement multiple IDs it will confused doc authors
and just complicate matters unncessarily (if it is determined later on we need
it, then it can be added).

Regards,
Ali

Re: [Scrollkeeper-devel] Unique identifiers

[Dan M. <d-...@uc...>]

On Tue, 28 Nov 2000, Ali Abdin wrote:

> * Dan Mueth (d-...@uc...) wrote at 01:20 on 28/11/00:
> > > We talked about this issue previously and we thought about an ISBN like
> > > ID generated by a server available throughout the internet. This server
> > > would generate unique IDs on request by document creators.
> > 
> > Alan and Daniel were kind enough to pop in on IRC to discuss scrollkeeper
> > yesterday and the conversation quickly moved to UUID's.  They recommended
> > (as Ali does) creating a UUID for each unique document.  Here a unique
> > document is probably a {doc, version, language, format} quadruplet, or
> > possibly even a superset of this.  It should uniquely specify the
> > resource.  We will need to decide where this belongs in the OMF, perhaps
> > an attribute to IDENTIFIER (ie. IDENTIFIER.UUID).  The UUID will be the
> > database key.  To make things simple, we can ship a script which generates
> > a UUID with scrollkeeper.  For systems with uuidgen, it will use that.  
> > For other systems, we may want it to request one from a server which runs
> > uuidgen (as suggested by Ali).  
> 
> It is important to point out that the UUID should not change for each
> /version/ of a doc. If you have a doc and then update it, it will have the
> _SAME_ uuid but a diff. version number, in my opinion. (by default, if you
> just use UUID you will get the "latest" doc.)
> 
> Also - different formats should have the same UUID. They are the same
> document, so they will have the same _METADATA_ (i.e. same author, same
> version, same category, etc.)

One of the reasons we want a UUID is as a database key.  So it needs to be
unique between multiple versions, formats, or languages of a document. 

We also want to guarantee that any two resources with the same UUID are in
fact the same.  I'm not sure how we will want to handle multiple
references to the same resource, but we will want to know if two resources
are identical or not. This will be particularly important if we are ever
merging multiple databases because you will have situations where both
databases will refer to the exact same resource (although they may have
different IDENTIFIERs.)  This is the reason for modifying the UUID for
each version.

I don't know if we would ever want to have duplicates of a resource in a
database. (eg. http://www.foo.org/faq.sgml and
file:/<stuff>/faq.sgml)  If this were the case, the database would need to
use a key which combined UUID with a code generated from the IDENTIFIER.

This is leaning slightly back to my original idea of basing the database
key on the URL, but Alan and DV insisted that URL was not a good key and
that we should use UUID's instead.

Dan

Re: [Scrollkeeper-devel] Unique identifiers

[Ali A. <ali...@au...>]

* Dan Mueth (d-...@uc...) wrote at 23:54 on 28/11/00:
> 
> On Tue, 28 Nov 2000, Ali Abdin wrote:
> 
> > * Dan Mueth (d-...@uc...) wrote at 01:20 on 28/11/00:
> > > > We talked about this issue previously and we thought about an ISBN like
> > > > ID generated by a server available throughout the internet. This server
> > > > would generate unique IDs on request by document creators.
> > > 
> > > Alan and Daniel were kind enough to pop in on IRC to discuss scrollkeeper
> > > yesterday and the conversation quickly moved to UUID's.  They recommended
> > > (as Ali does) creating a UUID for each unique document.  Here a unique
> > > document is probably a {doc, version, language, format} quadruplet, or
> > > possibly even a superset of this.  It should uniquely specify the
> > > resource.  We will need to decide where this belongs in the OMF, perhaps
> > > an attribute to IDENTIFIER (ie. IDENTIFIER.UUID).  The UUID will be the
> > > database key.  To make things simple, we can ship a script which generates
> > > a UUID with scrollkeeper.  For systems with uuidgen, it will use that.  
> > > For other systems, we may want it to request one from a server which runs
> > > uuidgen (as suggested by Ali).  
> > 
> > It is important to point out that the UUID should not change for each
> > /version/ of a doc. If you have a doc and then update it, it will have the
> > _SAME_ uuid but a diff. version number, in my opinion. (by default, if you
> > just use UUID you will get the "latest" doc.)
> > 
> > Also - different formats should have the same UUID. They are the same
> > document, so they will have the same _METADATA_ (i.e. same author, same
> > version, same category, etc.)
> 
> One of the reasons we want a UUID is as a database key.  So it needs to be
> unique between multiple versions, formats, or languages of a document. 
> 
> We also want to guarantee that any two resources with the same UUID are in
> fact the same.  I'm not sure how we will want to handle multiple
> references to the same resource, but we will want to know if two resources
> are identical or not. This will be particularly important if we are ever
> merging multiple databases because you will have situations where both
> databases will refer to the exact same resource (although they may have
> different IDENTIFIERs.)  This is the reason for modifying the UUID for
> each version.

I think duplicates (i.e. "collisions") in of keys should be handled. I still
think that Documents that are the "same" (but diff. version) should have the
same UUID. If you want "further distinction" then you also use the version
key (and possibly file format).

If version and/or format is not specified then you just use the default
(latest version and some sort of priority-based format).
 
> I don't know if we would ever want to have duplicates of a resource in a
> database. (eg. http://www.foo.org/faq.sgml and
> file:/<stuff>/faq.sgml)  If this were the case, the database would need to
> use a key which combined UUID with a code generated from the IDENTIFIER.

No, just use the same UUID and allow the API to be a bit intelligent (i.e. by
specific version, format, wether to only return files installed locally, etc.)

You are saying that the UUID should be /always/ unique. I am saying there can
be duplicates.

Look at .oafinfo files, they do not changed their ID just because the oafinfo
file changes (I think).

> This is leaning slightly back to my original idea of basing the database
> key on the URL, but Alan and DV insisted that URL was not a good key and
> that we should use UUID's instead.

I agree with them :) and even if I didn't, I would still agree with them cos
they are better experts at this stuff.

Regards,
Ali

Re: [Scrollkeeper-devel] Handling relocatable packages

[Laszlo K. <las...@su...>]

Dan Mueth wrote:
> 
> Hi everybody,
> 
> I've been trying to think of a solution to the problem of our current
> ScrollKeeper design not handling relocatable packages properly. (ie.
> packages where the installation location can be determined at install
> time, after the package was built).  I think I have a solution.
> 
> Before I get into my idea, I wanted to ask people who actually supports
> and uses relocatable packages?  Apparently all of Red Hat Linux and GNOME
> are non-relocatable.  Nick has indicated that *BSD at least sometimes uses
> relocatable packages.  I'm curious which operating system / packaging
> system / distributions actually support and use relocatable packages.
> 
> My suggestion:
> 
> For non-relocatable packages, everything stays exactly as it is now and
> was described in proposal #3.
> 
> For relocatable packages, we provide an extra script
> ('scrollkeeper-update-identifier') which can be called as a post-install
> script before 'scrollkeeper-update' to "fix" the doc path in the
> IDENTIFIER field of the OMF file.  This makes a little bit more work to
> make your package relocatable, but I don't think that should surprise
> anybody.
> 
> To make things as convenient as possible, we probably want to make
> scrollkeeper-update-identifier flexible to handle a few different common
> ways a person would want to use it:
> 
> 1) scrollkeeper-update-identifier <omffile> <doc>
> eg: scrollkeeper-update-identifier /usr/share/omf/foo.omf
> /use/local/doc/foo/foo.sgml
> 
> It simply verifies that <doc> exists and then sticks <doc> into <omffile>
> as the IDENTIFIER.
> 
> 2) scrollkeeper-update-identifier <omffile> <path>
> eg: scrollkeeper-update-identifier /usr/share/omf/foo.omf
> /use/local/doc/foo/
> 
> It first reads the IDENTIFIER in <omffile>.
> 
> A) If the IDENTIFIER is just a file name it looks in <path> for the file.
> If it does not find it there, it descends any subdirectories, searching
> for the doc.
> 
> B) If the IDENTIFIER is a path with a file name, it tacks that onto <path>
> and then looks for that file.  This way a person could set up all their
> OMF files initially to hold relative paths to the doc from the main
> director(ies) they are installed in.  So the relative paths are specified
> at build time, and the toplevel directory for the doc is specified at
> install time.
> 
> I think this should make it fairly easy for people to use.  If you have
> many docs, you would probably want to place relative paths in the original
> OMF files. Then in the post-install part of the package, loop over all
> these OMF files passing the <path> under which all of the docs are
> installed. (Similar to what was done in the 'info' installation example
> Nick pointed out.)
> 
> After all the OMF files have their IDENTIFIERs updated, one runs
> 'scrollkeeper-update' as usual.
> 
> Does this sound reasonable to everybody?  Feedback is welcome :)
Yes, it sounds reasonable. (2) can be used if the docs go below the same
relative path. If they don't then (1) has to be used which is ok.

Besides I agree that there are loads of non-relocatable packages so the
option of not using the path updating script/binary should be kept. 

What about scrollkeeper-preinstall? How would it fit into this proposal?
It should be kept for non-relocatable packages probably.

Laszlo