scrollkeeper-devel — Developer discussion list for ScrollKeeper

Re: [Scrollkeeper-devel] Handling relocatable packages

[Laszlo K. <las...@su...>]

Dan Mueth wrote:
> 
> Hi everybody,
> 
> I've been trying to think of a solution to the problem of our current
> ScrollKeeper design not handling relocatable packages properly. (ie.
> packages where the installation location can be determined at install
> time, after the package was built).  I think I have a solution.
> 
> Before I get into my idea, I wanted to ask people who actually supports
> and uses relocatable packages?  Apparently all of Red Hat Linux and GNOME
> are non-relocatable.  Nick has indicated that *BSD at least sometimes uses
> relocatable packages.  I'm curious which operating system / packaging
> system / distributions actually support and use relocatable packages.
> 
> My suggestion:
> 
> For non-relocatable packages, everything stays exactly as it is now and
> was described in proposal #3.
> 
> For relocatable packages, we provide an extra script
> ('scrollkeeper-update-identifier') which can be called as a post-install
> script before 'scrollkeeper-update' to "fix" the doc path in the
> IDENTIFIER field of the OMF file.  This makes a little bit more work to
> make your package relocatable, but I don't think that should surprise
> anybody.
> 
> To make things as convenient as possible, we probably want to make
> scrollkeeper-update-identifier flexible to handle a few different common
> ways a person would want to use it:
> 
> 1) scrollkeeper-update-identifier <omffile> <doc>
> eg: scrollkeeper-update-identifier /usr/share/omf/foo.omf
> /use/local/doc/foo/foo.sgml
> 
> It simply verifies that <doc> exists and then sticks <doc> into <omffile>
> as the IDENTIFIER.
> 
> 2) scrollkeeper-update-identifier <omffile> <path>
> eg: scrollkeeper-update-identifier /usr/share/omf/foo.omf
> /use/local/doc/foo/
> 
> It first reads the IDENTIFIER in <omffile>.
> 
> A) If the IDENTIFIER is just a file name it looks in <path> for the file.
> If it does not find it there, it descends any subdirectories, searching
> for the doc.
> 
> B) If the IDENTIFIER is a path with a file name, it tacks that onto <path>
> and then looks for that file.  This way a person could set up all their
> OMF files initially to hold relative paths to the doc from the main
> director(ies) they are installed in.  So the relative paths are specified
> at build time, and the toplevel directory for the doc is specified at
> install time.
> 
> I think this should make it fairly easy for people to use.  If you have
> many docs, you would probably want to place relative paths in the original
> OMF files. Then in the post-install part of the package, loop over all
> these OMF files passing the <path> under which all of the docs are
> installed. (Similar to what was done in the 'info' installation example
> Nick pointed out.)
> 
> After all the OMF files have their IDENTIFIERs updated, one runs
> 'scrollkeeper-update' as usual.
> 
> Does this sound reasonable to everybody?  Feedback is welcome :)
Yes, it sounds reasonable. (2) can be used if the docs go below the same
relative path. If they don't then (1) has to be used which is ok.

Besides I agree that there are loads of non-relocatable packages so the
option of not using the path updating script/binary should be kept. 

What about scrollkeeper-preinstall? How would it fit into this proposal?
It should be kept for non-relocatable packages probably.

Laszlo

Re: [Scrollkeeper-devel] Handling relocatable packages

[Dan M. <d-...@uc...>]

On Fri, 17 Nov 2000, Nik Clayton wrote:

> On Fri, Nov 17, 2000 at 10:46:20AM -0600, Dan Mueth wrote:
> > Before I get into my idea, I wanted to ask people who actually supports
> > and uses relocatable packages?  Apparently all of Red Hat Linux and GNOME
> > are non-relocatable.  Nick has indicated that *BSD at least sometimes uses
> > relocatable packages.  I'm curious which operating system / packaging
> > system / distributions actually support and use relocatable packages.
> 
> You *absolutely* have to support relocatable packages.  

My questions wasn't *whether* to support relocatable packages.  (As you
can see I proposed a method on how we can support them.)  Instead I was
asking who makes packages which are relocatable.  Presumably whoever
replies with "We use relocatable packages." will have a vested interest in
helping us find the best way to support them.

> The BSD ports/packages systems support it, as does RPM.  I'd be incredibly
> surprised if Debian's format didn't.
> 
> For example, suppose I want to install a Debian administrator's guide
> on to my FreeBSD system (perhaps I want to learn more about Debian).
> 
> If I read the Debian docs properly, it's going to want to install somewhere
> under /usr/doc. 
> 
> Sorry, on a FreeBSD system that's a no go area -- /usr/doc is reserved for
> documentation that comes with the OS (in fact, it's not even /usr/doc anymore
> it's /usr/share/doc/, because /usr/share/ is where architecturally neutral
> files live).
> 
> Since this is a local addition to the system, it would (on FreeBSD) live 
> in /usr/local/share/doc/<mumble>.  Of course, I might not be root on this
> box, in which case I'm probably going to want to install it somewhere
> under $HOME/share/doc/, or similar (or maybe even $HOME/public_html, so
> I can browse it easily from elsewhere).
> 
> > My suggestion:
> > 
> > For non-relocatable packages, everything stays exactly as it is now and
> > was described in proposal #3.
> 
> There should be no such thing as a non-relocatable package.

Perhaps. However that is more idealogical.  In reality many (possibly    
most?) packages are non-relocatable.  For example all of Red Hat Linux is
non-relocatable.  (I have no idea about the other Linux distributions or   
Solaris.)  Those people who already have non-relocatable packages will
leave out the step which makes their docs relocatable.

> For FreeBSD, this means that some of the guys that produce the ports do
> actually go through the code, substituting "/usr/bin" with "$PREFIX/bin"
> (or whatever) so that the application can be installed outside of it's
> expected directory hierarchy.

So what do you think of my suggestion for how we allow people to make   
relocatable packages which use scrollkeeper?  Does it seem like a decent
solution?  Do you have any ideas on a better solution or how we can
improve this one?  

It seems to me that it makes things substantially easier than the original
suggestion which required the install script to keep track of the location
of every OMF file and every document.  

At the same time it keeps things as simple as possible for those people
who do not make relocatable packages.

Dan

Re: [Scrollkeeper-devel] Handling relocatable packages

[Nik C. <ni...@no...>]

On Fri, Nov 17, 2000 at 10:46:20AM -0600, Dan Mueth wrote:
> Before I get into my idea, I wanted to ask people who actually supports
> and uses relocatable packages?  Apparently all of Red Hat Linux and GNOME
> are non-relocatable.  Nick has indicated that *BSD at least sometimes uses
> relocatable packages.  I'm curious which operating system / packaging
> system / distributions actually support and use relocatable packages.

You *absolutely* have to support relocatable packages.  

The BSD ports/packages systems support it, as does RPM.  I'd be incredibly
surprised if Debian's format didn't.

For example, suppose I want to install a Debian administrator's guide
on to my FreeBSD system (perhaps I want to learn more about Debian).

If I read the Debian docs properly, it's going to want to install somewhere
under /usr/doc. 

Sorry, on a FreeBSD system that's a no go area -- /usr/doc is reserved for
documentation that comes with the OS (in fact, it's not even /usr/doc anymore
it's /usr/share/doc/, because /usr/share/ is where architecturally neutral
files live).

Since this is a local addition to the system, it would (on FreeBSD) live 
in /usr/local/share/doc/<mumble>.  Of course, I might not be root on this
box, in which case I'm probably going to want to install it somewhere
under $HOME/share/doc/, or similar (or maybe even $HOME/public_html, so
I can browse it easily from elsewhere).

> My suggestion:
> 
> For non-relocatable packages, everything stays exactly as it is now and
> was described in proposal #3.

There should be no such thing as a non-relocatable package.

For FreeBSD, this means that some of the guys that produce the ports do
actually go through the code, substituting "/usr/bin" with "$PREFIX/bin"
(or whatever) so that the application can be installed outside of it's
expected directory hierarchy.

N
-- 
Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
Telephone line, $24.95 a month.  Software, free.  USENET transmission,
hundreds if not thousands of dollars.  Thinking before posting, priceless.
Somethings in life you can't buy.  For everything else, there's MasterCard.
  -- Graham Reed, in the Scary Devil Monastery

[Scrollkeeper-devel] Handling relocatable packages

[Dan M. <d-...@uc...>]

Hi everybody,

I've been trying to think of a solution to the problem of our current
ScrollKeeper design not handling relocatable packages properly. (ie.
packages where the installation location can be determined at install
time, after the package was built).  I think I have a solution.

Before I get into my idea, I wanted to ask people who actually supports
and uses relocatable packages?  Apparently all of Red Hat Linux and GNOME
are non-relocatable.  Nick has indicated that *BSD at least sometimes uses
relocatable packages.  I'm curious which operating system / packaging
system / distributions actually support and use relocatable packages.

My suggestion:

For non-relocatable packages, everything stays exactly as it is now and
was described in proposal #3.

For relocatable packages, we provide an extra script
('scrollkeeper-update-identifier') which can be called as a post-install
script before 'scrollkeeper-update' to "fix" the doc path in the
IDENTIFIER field of the OMF file.  This makes a little bit more work to
make your package relocatable, but I don't think that should surprise
anybody.

To make things as convenient as possible, we probably want to make
scrollkeeper-update-identifier flexible to handle a few different common
ways a person would want to use it:

1) scrollkeeper-update-identifier <omffile> <doc>
eg: scrollkeeper-update-identifier /usr/share/omf/foo.omf
/use/local/doc/foo/foo.sgml

It simply verifies that <doc> exists and then sticks <doc> into <omffile>
as the IDENTIFIER.

2) scrollkeeper-update-identifier <omffile> <path>
eg: scrollkeeper-update-identifier /usr/share/omf/foo.omf
/use/local/doc/foo/

It first reads the IDENTIFIER in <omffile>.

A) If the IDENTIFIER is just a file name it looks in <path> for the file.
If it does not find it there, it descends any subdirectories, searching
for the doc.

B) If the IDENTIFIER is a path with a file name, it tacks that onto <path>
and then looks for that file.  This way a person could set up all their
OMF files initially to hold relative paths to the doc from the main
director(ies) they are installed in.  So the relative paths are specified
at build time, and the toplevel directory for the doc is specified at
install time.

I think this should make it fairly easy for people to use.  If you have
many docs, you would probably want to place relative paths in the original
OMF files. Then in the post-install part of the package, loop over all
these OMF files passing the <path> under which all of the docs are
installed. (Similar to what was done in the 'info' installation example
Nick pointed out.)

After all the OMF files have their IDENTIFIERs updated, one runs
'scrollkeeper-update' as usual.

Does this sound reasonable to everybody?  Feedback is welcome :)

Dan

Re: [Scrollkeeper-devel] Registering documents with ScrollKeeper

[Dan M. <d-...@uc...>]

On Wed, 15 Nov 2000, Nik Clayton wrote:

> On Tue, Nov 14, 2000 at 11:05:09AM -0600, Dan Mueth wrote:
> > Plan #1
> > -------
> > After the document and OMF file are installed, scrollkeeper is informed
> > (by way of a post-install script) where the document and the OMF file
> > are at with something like:
> > 
> > scrollkeeper-install <docfile> <omffile>
> > 
> > Pro's: If people change where things get installed during installation of
> > the package (and after the package is built), things do not break.
> > 
> > Con's: This requires one post-install script per document in the
> > package.  It also requires that the packager keep track of the appropriate
> > path variables.  (ie. it is a substantial burden on packagers)
> 
> By "per document", do you mean "per document", or do you mean "per file 
> that might make up a document, such as if a FAQ is broken up in to 15
> different files"?

I mean "per document".  So a directory full of HTML files and PNG's which
comprise one single document is just considered one document, and it has
one OMF file which points to the main file for the document (perhaps
/usr/share/doc/<appname>/<local>/<docname>/index.html.)

> If it's the former, I really don't see this being a huge problem.  It's
> exactly what package maintainers have to do at the moment when they 
> install a library, and need to run ldconfig afterwards.

I'm glad you don't think this would be a problem.  Would anybody like to
argue that it would be a problem?  (Eric?)

I don't have a very good feeling for how much packagers are willing to
do.  The argument Eric made was that this would be too much to expect from
packagers.

> In addition, I would expect most documentation for an application is
> going to install in a subdirectory that's specific to that application.
> So the package maintainer should be able to get away with something like
> 
>     scrollkeeper-update /usr/local/share/doc/app-name
> 
> and let scrollkeeper-update work out what to pull in.

For this to work we would have to require each OMF file lives in the same
directory as the document it describes (or possibly have a reliable
relative path to the document).  Then we just point scrollkeeper-update to
all directories where doc/omf pairs are installed.  We would still need
one call (or one argument) to the script per directory we install docs in.  
But this is just an incremental improvement over Plan #1, at the cost of
requiring the OMF file live in the same directory as the document.  

I suspect that generally each document (and each locale) will have its own
directory.  For something like gnome-applets, which is admittedly
atypical, this will be 300 or more distinct directories once the
translations are done.  So we would have to loop over the directories as
your info file example below does.  

> As long as it's not more complicated than this process to manage info files,
> we should be OK;
> 
>     http://www.freebsd.org/porters-handbook/porting-info.html

Dan

Re: [Scrollkeeper-devel] Registering documents with ScrollKeeper

[Nik C. <ni...@no...>]

On Tue, Nov 14, 2000 at 11:05:09AM -0600, Dan Mueth wrote:
> Plan #1
> -------
> After the document and OMF file are installed, scrollkeeper is informed
> (by way of a post-install script) where the document and the OMF file
> are at with something like:
> 
> scrollkeeper-install <docfile> <omffile>
> 
> Pro's: If people change where things get installed during installation of
> the package (and after the package is built), things do not break.
> 
> Con's: This requires one post-install script per document in the
> package.  It also requires that the packager keep track of the appropriate
> path variables.  (ie. it is a substantial burden on packagers)

By "per document", do you mean "per document", or do you mean "per file 
that might make up a document, such as if a FAQ is broken up in to 15
different files"?

If it's the former, I really don't see this being a huge problem.  It's
exactly what package maintainers have to do at the moment when they 
install a library, and need to run ldconfig afterwards.

In addition, I would expect most documentation for an application is
going to install in a subdirectory that's specific to that application.
So the package maintainer should be able to get away with something like

    scrollkeeper-update /usr/local/share/doc/app-name

and let scrollkeeper-update work out what to pull in.

As long as it's not more complicated than this process to manage info files,
we should be OK;

    http://www.freebsd.org/porters-handbook/porting-info.html

N
-- 
Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
Telephone line, $24.95 a month.  Software, free.  USENET transmission,
hundreds if not thousands of dollars.  Thinking before posting, priceless.
Somethings in life you can't buy.  For everything else, there's MasterCard.
  -- Graham Reed, in the Scary Devil Monastery

[Scrollkeeper-devel] Registering documents with ScrollKeeper

[Dan M. <d-...@uc...>]

We are back to our favorite debate... "How should a package register its
documentation with scrollkeeper?"

Let me try to catch people up with our previous plans:

Plan #1
-------
After the document and OMF file are installed, scrollkeeper is informed
(by way of a post-install script) where the document and the OMF file
are at with something like:

scrollkeeper-install <docfile> <omffile>

Pro's: If people change where things get installed during installation of
the package (and after the package is built), things do not break.

Con's: This requires one post-install script per document in the
package.  It also requires that the packager keep track of the appropriate
path variables.  (ie. it is a substantial burden on packagers)


Plan #2
-------
We use a build-time script which (a) writes the paths of the docs into
the omf files, and (b) merges the omf files together.  Then we call a
post-install script:

scrollkeeper-install <bigomffile>

Pro's: We only have one post-install script, even if we have many
documents.

Con's: This solution breaks if the package is not installed in its default
location since the doc paths are written into <bigomffile> at build
time.  There are small problems with the build-time script (we called it
scrollkeeper-preinstall).

This solution made things much simpler for the developer than #1, but not
as simple as #3 below.  It seems to share the same problem as #3, but is
more complex.

Plan #3 
------- 

A build-time script copies each omf file to a new file which will be
installed, modifying the URI. We require all packages to install their OMF
files into one of one or more standard paths.  After installation,
modification, or uninstallation of an OMF file and doc, a script is called
(scrollkeeper-update) which looks in these OMF paths, detects what has
changed, and acts accordingly.

Pro's: Very simple for developers.  Although it still uses a post-install
script, it is a completely trivial line to add.

Con's: This solution is broken if the package is not installed in its
default location.

-----------------

Generally, scrollkeeper needs to know when an OMF file has been added,
modified, or removed. It also needs to know which document the OMF file
describes.  (The OMF file has a tag which should hold the location of the
doc.)

I think we really want to avoid any solution which requires one
post-install script per doc (like #1 above), since this becomes a huge
burden for packages with more than one doc and people are likely to forget
to update things after adding each new doc.

The problem with the simpler solutions is that they generally require the
paths to be merged into the OMF files at build time which means the paths
tend to be wrong if people change where things get installed at install
time.

Would it be possible to use Plan #3, but have scrollkeeper-install take a
parameter which describes the prefix the document is installed under?  
This way scrollkeeper would know if the user is installing the docs under
a different path and could still find them?  

I'm still looking for Plan #4 (The Perfect Plan), so if you have any ideas
please post them.

Dan

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Gregory L. <gle...@cu...>]

I'm going to snip this CC: line as I assume everybody is on the list,
and that you don't really need two copies of this stuff.


> On Tue, Nov 14, 2000 at 09:39:56AM +0000, Laszlo Kovacs wrote:
> > > Most package systems should provide a mechanism to call scrollkeeper-install
> > > after a document has been added to the system, ideally with the path in
> > > which the document was installed.  For those that don't, the system can
> > > run something periodically that re-indexes the list of installed
> > > documentation, in the same way that the apropos database is built at the
> > > moment.
> >
> > The problem with this is that packagers will have to create large
> > postinstall scripts to get it working right and some of the people on
> > this list think they will not do it if the changes are not as simple as
> > possible. If we disregard the packagers possible reaction then most/all
> > of the Scrollkeeper related process should be launched from postinstall
> > scripts. But as I said some people strongly opposed that.
> 
> How large?  
> 
> Installing the documentation and running a scrollkeeper command should be
> as trivial as installing a new library and having to run ldconfig after
> the install.


This is how the current implementation works.  The packager adds a line
that says 'scrollkeeper-update' in their post-install script, which
works in a similar manner to ldconfig.  I'd recomend download from CVS
and building Scrollkeeper.  It takes less than 5 minutes to download,
configure, build, and install here, and then I can take a look at the
source and the stuff that gets installed, so it's easy to answer
questions.  Mind you, it changes a lot from day to day, but it's still a
tiny job to update it if I want to take a look.

    Greg

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Laszlo K. <las...@su...>]

> How large?
> 
> Installing the documentation and running a scrollkeeper command should be
> as trivial as installing a new library and having to run ldconfig after
> the install.

Each would be described by a separate metadata file (or it can be one
only for the whole package). A script has to be run at postinstall that
changes the url(s) in the metadata file to the final installed path of
the doc. Then the scrollkeeper-update binary has to be run. Hopefully I
did not forget anything. This can be from 2 lines to (the number of docs
+ 1) lines in the postinstall script, depending on whether one big
metadata file is used for the package or a separate one for each doc. In
the first case the url changing script will have a very long command
line. I assume that the relocation path is available in the postinstall
script in every packaging system.

Laszlo

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Nik C. <ni...@fr...>]

On Tue, Nov 14, 2000 at 09:39:56AM +0000, Laszlo Kovacs wrote:
> > > This would pose a significant problem.  The original design installed each
> > > document with something like:
> > >
> > >     scrollkeeper-install <omffile> <doc>
> > 
> > I think that's probably a bad idea.  Each OS is going to have its own idea
> > of what program to use to physically install the document, and set the
> > ownership and permissions properly.  I don't see much point in reinventing
> > install(1) or cp(1).
> 
> This is a misunderstanding. It is an addition to install and cp, not a
> reinvention. It installs the doc into the Scrollkeeper database after it
> was installed with install or cp to the right location.

Ah, OK.

[snip]

> > Most package systems should provide a mechanism to call scrollkeeper-install
> > after a document has been added to the system, ideally with the path in
> > which the document was installed.  For those that don't, the system can
> > run something periodically that re-indexes the list of installed
> > documentation, in the same way that the apropos database is built at the
> > moment.
>
> The problem with this is that packagers will have to create large
> postinstall scripts to get it working right and some of the people on
> this list think they will not do it if the changes are not as simple as
> possible. If we disregard the packagers possible reaction then most/all
> of the Scrollkeeper related process should be launched from postinstall
> scripts. But as I said some people strongly opposed that.

How large?  

Installing the documentation and running a scrollkeeper command should be
as trivial as installing a new library and having to run ldconfig after
the install.

N
-- 
Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
Telephone line, $24.95 a month.  Software, free.  USENET transmission,
hundreds if not thousands of dollars.  Thinking before posting, priceless.
Somethings in life you can't buy.  For everything else, there's MasterCard.
  -- Graham Reed, in the Scary Devil Monastery

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Ali A. <ali...@au...>]

* Laszlo Kovacs (las...@su...) wrote at 17:09 on 14/11/00:
> > Perhaps we should use a URI instead of a URL (so a local file would be
> > 'file:///usr/doc/foo.sgml' or
> > 'http://scrollkeeper.sourceforge.net/manual.sgml' - but that would suck,
> > because you'd have to handle the URI parsing (which will just duplicate
> > code that is in gnome-vfs (and prolly some other KDE library out there))
> 
> Any suggestions to fix this problem? Btw as we all know Scrollkeeper is
> supposed to be independent from Gnome. 

Well - you could implement a very small subset of URI functions (only the ones
that you need (and only handle file/http/ftp)...That way we can stay
"neutral"

Actually - I don't think that scrollkeeper would need to handle anything
except the 'file:///' URI type...

Please the burden on the browser to go online and retrieve docs (or go online
and fetch the metadata) - That way each project can use its own libraries.

If you _DO_ have the metadata to an online doc installed on your system, and
somebody tries to "retrieve" that doc, ScrollKeeper would return an error
(NOTAVAILABLE (note: this is not the same as NOTFOUND))

One of the problems I see is two "versions" of metadata (one for online, one
for offline) - how do you handle those? ("merge" them?)

Regards,
Ali
 

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Laszlo K. <las...@su...>]

> Perhaps we should use a URI instead of a URL (so a local file would be
> 'file:///usr/doc/foo.sgml' or
> 'http://scrollkeeper.sourceforge.net/manual.sgml' - but that would suck,
> because you'd have to handle the URI parsing (which will just duplicate
> code that is in gnome-vfs (and prolly some other KDE library out there))

Any suggestions to fix this problem? Btw as we all know Scrollkeeper is
supposed to be independent from Gnome. 

Laszlo

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Ali A. <ALI...@au...>]

On Tue, 14 Nov 2000, Laszlo Kovacs wrote:

> > > This would pose a significant problem.  The original design installed each
> > > document with something like:
> > >
> > >     scrollkeeper-install <omffile> <doc>
> > 
> > I think that's probably a bad idea.  Each OS is going to have its own idea
> > of what program to use to physically install the document, and set the
> > ownership and permissions properly.  I don't see much point in reinventing
> > install(1) or cp(1).
> 
> This is a misunderstanding. It is an addition to install and cp, not a
> reinvention. It installs the doc into the Scrollkeeper database after it
> was installed with install or cp to the right location.
> 
> > I think it's probably better to provide a scrollkeeper configuration
> > file (in XML would be nice :-) ) that describes a list of paths down
> > which scrollkeeper (sk from now on) should recurse to look for new
> > documentation.
> There will be some solution in the future to extend the one directory
> where metadata files are now to a set of directories. This is for
> metadata only, Scrollkeeper intends to support cataloging of docs based
> anywhere locally and on the web, this is why we thought the metadata
> should contain the url.

Perhaps we should use a URI instead of a URL (so a local file would be 
'file:///usr/doc/foo.sgml' or 
'http://scrollkeeper.sourceforge.net/manual.sgml' - but that would suck, 
because you'd have to handle the URI parsing (which will just duplicate 
code that is in gnome-vfs (and prolly some other KDE library out there))
  
> > Most package systems should provide a mechanism to call scrollkeeper-install
> > after a document has been added to the system, ideally with the path in
> > which the document was installed.  For those that don't, the system can
> > run something periodically that re-indexes the list of installed
> > documentation, in the same way that the apropos database is built at the
> > moment.
> The problem with this is that packagers will have to create large
> postinstall scripts to get it working right and some of the people on
> this list think they will not do it if the changes are not as simple as
> possible. If we disregard the packagers possible reaction then most/all
> of the Scrollkeeper related process should be launched from postinstall
> scripts. But as I said some people strongly opposed that.
> 
> > Depends how well written the package is.  Most of the FreeBSD ports install
> > with $PREFIX set to something other than /usr/local with no problems.
> > 
> > Also, keep in mind that some of these packages are going to consist of
> > nothing but documentation.
> > 
> > For example, *right now*, a FreeBSD user can do something like
> > 
> >     pkg_add -r -p /opt/share/doc \
> >       ftp://ftp.freebsd.org/pub/FreeBSD/doc/packages/handbook.en_US.ISO_8859-1.html.tgz
> > 
> > This will download the HTML version of the English FreeBSD handbook, and
> > install it under /opt/share/doc (instead of the more usual /usr/share/doc).
> > 
> > Now, pkg_add in FreeBSD has hooks to allow post-install scripts to be
> > run, that could call an sk command to update the index.  But the sysadmin
> > should also be free to create /usr/local/etc/scrollkeeper.cf, and make sure
> > that "/opt/share/doc/" is listed as a directory to be indexed.
> This suggestion does work only if the doc is locally installed, what if
> I point to a page on the web, where the document is? 

[Scrollkeeper-devel] Scrollkeeper example apps

[Laszlo K. <las...@su...>]

Hello,

I created the first example application for Scrollkeeper. It is in the
scrollkeeper_examples toplevel directory in the CVS and it is called
hello-world-1. It does not have any binaries, only one .sgml doc
translated to four languages. The doc itself is not translated, I took
desktop-docs/panel.sgml from the Gnome CVS for this.

The Scrollkeeper code from the CVS implements most of the #3 proposal.
This can be found here:

http://www.geocrawler.com/lists/3/SourceForge/6429/0/4650283/

The example application's README file describes what the installation of
this package will explain about Scrollkeeper. However a read through the
proposal is needed to know where to look for the results of the
installation process.

Please give it a try and let me know how you find it.

Thanks,

Laszlo

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Laszlo K. <las...@su...>]

> > This would pose a significant problem.  The original design installed each
> > document with something like:
> >
> >     scrollkeeper-install <omffile> <doc>
> 
> I think that's probably a bad idea.  Each OS is going to have its own idea
> of what program to use to physically install the document, and set the
> ownership and permissions properly.  I don't see much point in reinventing
> install(1) or cp(1).

This is a misunderstanding. It is an addition to install and cp, not a
reinvention. It installs the doc into the Scrollkeeper database after it
was installed with install or cp to the right location.

> I think it's probably better to provide a scrollkeeper configuration
> file (in XML would be nice :-) ) that describes a list of paths down
> which scrollkeeper (sk from now on) should recurse to look for new
> documentation.
There will be some solution in the future to extend the one directory
where metadata files are now to a set of directories. This is for
metadata only, Scrollkeeper intends to support cataloging of docs based
anywhere locally and on the web, this is why we thought the metadata
should contain the url.
 
> Most package systems should provide a mechanism to call scrollkeeper-install
> after a document has been added to the system, ideally with the path in
> which the document was installed.  For those that don't, the system can
> run something periodically that re-indexes the list of installed
> documentation, in the same way that the apropos database is built at the
> moment.
The problem with this is that packagers will have to create large
postinstall scripts to get it working right and some of the people on
this list think they will not do it if the changes are not as simple as
possible. If we disregard the packagers possible reaction then most/all
of the Scrollkeeper related process should be launched from postinstall
scripts. But as I said some people strongly opposed that.

> Depends how well written the package is.  Most of the FreeBSD ports install
> with $PREFIX set to something other than /usr/local with no problems.
> 
> Also, keep in mind that some of these packages are going to consist of
> nothing but documentation.
> 
> For example, *right now*, a FreeBSD user can do something like
> 
>     pkg_add -r -p /opt/share/doc \
>       ftp://ftp.freebsd.org/pub/FreeBSD/doc/packages/handbook.en_US.ISO_8859-1.html.tgz
> 
> This will download the HTML version of the English FreeBSD handbook, and
> install it under /opt/share/doc (instead of the more usual /usr/share/doc).
> 
> Now, pkg_add in FreeBSD has hooks to allow post-install scripts to be
> run, that could call an sk command to update the index.  But the sysadmin
> should also be free to create /usr/local/etc/scrollkeeper.cf, and make sure
> that "/opt/share/doc/" is listed as a directory to be indexed.
This suggestion does work only if the doc is locally installed, what if
I point to a page on the web, where the document is? 


Laszlo

Re: [Scrollkeeper-devel] What about the distributions?

[Ali A. <ALI...@au...>]

On Mon, 13 Nov 2000, Eric Bischoff wrote:

> Le Dimanche 29 Octobre 2000 23:10, Dan Mueth a écrit :
> > On Fri, 27 Oct 2000, Eric Bischoff wrote:
> > It is worse, because an admin is willing to wait five seconds or more for
> > a package to install, but users don't want to wait 5 extra seconds when
> > they open their help browser.
> 
> Yes.
> 
> Well the 5 extra seconds will be on the *first* time they open the browser. 
> The next times it will be much quicker. You don't install tons of packages 
> every day.

Those 5 seconds are unacceptable to me. And they won't be done only the 
*first* time, they will be done everytime you install a new doc.
 
> > > Another solution, if the contents list is dynamic but small, is to create
> > > it in the user's home directory. It's a bit of redundant work to
> > > scrollkeeper to do this for each user, but it also enables each user to
> > > build a contents list that matches the best its inline searches.
> >
> > I am hesitant to do this, partly because it seems strange and partly
> > because it will cause a performance problem. At the same time, I like the
> > idea of having this as an option for the future since it would potentially
> > allow each user to configure their own Contents List, which would be
> > great.  I think the default behavior should not be to replicate itself
> > into each user's $HOME directory, but to use the system-wide database
> > unless a user explicitly wants to add or remove contents.
> 
> Yes.
> 
> >  This would be
> > feature which we would not implement for a long time anyway.  If and when
> > we implement it, we can probably get around the performance problem by
> > copying the system-wide database to start with.  However not having a
> > system-wide database means that each user gets completely hammered the
> > first time they open their browser.
> 
> Yup.
> 
> > > But at install time it means postinstall scripts. It's technically
> > > better, but I can't figure out how distributions will react. It was
> > > painful enough to change to LSB directory tree at Caldera.
> >
> > I agree this is a very important issue.  We have to be confident the
> > distributions will accept this.  However changing the directory tree is a
> > *much* bigger concern than adding scrollkeeper.
> 
> Are you sure? If you count the changed lines in the SPEC files it would even 
> be worse.

I do not think that changing one line in several thousand spec files is a 
big issue. Its not like they are done by "one person". 

I agree that it would be nice to avoid thise post-install script. But the 
alternatives that have been suggested are not feasibly (in my opinion)
 
> >  Plus, scrollkeeper can be
> > added in little pieces.  Scrollkeeper will be adopted slowly, giving
> > distributions time to get familiar with it.  If GNOME 1.4 can use it, as I
> > am desperately hoping, then that means that maybe 20 packages will be
> > using it in the next few months.  (GNOME 1.4 should be out in about 6
> > weeks, if it can stay on schedule ;)  If KDE adopts it for their next
> > release along with a few other people, we are looking at maybe 70 packages
> > 5 or 6 months from now.  That isn't so much to really swamp anybody.
> > Plus, I'm planning that the lines we add to the spec files are completely
> > generic.  Basically just cut and paste 4 lines into the spec file and you
> > are done.  If you forget, then that doc doesn't show up in the contents
> > list. It doesn't really break anything, so I don't think anybody is going
> > to scream about a distribution forgetting to add it to a couple spec
> > files.
> 
> Hmmm if your help browser "forgets" to present you with the results of your 
> query it's really bad - once you have a tool you start trusting it.

It doesn't "forget" - it just doesn't mention the fact that your help 
file is installed because the OMF file is not installed. Somebody will 
then complain "this app doesn't have a help file". The author would then 
properly install the OMF file.

> > > Again, post-install scripts *are* the technical solution, but will upset
> > > the distributions. There's no need for you to prove that they are better,
> > > we already know it.
> >
> > My talks with Jonathan and Jeff (via. Jonathan) at Red Hat have indicated
> > that they did not feel this would place an undo burden on distributors.
> > That is my impression as well, although you (Eric), Jonathan, and Jeff
> > have much more expertise in this area than I do.
> 
> > > Yes. I think we are in the heart of the problem. Sorry to have started
> > > all of this, but it was a mere fact than in a distribution a "make
> > > install" is done by the packager, not by the end user.
> >
> > It is great to have your help.  My original plan would have had many OMF
> > files in each spec file and many calls to post-install scripts, which
> > would have made quite a bit of work for packagers.
> 
> Even if we stick to the post-install script solution, which I do not 
> encourage, it's true we did a lot of progress on this front.

I'd like to know "other" people's opinion on the post-install script 
thing (so far its mainly been a "GNOME-affiliated camp" vs. 
"KDE-affiliated camp" debate). What about the various other people's 
opinions? ;)
 
> > > Let me state the problem the other way round. We won't merge the OMF nor
> > > the DocBook files either. So why should we merge the extra data and not
> > > these ones?
> >
> > Aside from all the reasons I've already stated? ;)
> >
> > The docs and OMF files belong to the individual packages to install and
> > uninstall as necessary to remain in sync with the packages.  The
> > scrollkeeper database belongs to scrollkeeper, to maintain, update,
> > synchronize, modify, and use as scrollkeeper needs.  I think having
> > scrollkeeper's database created, shipped, and maintained (or in this case
> > *not maintained*) by the applications (in many pieces) instead of
> > by scrollkeeper greatly restricts scrollkeeper's ability to do its job.
> 
> Do we have a real evaluation of the cost of having a split database?
> 
> > Well, I prefer the post-install script, which is why I do not emphasize
> > this solution.  However, if we *really* wanted to avoid the script, we
> > could do something like: Have a magic directory every package installs a
> > file into, say /var/lib/scrollkeeper/packages.  Each package installs a
> > short file in here which lists the full path to the OMF file(s) installed
> > by the package.  Then whenever scrollkeeper is run it checks to see if
> > there are any new, modified, or missing files in this directory.  If so,
> > it updates its database.
> 
> It's what I'm suggesting for ages. The only real drawback with this solution 
> is the delay when you first need the merged database. And I'm not sure it 
> will be 5 seconds. Of course there are the other problems like the rights but 
> we know there are solutions.
> 
> >  The advantage is that it still uses one database
> > making things simpler and it can keep the versions correct.  The downside
> > is that it still has the permissions and performance problem, it requires
> > all packages to install something into a special directory, and it is
> > generally hokey.  Plus it has to detect when a file has been modified,
> > which means either comparing the contents or checking timestamps, neither
> > of which seem pretty.
> 
> 
> -- 
> Éric Bischoff  -  Documentation and Localization
> Caldera (Deutschland) GmbH - Linux for eBusiness
> Tel: +49 9131 7192 300 -  Fax: +49 9131 7192 399
> http://www.caldera.de/
> _______________________________________________
> Scrollkeeper-devel mailing list
> Scr...@li...
> http://lists.sourceforge.net/mailman/listinfo/scrollkeeper-devel
> 

Re: [Scrollkeeper-devel] What about the distributions?

[Ali A. <ALI...@au...>]

On Mon, 13 Nov 2000, Eric Bischoff wrote:

> Le Samedi 28 Octobre 2000 00:38, Ali Abdin a écrit :
> > > This will probably be autodetected, thanks to Unix rights system. A
> > > package is always built as a normal user the first time. The usual
> > > process is then to add the right arguments for buildrooting in the spec
> > > file.
> >
> > No it won't - most SPEC files do not do this!!
> 
> I suppose this is a misunderstanding. Yes they do. They need it at least to 
> pass the build root to the make files.

No - most specfiles do not specify a '--localstatedir' - so the autoconf 
uses the "default" by assuming it is the same as --prefix (i.e. usually 
/usr or /usr/local)

Regards,
Ali

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Nik C. <ni...@fr...>]

On Mon, Nov 13, 2000 at 11:45:26AM -0600, Dan Mueth wrote:
> > You probably don't know where the application is going to be installed until
> > install time, not build time.  Using the FreeBSD ports tree, it's trivial
> > to do
> > 
> >     cd /usr/ports/foo/bar
> >     make
> >     make PREFIX=/somewhere/else install
> > 
> > Similarly, if you're building packages on BSD (or RPMs, or whatever) the
> > end user can change the install path after the package has been built, but
> > before it has been installed.
> 
> This would pose a significant problem.  The original design installed each
> document with something like:
> 
>     scrollkeeper-install <omffile> <doc>

I think that's probably a bad idea.  Each OS is going to have its own idea
of what program to use to physically install the document, and set the 
ownership and permissions properly.  I don't see much point in reinventing
install(1) or cp(1).

> so that the path of the doc was given to ScrollKeeper at install
> time.  This would solve the problem of paths changing between build and
> install time.  However we decided to change the process to simplify things
> for developers and packagers.  (see the mailing list archives...)

I think it's probably better to provide a scrollkeeper configuration
file (in XML would be nice :-) ) that describes a list of paths down
which scrollkeeper (sk from now on) should recurse to look for new 
documentation.

Most package systems should provide a mechanism to call scrollkeeper-install
after a document has been added to the system, ideally with the path in
which the document was installed.  For those that don't, the system can
run something periodically that re-indexes the list of installed 
documentation, in the same way that the apropos database is built at the 
moment.

> I would expect that changing the install path of a package after it is
> built would often wreak havoc on it.  Do people really do this very often,
> and are most packages robust against this?  (ie. Do we really have to
> worry about this?)

Depends how well written the package is.  Most of the FreeBSD ports install
with $PREFIX set to something other than /usr/local with no problems.

Also, keep in mind that some of these packages are going to consist of 
nothing but documentation.

For example, *right now*, a FreeBSD user can do something like

    pkg_add -r -p /opt/share/doc \
      ftp://ftp.freebsd.org/pub/FreeBSD/doc/packages/handbook.en_US.ISO_8859-1.html.tgz

This will download the HTML version of the English FreeBSD handbook, and 
install it under /opt/share/doc (instead of the more usual /usr/share/doc).

Now, pkg_add in FreeBSD has hooks to allow post-install scripts to be
run, that could call an sk command to update the index.  But the sysadmin
should also be free to create /usr/local/etc/scrollkeeper.cf, and make sure
that "/opt/share/doc/" is listed as a directory to be indexed.

> > Your examples don't use a sufficiently long <locale>.  For example, the user
> > might install docs in Chinese, using both EUC and Big 5 encoding.  For that,
> > you really need to be able to specify zh_CN.EUC and zh_TW.Big5.
> > 
> > Note that trying to rely on just zh_CN or zh_TW isn't sufficient for other
> > languages, like Japanese, where the various encodings hang off a ja_JP
> > prefix.
> 
> I don't think long locales pose any problems.

As long as people are aware of the issue, that's all.

N
-- 
Internet connection, $19.95 a month.  Computer, $799.95.  Modem, $149.95.
Telephone line, $24.95 a month.  Software, free.  USENET transmission,
hundreds if not thousands of dollars.  Thinking before posting, priceless.
Somethings in life you can't buy.  For everything else, there's MasterCard.
  -- Graham Reed, in the Scary Devil Monastery

Re: [Scrollkeeper-devel] scrollkeeper summary, please

[Paul J. <pj...@me...>]

On Fri, 10 Nov 2000, Guylhem Aznar wrote:

+>I'm especially interested in this guy with the Buffallo Bill haircut and
+>moustache who presented us with OMF and Dublin Core? What was his name and
+>project again?
+
+Paul, you've got a new fan :-)
+His name is Paul Jones, he is from ibiblio.org at the UNC
see http://ibiblio.org/pjones/whoyanni.html for other lookalikes
and yes, i've signed on now

==========================================================================
                             Paul Jones
                    "Gort! Klaatu barada nikto!"
http://www.ibiblio.org/pjones/ at the Site Formerly Known As MetaLab.unc.edu
  pj...@ib...   voice: (919) 962-7600     fax: (919) 962-8071
===========================================================================

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Dan M. <d-...@uc...>]

On Mon, 13 Nov 2000, Nik Clayton wrote:

> On Sun, Nov 12, 2000 at 10:22:33PM -0600, Dan Mueth wrote:
> > Hi everybody,
> 
> Ditto.  A brief intro -- I'm the FreeBSD Documentation Project cat herder.
> I was at the O'Reilly conference earlier this year, and I subscribed to this
> list a few days ago.  I've been working my way through the archives :-)
> 
> > Feedback is welcome :)
> 
> OK :-)
> 
> >                 --------------------------------------
> >                 |   ScrollKeeper 0.1 - Proposal #3   |
> >                 --------------------------------------
> >
> > II. Building, Installing, and Uninstalling
> > 
> > i. Building
> > 
> > The OMF documents are manipulated during the application
> > build process, generally as part of 'make all'.  The purpose of this
> > is to (1) substitute the correct URL of the document where it will
> > actually be installed, which is not generally known until build time,
> > and to (2) extract the OMF from DocBook documents into a seperate file.
> > (NOTE: #2 may not be supported in ScrollKeeper 0.1)
> 
> You probably don't know where the application is going to be installed until
> install time, not build time.  Using the FreeBSD ports tree, it's trivial
> to do
> 
>     cd /usr/ports/foo/bar
>     make
>     make PREFIX=/somewhere/else install
> 
> Similarly, if you're building packages on BSD (or RPMs, or whatever) the
> end user can change the install path after the package has been built, but
> before it has been installed.

This would pose a significant problem.  The original design installed each
document with something like:

    scrollkeeper-install <omffile> <doc>

so that the path of the doc was given to ScrollKeeper at install
time.  This would solve the problem of paths changing between build and
install time.  However we decided to change the process to simplify things
for developers and packagers.  (see the mailing list archives...)

I will have to consider this some more.

Anybody have an idea for a solution which is easy on the packagers but
solves the problem of people changing the default installation directory
of a package?

I would expect that changing the install path of a package after it is
built would often wreak havoc on it.  Do people really do this very often,
and are most packages robust against this?  (ie. Do we really have to
worry about this?)

> I know that FreeBSD packages and RPMs both support running commands after
> the install -- I assume the .deb format does as well. 
> 
> > Note that the OMF file is named <application>-<document_title>-<locale>.omf.
> > This is important to avoid collisions between OMF files when they are
> > installed. 
> 
> Your examples don't use a sufficiently long <locale>.  For example, the user
> might install docs in Chinese, using both EUC and Big 5 encoding.  For that,
> you really need to be able to specify zh_CN.EUC and zh_TW.Big5.
> 
> Note that trying to rely on just zh_CN or zh_TW isn't sufficient for other
> languages, like Japanese, where the various encodings hang off a ja_JP
> prefix.

I don't think long locales pose any problems.

> > Installation is done by first copying the document and OMF file into
> > place and then having ScrollKeeper update its database by calling 
> > 'scrollkeeper-update'.
> 
> Seems reasonable.
> 
> > install:
> > 	install foo-manual.sgml $(DOCDIR)
> > 	install omf-install/foo-manual-fr.omf $(OMFDIR)
> > 	scrollkeeper-update -p $(SCROLLKEEPER_DB_DIR)
> 
> An option to specify exactly where you installed the documentation is
> probably a good idea, so that scrollkeeper-update doesn't have to rescan
> a (potentially) huge directory tree of documentation.

We originally had the location of the document passed, but this became
redundant when we had the build scripts update the OMF file to contain the
real path.

The only directory we scan is OMFDIR.  These files tell us where all the
documentation is, so we never scan documentation trees.

> > -- For use by help browsers --
> > scrollkeeper-get-contents-list <language>:
> >   1) Returns the path:
> >         ${pkglocalstatedir}/<language>/scrollkeeper-contents-list.xml
> 
> Should a lot of this stuff be implemented only as separate programs?  What
> about a scrollkeeper library that applications can link with that provides
> functions that return this information from a config file somewhere.  Having
> to run several applications every time someone starts their help browser may
> not be very efficient.

Sure.  We just thought we'd keep things simple for the first version.

> > scrollkeeper-contents-list.xml:
> > 	This is a simple XML file which reveals what the Contents List
> > 	looks like and where the OMF files and doc are at. This is used
> > 	by the help browser to display the Contents List.
> > 	[We will need to write up a simple DTD.]
> > 
> >    <sect>
> >      <title>Applications</title>
> >      <sect>
> >        <title>Games</title>
> >        <doc docid="000050">
> >          <doctitle>FreeCell</doctitle>
> >          <docomf>/usr/local/games/freecell/freecell-omf.xml</docomf>
> >          <docsource>/usr/local/games/freecell/freecell.sgml</docsource>
> >          <docformat>SGML</docformat>
> >        </doc>
> >      </sect>
> >    </sect>
> 
> <snip>
> 
> > scrollkeeper-extended-contents-list.xml:
> > 	Similar to scrollkeeper-contents-list.xml, except has TOC info in
> > 	it too.  This is used by the help browser to display the
> > 	"extended Contents List".
> 
> What's the rationale for keeping these separate?  Any program iterating
> over their contents should know whether it needs the extended information or
> not, and ignore it, or process it, as necessary.

Right.  We could opt to dump the non-extended list, and always pass the
extended contents list.  Until we get a feeling for how the performance
is, it may be nice to keep the non-extended list since it will be much
shorter.

> The choice of element or attribute in your sample is inconsistent.  Either
> something like
> 
>    ...
>    <doc id="00050" omf="..." source="..." format="sgml">
>      <title>Freecell</title>
>    </doc>
> 
> or
> 
>    <doc>
>      <docid>00050</docid>
>      <title>...</title>
>      ...
>    </doc>
> 
> might be better.  When I first started writing DTDs, I was advised on 
> several mailing lists that if you expect to show content to the end user
> (such as the title) then make it an element.  Anything that is designed to
> help the application (such as the format, or the source) is an attribute.
> 
> Of course, there's a case for saying that "Well, the application will be
> able to show the user an icon for the source format, or the path to the
> underlying document" so it's not a cut and dried decision.  It's probably
> best to consider what the default view would be.

Don't they give any value to aesthetics? Mine is clearly the prettiest ;)

Actually I was following the convention you mention.  Everything the help
browser is supposed to use is an element, and things which the help
browser isn't supposed to use (the docid) is an attribute.  The docid is
only used by ScrollKeeper to make it easy to uninstall a document's
metadata.

Dan

Re: [Scrollkeeper-devel] What about the distributions?

[Eric B. <eb...@cy...>]

Le Dimanche 29 Octobre 2000 23:10, Dan Mueth a écrit :
> On Fri, 27 Oct 2000, Eric Bischoff wrote:
> It is worse, because an admin is willing to wait five seconds or more for
> a package to install, but users don't want to wait 5 extra seconds when
> they open their help browser.

Yes.

Well the 5 extra seconds will be on the *first* time they open the browser. 
The next times it will be much quicker. You don't install tons of packages 
every day.

> > Another solution, if the contents list is dynamic but small, is to create
> > it in the user's home directory. It's a bit of redundant work to
> > scrollkeeper to do this for each user, but it also enables each user to
> > build a contents list that matches the best its inline searches.
>
> I am hesitant to do this, partly because it seems strange and partly
> because it will cause a performance problem. At the same time, I like the
> idea of having this as an option for the future since it would potentially
> allow each user to configure their own Contents List, which would be
> great.  I think the default behavior should not be to replicate itself
> into each user's $HOME directory, but to use the system-wide database
> unless a user explicitly wants to add or remove contents.

Yes.

>  This would be
> feature which we would not implement for a long time anyway.  If and when
> we implement it, we can probably get around the performance problem by
> copying the system-wide database to start with.  However not having a
> system-wide database means that each user gets completely hammered the
> first time they open their browser.

Yup.

> > But at install time it means postinstall scripts. It's technically
> > better, but I can't figure out how distributions will react. It was
> > painful enough to change to LSB directory tree at Caldera.
>
> I agree this is a very important issue.  We have to be confident the
> distributions will accept this.  However changing the directory tree is a
> *much* bigger concern than adding scrollkeeper.

Are you sure? If you count the changed lines in the SPEC files it would even 
be worse.

>  Plus, scrollkeeper can be
> added in little pieces.  Scrollkeeper will be adopted slowly, giving
> distributions time to get familiar with it.  If GNOME 1.4 can use it, as I
> am desperately hoping, then that means that maybe 20 packages will be
> using it in the next few months.  (GNOME 1.4 should be out in about 6
> weeks, if it can stay on schedule ;)  If KDE adopts it for their next
> release along with a few other people, we are looking at maybe 70 packages
> 5 or 6 months from now.  That isn't so much to really swamp anybody.
> Plus, I'm planning that the lines we add to the spec files are completely
> generic.  Basically just cut and paste 4 lines into the spec file and you
> are done.  If you forget, then that doc doesn't show up in the contents
> list. It doesn't really break anything, so I don't think anybody is going
> to scream about a distribution forgetting to add it to a couple spec
> files.

Hmmm if your help browser "forgets" to present you with the results of your 
query it's really bad - once you have a tool you start trusting it.

> > Again, post-install scripts *are* the technical solution, but will upset
> > the distributions. There's no need for you to prove that they are better,
> > we already know it.
>
> My talks with Jonathan and Jeff (via. Jonathan) at Red Hat have indicated
> that they did not feel this would place an undo burden on distributors.
> That is my impression as well, although you (Eric), Jonathan, and Jeff
> have much more expertise in this area than I do.

> > Yes. I think we are in the heart of the problem. Sorry to have started
> > all of this, but it was a mere fact than in a distribution a "make
> > install" is done by the packager, not by the end user.
>
> It is great to have your help.  My original plan would have had many OMF
> files in each spec file and many calls to post-install scripts, which
> would have made quite a bit of work for packagers.

Even if we stick to the post-install script solution, which I do not 
encourage, it's true we did a lot of progress on this front.

> > Let me state the problem the other way round. We won't merge the OMF nor
> > the DocBook files either. So why should we merge the extra data and not
> > these ones?
>
> Aside from all the reasons I've already stated? ;)
>
> The docs and OMF files belong to the individual packages to install and
> uninstall as necessary to remain in sync with the packages.  The
> scrollkeeper database belongs to scrollkeeper, to maintain, update,
> synchronize, modify, and use as scrollkeeper needs.  I think having
> scrollkeeper's database created, shipped, and maintained (or in this case
> *not maintained*) by the applications (in many pieces) instead of
> by scrollkeeper greatly restricts scrollkeeper's ability to do its job.

Do we have a real evaluation of the cost of having a split database?

> Well, I prefer the post-install script, which is why I do not emphasize
> this solution.  However, if we *really* wanted to avoid the script, we
> could do something like: Have a magic directory every package installs a
> file into, say /var/lib/scrollkeeper/packages.  Each package installs a
> short file in here which lists the full path to the OMF file(s) installed
> by the package.  Then whenever scrollkeeper is run it checks to see if
> there are any new, modified, or missing files in this directory.  If so,
> it updates its database.

It's what I'm suggesting for ages. The only real drawback with this solution 
is the delay when you first need the merged database. And I'm not sure it 
will be 5 seconds. Of course there are the other problems like the rights but 
we know there are solutions.

>  The advantage is that it still uses one database
> making things simpler and it can keep the versions correct.  The downside
> is that it still has the permissions and performance problem, it requires
> all packages to install something into a special directory, and it is
> generally hokey.  Plus it has to detect when a file has been modified,
> which means either comparing the contents or checking timestamps, neither
> of which seem pretty.


-- 
Éric Bischoff  -  Documentation and Localization
Caldera (Deutschland) GmbH - Linux for eBusiness
Tel: +49 9131 7192 300 -  Fax: +49 9131 7192 399
http://www.caldera.de/

Re: [Scrollkeeper-devel] What about the distributions?

[Rich M. <rd...@cf...>]

>>  > You can't "make" people do what you want. If they don't want it, they
>>  > won't do it.
>>
>>  Exactly - people have to /want/ to use Scrollkeeper
>
>What do you do for that? Light candles and pray?
>
>Let's be serious. No one will want to adapt thousand of spec files for a new
>help indexing system.

Several counterexamples come to mind.  In the Bad Old Days, most C files
contained ifdefs of the form "#ifdef SunOS", containing semi-repetitive
blocks of code.  This has mostly gone away, thanks to Configure, et al.
Nobody forced folks to adopt the new technology; it simply won because it
served a perceived need and wasn't _that_ hard to use.  Eventually, the
force of public opinion became strong enough to convince many holdouts to
convert.  If you build it RIGHT, they will come.

Also, though this is not a perfect solution, you should allow folks who
are not the original developers to create file characterizations, etc.
This way, any hacker can spend an evening earning a few egoboo points
(and perhaps learning a bit) by annotating an existing package.  I have
some flaky-sounding ideas about formalizing egoboo points, if anyone is
interested...

-r
-- 
--
Rich Morin:          rd...@cf..., +1 650-873-7841, http://www.cfcl.com/rdm
Prime Time Freeware: in...@pt..., +1 408-433-9662, http://www.ptf.com
MacPerl: http://www.macperl.com, http://www.ptf.com/ptf/products/MPPE
MkLinux: http://www.mklinux.org, http://www.ptf.com/ptf/products/MKLP

Re: [Scrollkeeper-devel] What about the distributions?

[Eric B. <eb...@cy...>]

Le Samedi 28 Octobre 2000 00:34, Ali Abdin a écrit :
> > We can build a single one out of the "not-installed-yet" partial
> > databases.
>
> When do we do the "building" though? It is NOT acceptable to do the merging
> whenever we do an API function or whenever the "browser starts".
>
> We "could" make scrollkeeper a daemon (like oafd and gconfd). We could then
> have a 'scrollkeeper_init()' API function that would check if the daemon is
> running (if its not, it would execute it). You could also have a
> is_scrollkeeper_daemon_running or something. I think this is very similiar
> to the way oafd is done.
>
> The daemon would "merge" the not-installed-yet to the "single one" when
> started and every <X> amount of time
>
> This is up to Laszlo though.

If one examines how the existing help environments do, he notices that:
- man pages have a cron daemon which I suppose does some reindexing
- info pages require you to call an install-info utility from the postinstall 
and postuninstall scripts (and this sucks)

No one calls the reindexing function from the browser. This tends to confirm 
what you say.

> > You can't "make" people do what you want. If they don't want it, they
> > won't do it.
>
> Exactly - people have to /want/ to use Scrollkeeper

What do you do for that? Light candles and pray?

Let's be serious. No one will want to adapt thousand of spec files for a new 
help indexing system.

> > > If you want your docs
> > > to utilize scrollkeeper, then you will have to support it.
> >
> > "you" in the above sentence is two different persons.
>
> If you speaking about Debian, the package author is "usually" different
> than the packager.
>
> Unfortunately (or fortunately?) it is usually the opposite with RPMS. The
> author also handles the .spec files

????? This is simply completly wrong.

-- 
Éric Bischoff  -  Documentation and Localization
Caldera (Deutschland) GmbH - Linux for eBusiness
Tel: +49 9131 7192 300 -  Fax: +49 9131 7192 399
http://www.caldera.de/

Re: [Scrollkeeper-devel] ScrollKeeper-0.1 Proposal #3

[Will A. <wi...@ph...>]

On Mon, Nov 13, 2000 at 09:48:25AM +0000, Nik Clayton wrote:
> > The OMF documents are manipulated during the application
> > build process, generally as part of 'make all'.  The purpose of this
> > is to (1) substitute the correct URL of the document where it will
> > actually be installed, which is not generally known until build time,
> > and to (2) extract the OMF from DocBook documents into a seperate file.
> > (NOTE: #2 may not be supported in ScrollKeeper 0.1)
> 
> You probably don't know where the application is going to be installed until
> install time, not build time.  Using the FreeBSD ports tree, it's trivial
> to do
> 
>     cd /usr/ports/foo/bar
>     make
>     make PREFIX=/somewhere/else install
> 
> Similarly, if you're building packages on BSD (or RPMs, or whatever) the
> end user can change the install path after the package has been built, but
> before it has been installed.

On a similar topic..

Personally, I've always hoped that the ScrollKeeper protocol required a
package listing.  This would reduce turnaround time in creating FreeBSD
ports, but probably has the drawback of being difficult to maintain for
the application vendor, if there are a large number of files.

However, it would be a nice thing.. I don't know what they use for debs,
rpms and such, but PLISTs are currently the biggest pain in the ass for
ports maintenance.

Nik, I probably should have talked to you about this at BSDCon... sorry
it didn't cross my mind at the aqarium.  :-(

-- 
wca

Re: [Scrollkeeper-devel] What about the distributions?

[Eric B. <eb...@cy...>]

Le Samedi 28 Octobre 2000 00:38, Ali Abdin a écrit :
> > This will probably be autodetected, thanks to Unix rights system. A
> > package is always built as a normal user the first time. The usual
> > process is then to add the right arguments for buildrooting in the spec
> > file.
>
> No it won't - most SPEC files do not do this!!

I suppose this is a misunderstanding. Yes they do. They need it at least to 
pass the build root to the make files.

-- 
Éric Bischoff  -  Documentation and Localization
Caldera (Deutschland) GmbH - Linux for eBusiness
Tel: +49 9131 7192 300 -  Fax: +49 9131 7192 399
http://www.caldera.de/