scrollkeeper-devel

[Scrollkeeper-devel] Plan for Category Tree for ScrollKeeper >= 0.4

[Dan M. <da...@ea...>]

Hi,

I have been thinking a bit about the category tree and how we can should
move forward.  Here are a few of my thoughts for what we should do between
now and ScrollKeeper 0.4:

1) Avoid changing any existing categories which would break compatibility
with existing OMF files.  We can happily break this rule if we decide to
do #3 below, after we implement it :)

2) Reasonable i18n.  We should use xml-i18n-tools or some nice way of
making translators lives easier.  This may include:

  A) po files to make translation of categories easy

     * this makes the lives of translators much easier and happier

  B) mapping between C and non-C locales for categories.  This is probably
     required in our new po-file technique of i18n.  (Note that now, the C
     and non-C locales could have completely unrelated entries in theory.)

     Question: What should we use for our key?  Should it just be the 
     category in C?  Or should we have a seperate attribute containing
     some sort of a key?

     * This (along with document mappings) allows us to merge
       the category trees for two locales.  Suppose for example, that
       there is a document installed only in C.  Using the document 
       mapping, we could identify that it is not available in locale <xx>
       and determine which localized category the C doc belongs under.
       This is a feature we really need in 0.4, and the first feature
       request I heard from translators.

  C) OMF-file category compatibility between C and non-C locales. So if an
     OMF file for locale <xx> has a category of "string", then it tries to
     match the "string" with the localized category tree as well as with
     the C tree.  This way people don't have to translate a category in
     the OMF file if they don't want to, and everything will still work.

     * This is just a practical issue.  Sometimes people forget to
       translate the category, so this is extra rubustness. Plus, life
       is easier if we have less stuff to translate, so we may even
       encourage people to not translate the category for the OMF file
       if they don't want to.

3) Backwards compatibility in the category tree.  Ideally we would have a
perfect category tree and never change it.  That is not reality.  Really,
we will get very frequent requests for changes to be made.  These
potentially break existing OMF files.  However, I think I have a
reasonable way to change the category system while remaining backwards
compatible:

Conceptually we would need a set of instructions for what the obsoleted
categories are and how they should be mapped. For example, suppose we used
to have a category "GNOME|Applets|Utility" and we changed the name to
"GNOME|Applets|Utilities", we would specify (in a seperate XML file
probably - perhaps scrollkeeper_cl_aliases.xml) that anything that says it
is in "GNOME|Applets|Utility" should be placed into
"GNOME|Applets|Utilities".

The file scrollkeeper_cl_aliases.xml might look like:
<?xml version="1.0"?>
<ScrollKeeperContentsAliasList>
 <omfalias>
  <oldcategory>
   <sect>
    <title>GNOME</title>
    <sect>
     <title>Applets</title>
     <sect>
      <title>Utility</title>
     </sect>
    </sect>
   </sect>
  </oldcategory>
  <oldcategory>
   <sect>
    <title>GNOME</title>
    <sect>
     <title>Applets</title>
     <sect>
      <title>Utility</title>
     </sect>
    </sect>
   </sect>
  </oldcategory>
 </omfalias>
</ScrollKeeperContentsAliasList>

This is rather bulky to type, but should do the job nicely.  We really
need to do this in such a way that translators don't have to manually
translate this.  This can either be done by using the translated terms in
po files, since they are already translated, or possibly by using the
correspondence system above.  This will require a bit more thought.

4) Introduce the proper XML notation for the OMF files for categories,
instead of using our pipe-delimeted form which is lame.  We should remain
backward compatible with the pipe form, but strongly discourage it since
it is inconsistant with the XML concept.

5) Start making and publishing official ScrollKeeper category trees.  They
should be on the web page in an easy-to-read and easy-to-use form, with
versioning info.

6) Get the OMF specification and DTD updated to reflect the category
child to SUBJECT

7) (not really category-specific) Look into character encoding issues and
compatibility with the new (as-yet-unreleased) libxml.

This is a lot of stuff, so some of it may have to wait for ScrollKeeper
0.6, depending on how long it takes to get this done.  

Thoughts?

Dan

Re: [Scrollkeeper-devel] Plan for Category Tree for ScrollKeeper >= 0.4

[Laszlo K. <las...@Su...>]

Dan Mueth wrote:
> 
> Hi,
> 
> I have been thinking a bit about the category tree and how we can should
> move forward.  Here are a few of my thoughts for what we should do between
> now and ScrollKeeper 0.4:
> 
> 1) Avoid changing any existing categories which would break compatibility
> with existing OMF files.  We can happily break this rule if we decide to
> do #3 below, after we implement it :)
> 
> 2) Reasonable i18n.  We should use xml-i18n-tools or some nice way of
> making translators lives easier.  This may include:
> 
>   A) po files to make translation of categories easy
> 
>      * this makes the lives of translators much easier and happier
xml-i18n-tools means one tree that holds the docs for every locale and
all the branch translations. Might be a good idea, but it might
introduce some overhead. If we dont want the browser to filter out the
docs for the current locale then the content tree retrieval scripts will
have to do this. Then we might need to return the XML tree as
represented by libxml in memory rather than a p[ointer to the content
tree file itself. I dont think reading in the big XML tree, filtering
out the current locale, writing that out to a temporary file and
returning that is a good idea because it will be slow. 

> 
>   B) mapping between C and non-C locales for categories.  This is probably
>      required in our new po-file technique of i18n.  (Note that now, the C
>      and non-C locales could have completely unrelated entries in theory.)
> 
>      Question: What should we use for our key?  Should it just be the
>      category in C?  Or should we have a seperate attribute containing
>      some sort of a key?
> 
>      * This (along with document mappings) allows us to merge
>        the category trees for two locales.  Suppose for example, that
>        there is a document installed only in C.  Using the document
>        mapping, we could identify that it is not available in locale <xx>
>        and determine which localized category the C doc belongs under.
>        This is a feature we really need in 0.4, and the first feature
>        request I heard from translators.
Are you suggesting a document mapping and a tree category mapping
database? It is certainly possible to implement it and it would easily
allow us to find the various translations of a doc and add the English
instead of the translated one in any locale. The document ID is needed
to be implemented for this.

> 
>   C) OMF-file category compatibility between C and non-C locales. So if an
>      OMF file for locale <xx> has a category of "string", then it tries to
>      match the "string" with the localized category tree as well as with
>      the C tree.  This way people don't have to translate a category in
>      the OMF file if they don't want to, and everything will still work.
> 
>      * This is just a practical issue.  Sometimes people forget to
>        translate the category, so this is extra rubustness. Plus, life
>        is easier if we have less stuff to translate, so we may even
>        encourage people to not translate the category for the OMF file
>        if they don't want to.
I dont understand everything above. I guess you are suggesting that with
the above mentioned category mapping we can install a translated doc in
the tree into the right locale position even if the OMF category fields
are not translated? Why would one want to match the translated doc with
the C tree?

> 
> 3) Backwards compatibility in the category tree.  Ideally we would have a
> perfect category tree and never change it.  That is not reality.  Really,
> we will get very frequent requests for changes to be made.  These
> potentially break existing OMF files.  However, I think I have a
> reasonable way to change the category system while remaining backwards
> compatible:
> 
> Conceptually we would need a set of instructions for what the obsoleted
> categories are and how they should be mapped. For example, suppose we used
> to have a category "GNOME|Applets|Utility" and we changed the name to
> "GNOME|Applets|Utilities", we would specify (in a seperate XML file
> probably - perhaps scrollkeeper_cl_aliases.xml) that anything that says it
> is in "GNOME|Applets|Utility" should be placed into
> "GNOME|Applets|Utilities".
> 
> The file scrollkeeper_cl_aliases.xml might look like:
> <?xml version="1.0"?>
> <ScrollKeeperContentsAliasList>
>  <omfalias>
>   <oldcategory>
>    <sect>
>     <title>GNOME</title>
>     <sect>
>      <title>Applets</title>
>      <sect>
>       <title>Utility</title>
>      </sect>
>     </sect>
>    </sect>
>   </oldcategory>
>   <oldcategory>
>    <sect>
>     <title>GNOME</title>
>     <sect>
>      <title>Applets</title>
>      <sect>
>       <title>Utility</title>
>      </sect>
>     </sect>
>    </sect>
>   </oldcategory>
>  </omfalias>
> </ScrollKeeperContentsAliasList>
> 
> This is rather bulky to type, but should do the job nicely.  We really
> need to do this in such a way that translators don't have to manually
> translate this.  This can either be done by using the translated terms in
> po files, since they are already translated, or possibly by using the
> correspondence system above.  This will require a bit more thought.
Sounds good, it has to allow more than one old category mapped to the
current new category. 

> 
> 4) Introduce the proper XML notation for the OMF files for categories,
> instead of using our pipe-delimeted form which is lame.  We should remain
> backward compatible with the pipe form, but strongly discourage it since
> it is inconsistant with the XML concept.
yes.

> 
> 5) Start making and publishing official ScrollKeeper category trees.  They
> should be on the web page in an easy-to-read and easy-to-use form, with
> versioning info.
yes.

> 
> 6) Get the OMF specification and DTD updated to reflect the category
> child to SUBJECT
yes.

> 
> 7) (not really category-specific) Look into character encoding issues and
> compatibility with the new (as-yet-unreleased) libxml.
yes. DV should be pulled into this.

> 
> This is a lot of stuff, so some of it may have to wait for ScrollKeeper
> 0.6, depending on how long it takes to get this done.
> 
> Thoughts?

As I said to Dan before, priorities in the Sun Gnome team were
reshuffled and I will be able to spend about 30% of my time on
Scrollkeeper (until know this was 100%). So we will need to take design
related decisions much faster in order to get things to the stage when
they can be implemented considering that only myself and another Sun
engineer are implementing features in Scrollkeeper right now.

Laszlo

Re: [Scrollkeeper-devel] Plan for Category Tree for ScrollKeeper >= 0.4

[John F. <jf...@sw...>]

On Fri, Mar 30, 2001 at 01:12:53AM -0800, Dan Mueth wrote:
> 
>   B) mapping between C and non-C locales for categories.  This is probably
>      required in our new po-file technique of i18n.  (Note that now, the C
>      and non-C locales could have completely unrelated entries in theory.)
> 
>      Question: What should we use for our key?  Should it just be the 
>      category in C?  Or should we have a seperate attribute containing
>      some sort of a key?
> 

An attribute with a key would be more complicated to implement, but
would guarantee uniqueness.

>      * This (along with document mappings) allows us to merge
>        the category trees for two locales.  Suppose for example, that
>        there is a document installed only in C.  Using the document 
>        mapping, we could identify that it is not available in locale <xx>
>        and determine which localized category the C doc belongs under.
>        This is a feature we really need in 0.4, and the first feature
>        request I heard from translators.
> 

This would be of great value.

>   C) OMF-file category compatibility between C and non-C locales. So if an
>      OMF file for locale <xx> has a category of "string", then it tries to
>      match the "string" with the localized category tree as well as with
>      the C tree.  This way people don't have to translate a category in
>      the OMF file if they don't want to, and everything will still work.
> 
>      * This is just a practical issue.  Sometimes people forget to
>        translate the category, so this is extra rubustness. Plus, life
>        is easier if we have less stuff to translate, so we may even
>        encourage people to not translate the category for the OMF file
>        if they don't want to.
> 

This would solve a major problem I have had in creating omf files, but
authors writing in other than English should be able to use their own
language for the category name.

> 
> 4) Introduce the proper XML notation for the OMF files for categories,
> instead of using our pipe-delimeted form which is lame.  We should remain
> backward compatible with the pipe form, but strongly discourage it since
> it is inconsistant with the XML concept.
> 

Could you give an xml example of what this would look like?

> 
> 7) (not really category-specific) Look into character encoding issues and
> compatibility with the new (as-yet-unreleased) libxml.
> 

This probably needs to be done right away, doesn't it, so we are ready
for the release of the new libxml?

Cheers,
-- 
John Fleck
jf...@sw... (h)
jf...@ab... (w)
http://www.abqjournal.com/scitech/