Re: [Scrollkeeper-devel] omf encoding

[John F. <jf...@in...>]

On Thu, 2002-05-09 at 20:50, Malcolm Tredinnick wrote:
> On Thu, May 09, 2002 at 06:45:33PM -0600, John Fleck wrote:
> > Unless the document's encoding is declared otherwise, libxml (and
> > therefore ScrollKeeper) assumes UTF-8:
> > 
> > http://xmlsoft.org/encoding.html
> > 
> > "If there is no encoding declaration, then the input has to be in either
> > UTF-8 or UTF-16, if it is not then at some point when processing the
> > input, the converter/checker of UTF-8 form will raise an encoding error.
> > You may end-up with a garbled document, or no document at all !"
> > 
> > This means that in the case of OMF files in other encodings (like the
> > one for ggv that started this discussion) we need to specify the
> > encoding in the xml declaration.
> > 
> > The encodings that seem to be of importance to us:
> > 
> >    1. UTF-8 is supported by default (null handlers)
> >    2. UTF-16, both little and big endian
> 
> Where is UTF-16 being used commonly? Asian languages I know about (so
> they tend to prefer local encodings at the moment), but are there any
> others.
> 
> >    3. ISO-Latin-1 (ISO-8859-1) covering most western languages
> 
> We'll need to be aware of all iso-8859, not just iso-8869-1, since many
> European languages need them. For example, Czechs and the Slovaks use
> 8859-2, a lot of the Scandanavian countries use 8859-6, etc. Unless you
> add the requirement that everything must be UTF-8, then we just need to
> be able to do this generally (the Japanese and Chinese translators might
> like us more, then, too).
> 
> And that's all assuming you don't want the Euro symbol, and ... oh ...
> it's all getting too hard ... I need to go lie down now. :-(
> 

Well, unless we're particularly industrious, we're restricted to the
encodings supported by libxml2, which are the ones I listed above plus a
couple that didn't seem particularly relevant. The full list*:

   1. UTF-8 is supported by default (null handlers)
   2. UTF-16, both little and big endian
   3. ISO-Latin-1 (ISO-8859-1) covering most western languages
   4. ASCII, useful mostly for saving
   5. HTML, a specific handler for the conversion of UTF-8 to ASCII with
HTML predefined entities like © for the Copyright sign.

* http://www.xmlsoft.org/encoding.html

Beyond that, we'll have to write our own encoding converters.

Cheers,
John
-- 
John Fleck
jf...@in... (h) jf...@ab... (w)
http://www.inkstain.net http://www.abqjournal.com

"You don't want to die with the music still in you."
 - John Gardner