[Indic-computing-devel] NCST IndiX examined

[<jk...@Fr...>]

NCST has, at [http://rohini.ncst.ernet.in/indix/], a modified XFree86
distribution that attempts to provide support for Indic glyph
rendering over X11.

[[ This is a 28MB download. I wonder why they didn't (also) put up a
   "diff" wrt. XFree86 sources instead. ]]

IMO, the plus points of this work are:

    - there is source code for an Indic shaping engine available for
      study, for tamil, gujarati, and devanagari [xc/lib/indic/]. This
      has been cleanly separated into a library that can be used in
      other contexts.

    - the changes seem to have been done by someone with an
      understanding of the way the X Window System is built.  Indic
      language support gets "turned on" if the system is compiled with
      -DINDIC_SUPPORT, and this can be controlled system-wide using
      the standard X configuration mechanism.

    - the changes are complete, in that most of the places that needed
      to be changed, have been changed.

    - the code was a pleasure to read.

Thank you for putting your work up, NCST!

The NCST changes are, unfortunately, intrusive, and break the
semantics of the X Window system protocol.

The problem with the NCST design arises from confusion over character
codes and the glyph indices used in X11 text drawing calls.

In the X11 design, the X protocol does NOT assign any semantic meaning
to the glyph indices that are passed to a PolyText or ImageText
request.  From the X protocol specification:
   
    ``Font: A font is a matrix of glyphs (typically characters).  The
      protocol does no translations or interpretation of character
      sets.  The client simply indicates values used to index the
      glyph array.  A font contains additional metric information to
      determine interglyph and interline spacing.''
     
     X Protocol Specification, Glossary, pp 154.

In the NCST work however, all text strings fed into X11 text calls are
assumed to be UNICODE character streams encoded in UTF-8 format.

They then have a complicated sequence of checks and conversions that
happen when rendering text on-screen.

The NCST system cannot be considered an implementation of the X Window
System protocol.  Applications using the NCST X library will not work
correctly on other X servers and applications compiled on other
systems will not work correctly on the NCST X server.

>> The details:

Clients call `XDrawString()' and `XDrawImageString()' as usual.

The X server interprets the strings passed as ranges UNICODE
characters encoding according to UTF-8.  Fonts are associated with
Unicode ranges that they cover.  The X server, when servicing a text
drawing request, checks whether the requested characters fall in the
``Indic'' character ranges.  If so, special processing is triggered
including grouping characters into ``syllables'', re-ordering of
glyphs etc.

The NCST X server maintains a global list of ``Indic Fonts'' pointed
to by `pFirstIndicFont', which seemed to be used as fall-back fonts.

There are number of new changes in the device independent code (e.g:-
changes in "dix.c", "main.c", new globals `FontPtr devanagariFont'
etc).

Compatibility of clients using the NCST X11 library with `stock' X
servers is broken because of a change to XQueryTextExtents(): in the
NCST system, the text string sent over to the X server is assumed to
be in UTF-8 format and is first converted to UCS-2 by their X11
library.  Thus the bytes (in UCS-2 format) that get sent out will be
quite different from what the client passed in.  The NCST X server
will deal correctly with this UCS-2 encoded data, but stock X servers
will not.

"xc/lib/indic/" contains sources for an OpenType capable rendering
engine based on FreeType.  I have not reviewed this code, since I do
not at this point have a clear idea of how OpenType rendering works.

>> Summary

Nice system, nice code; unfortunately not compatible with the X Window
System protocol.

Regards,
Koshy
<jk...@fr...>