Unicode in libfaeutil

[Justin F. <vh...@us...>]

Back to life.

Several functions for working with Unicode were added to libfaeutil. 
This includes simple functions for converting characters and string from 
UCS-2 to UTF-8 and back, and "heavier" functions to work with specific 
charset mappings.

The main part of the code was taken from Faerion, of course.  There are 
only four functions that work with mappings now, without any structures 
and constants.  Two functions to load/unload an encoding (identified by 
a void* handle), two functions to convert from the encoding to UCS-2 and 
back.  Note that the intermediate form is no longer UTF-8, which was 
rather ineffective.  Neither is UTF-8 a mapping, as it was treated in 
Faerion.  This means that applications that want to use this part of 
libfaeutil need to distinguish between UTF-8 and a generic mapping. 
They also need to implement transliteration on their own (which is, 
however, rather simple).

Libfaeutil is also not a stand-alone library anymore.  It is a 
collection of files to cut'n'paste from.  Just a heap of files collected 
in a single place.  The reason for this change is the fact that some 
developers dislike the idea of including a dependency for what was 
originally meant to be a collection of back-up functions (e.g., invoked 
when the dependant functions fail to exist).  Being a dependency itself 
kind of mitigates this revenue.

Libfaeutil now has a documentation, called README.  It's a RFC-like 
document with detailed descriptions and example.  Tired of DocBook at 
the moment. ;)

Back to Faerion, I think I'll try moving it to the new Unicode functions 
soon.  Perhaps I'lll be working on enabling national characters in 
nicknames.  This means that certain amount of data needs to be stored in 
Unicode now (UCS-2, not even UTF-8).  This also means we'll need a 
tranlation table to convert all similarly looking characters (e.g., the 
letter "e" exists in both English and Russian code pages, with different 
codes) to prevent abuses.

Nothing overly complex.

Some sleep now.