Re: clisp/utils/unicode & gnulib/lib/unicase

[Bruno H. <br...@cl...>]

Hi Sam,

> any chance you could replace clisp/utils/unicode with an import from gnulib?

My last answer to this question, on 2010-09-15, was the following.
It is still valid today.

> that said, I wonder if I could kill clisp/utils/unicode/ too.
> maybe you could move all those generated nls_*.c files to gnulib?

1) gen-ctype.c
This is used to generate uni_upcase.c, uni_downcase.c, uni_attribute.c.

Instead of uni_upcase.c you can use the gnulib module 'unicase/toupper'.
Instead of uni_downcasee.c you can use the gnulib module 'unicase/tolower'.
But be careful to think about how to preserve this:

/* Common Lisp only wants the bijective lower/upper case conversions.  */
static bool
is_CL_both_case (unsigned int ch)
{
  unsigned int ch1 = to_upper (ch);
  unsigned int ch2 = to_lower (ch);
  return (ch1 == ch || ch2 == ch)
         && (ch1 != ch2)
         && to_upper (ch1) == ch1
         && to_upper (ch2) == ch1
         && to_lower (ch1) == ch2
         && to_lower (ch2) == ch2;
}

Unicode's lower/upper case conversions are not bijective. But Common Lisp's
are.

Instead of uni_attribute.c you can use the gnulib modules 'unictype/ctype-alpha'
or 'unictype/property-alphabetic' (which one, you need to find out by testing),
and 'unictype/category-Nd'.

The obvious benefit of switching to gnulib will be immediate support for
Unicode 5. The drawback will be that it's a bit slower to make 2 or 3 calls to
uc_toupper or uc_tolower than to access a precomputed table.

2) nls_table.c

As you know, these nls_* files implement a more efficient character conversion
than iconv can do. gnulib bets on POSIX, and POSIX specifies iconv(). There is
not enough reason to add special character set converter pieces to gnulib,
because 99% of the applications don't need so much speed. Only clisp (and SBCL
and similar) need the extra speed that you obtain by
  1. assuming a 1 byte = 1 character behaviour,
  2. using a direct access to the conversion table.

There's no maintenance needed on these files: The mapping tables don't change.
So I would leave nls_table.c as it is.

Bruno
-- 
In memoriam Jeane Gardiner <http://en.wikipedia.org/wiki/Jeane_Gardiner>