Re: [Ocamlnet-devel] an experimental impelmentation of Unicode support.

[Patrick M D. <pa...@wa...>]

On Fri, 16 Nov 2001, Gerd Stolpmann wrote:

>
> On 2001.11.15 17:32 YAMAGATA yoriyuki wrote:
> > I'm interested in a plan about internationalization support of
> > ocamlnet.  Is it just out of the scope?
>
> It's not, but the need for internationalization within ocamlnet is
> quite limited. Today it is possible for many network protocols to
> specify a certain character set, and good implementations will
> recognize the character set and signal it to the user, but that's it
> already for the protocol library. For example, there are several
> mechanisms for emails to specify the character set (one for the
> message body, and two for the message header (RFC2047 and RFC2184)).
> It is enough for the protocol library to switch to pairs
> (character_set X contents)  instead of using strings with undefined
> encodings. In Mimestring, RFC2047 has been implemented by adding
> EncodedWord as a triple (character_set X transfer_encoding X
> contents). The worst case is that it is necessary to convert such
> strings to an ASCII-compatible encoding in the case it contains some
> keywords to be recognized. Of course, if you write a mail user agent
> you have to do more, but this is out of scope of ocamlnet.

This is a good example to discuss. While developing an e-mail user agent,
I needed to handle encoded content in many headers (e.g. addresses). The
address parsing code in Ocamlnet is based on that e-mail agent. My design
was to keep the address parsing code ignorant with regard to encoded
content.

This design simplified the parser and data structures, as well as code
that wants to manipulate the addresses. When my GUI wants to finally
display an address, it uses Mimestring again to handle the encoding.

Now, if there had been good support for Unicode, I might have adopted a
different design with the parser being aware of EncodedWords. Both
approaches do seem to have their advantages.

> Do you know a certain protocol that requires more?

I'm not aware of any, at least within the scope of what I feel that
Ocamlnet should cover (i.e. IETF-like protocols and specifications). This
might become much more of an issue if we were to look at W3-like
protocols, but that is probably best suited for a different project.

> So I think ocamlnet should only contain the very basic definitions:
>
> - A type enumerating the available encodings
>
> - Conversion between encodings
>
> It is unlikely that we need more for ocamlnet, and these definitions
> may also be seen as the minimal interface to other libraries that
> support several encodings, too. As the type for the internationalized
> string I would suggest (encoding * string).

I agree. How feasible would it be to add the Japanese encodings to the
Netmappings table?

Patrick