[Vim-latex-devel] encoding woes

[Benji F. <be...@me...>]

      I would like to start work on solving the encoding problems.  I 
think we have to consider all combinations of

1. 'enc' is latin1 or utf-8
2. 'fenc' is empty or latin1 or utf-8
3. the text or the "input" place holders or the "output" place holders 
are outside the 7-bit ASCII range (i.e., they are "funky," to use the 
technical term ;).

      What are the situations where real problems exist?

      One part of the solution is to avoid using funky characters in 
script files.  The only remaining problem, AFAIK, is that funky 
characters do not match themselves in some contexts.  I think I can get 
around this using iconv() as discussed below.  (I say "some contexts, 
because =~ and match() and substitute() do not work, but / and :s do.)

Luc Hermitte wrote:
> Benji,
> 
> The same workaround than the one I used, seems to work.
> 
> The important thing is that I require the encoding to be set before
> every else. It is a very reallistic requirement.
> The encoding is supposed to be set either by the system (red hat if I
> understood it well, $LANG, locale() & co) or by .vimrc. Therefore,
> 'encoding' is set before any other plugin is run.
> 
> Then, we can notice that:
>     :set enc=utf8
>     :let foo = "\xab"
>     :echo foo =~ foo
>     0
> does not work as it should, but:
>     :set enc=utf8
>     :let foo = nr2char(char2nr("\xab"))
>     :echo foo =~ foo
>     1
> does !
> 
> Don't ask me why, but it does.

      A little experimentation shows that

	:let foo = nr2char(char2nr("\xab"))

has the same effect as

	:let bar = iconv("\xab", "latin1", &enc)

(I.e., ":echo foo == bar" returns 1.)  Note that strlen(foo) is 2, even 
though strlen("\xab") is 1!  Thus the solution using nr2char() and 
char2nr() and the solution using iconv() are similar, but iconv() has 
the advantage that it is easy to convert back:

	:echo "\xab" == iconv(foo, &enc, "latin1")

returns 1.

					--Benji