Re: [Cheetahtemplate-discuss] Exception with Unicode data

[Rene P. <re...@la...>]

Tavis Rudd:
>thanks for the patch.  I'm about to add your new filter to the=20
>cvs. Can you test it once added to make sure it works as expected?

Sure, I'll be glad to give it a try.

>I think we should either use it as the default filter or add some=20
>more explicit notes to the Users Guide about unicode support.

I can't understand why unicode would not be supported by
default. It's completely integrated into Python so I expected it
to "just work". An exception in the first "hello world" attempt
is scaring newbies off unnecessarily.

One might argue that Cheetah doesn't know the encoding the user
wants to use. This is true. But Cheetah also doesn't know wether
the user wants ASCII-encoding, decimal numbers, the default date
format and so on, so by default it shouldn't print any data at
all and just raise exceptions :-)

My patch defaulted to utf-8, since Cheetah is primarily used to
generate web pages and utf-8 is the recommended encoding on the
web.

"Note that UTF-8 can be used for all languages and is the
recommended charset on the Internet. Support for it is rapidly
increasing."
http://www.w3.org/International/O-charset-lang.html

See also:

"UTF-8 and Unicode FAQ for Unix/Linux"
http://www.cl.cam.ac.uk/~mgk25/unicode.html

The RFC: "UTF-8, a transformation format of ISO 10646"
http://www.ietf.org/rfc/rfc2279.txt

My patch defaults to utf-8, and this can be overridden with the
'encoding' keyword, per substitution. Perhaps it would be useful
to allow the user to change the default when instantiating the
filter. But since this was my first attempt to write a Cheetah
filter I wanted to keep it simple. Perhaps someone can add this
enhancement (if needed).

>If anyone is interested in writing up a new unicode section for=20
>the guide I can give them cvs access to do so.

I'll write down what I encounter while developing my
multilingual unicode website with Cheetah and post it on this
list. But since English is a second language for me, someone
will probably have to edit it a bit before it goes in the users'
guide.

--=20
Regards / Groeten,  http://www.leren.nl
Ren=E9 Pijlman        http://www.applinet.nl
 =20