#3 Non-english language characters are not displayed correctly

[Chris Smith]

Non-english language characters are not displayed
correctly. This is because these characters need to be
converted to HTML entities similar to non-breaking
spaces or quotation marks.

[mcfang]

Logged In: YES
user_id=896478

I believe that the data is encoded correctly, but the HTML
charset breaks it as it is locked as UTF-8.

I've found that by overriding the browser's encoding to the
relevant setting (for example: Shift-JIS for japanese) shows
the correct characters.

Possible fixes:

1) remove the charset specifier from the HTML code, this
might allow the browser to auto-detect the encoding.

2) set a user option in the config to specify the HTML encoding.

3) remove all charset specifies from HTML, and place in the CSS.
( @charset "utf-8"; ). Here at least the user can edit it
manually.

[Chris Smith]

Logged In: YES
user_id=1025088

We can definitely fix most of the problems by just doing the
proper HTML translation of the characters and using UTF-8 as
the encoding (i've had this working in a development version
before).

That said, it would be difficult to fully support all the
HTML entities (there appear to be hundreds for asian
languages). It would be a good idea to make thie encoding
scheme a preference as suggested.

I don't think option 1 or 3 are really viable as they lead
to non-validating pages.

[Chris Smith]

Logged In: YES
user_id=1025088

Partial fix should be in the next version: all ASCII
characters (0->255) should be handled properly.

Will include support for setting the encoding in preferences
at a later date

[Chris Smith]

Logged In: YES
user_id=1025088

Support has been added for setting the language and encoding
in the preferences.