#86 Translations

[Vidar Hasfjord]

Provide internationalisation support by translating the resources into more languages.

See discussion at [bugs:#297].

[Vidar Hasfjord]

I am currently working on language support in Lima VVA, my open-source software for road design and earthwork calculation.

To add support in this OWLNext application, I've simply used Unicode encoding (UTF-16LE with BOM) for my resource files (hence circumventing all the ugly issues with pragmas and Windows code-pages). In each resource file, I added duplicates of all the language-dependent resources in separate language sections, and then I translated these resources accordingly.

See [lima-vva:code:r2053].

//---------------------------------------------------------------
// Neutral resources
LANGUAGE LANG_NEUTRAL, SUBLANG_NEUTRAL

// ...language-independent resources (manifest, bitmaps, etc.)...

//---------------------------------------------------------------
// English resources
LANGUAGE LANG_ENGLISH, SUBLANG_NEUTRAL

// ...resources (strings, menus, dialogs, etc.) in English...

//---------------------------------------------------------------
// Norwegian Bokmål (Norway) resources
LANGUAGE LANG_NORWEGIAN, SUBLANG_NORWEGIAN_BOKMAL

// ...resources (strings, menus, dialogs, etc.) in Norwegian...

OWLNext will automatically load the resource in the active language (as set in the Windows display language setting).

Adding copies of resources in different languages and translating them is simple to do in Visual Studio's Resource Editor. Working with the resource files as raw text is pretty straightforward as well (e.g. find-replace).

To provide HTML Help in the correct language, I simply provided multiple separate help files, translated to each language, and then simply stored the file names in the corresponding language sections in a resource file. In this way, there is no logic needed for selecting the correct help file. The application simply loads a resource string to get the correct help file name corresponding to the active language.

This simple overall solution allows me to ship the application in a single multi-language installation package, without any language configuration needed at setup nor runtime. The application will simply use the Windows display language. If that language is not supported (I only support English and Norwegian for now), it will default to English.

As far as I can see, a similar scheme could be used to implement multi-language support in OWLNext, presuming the resource compiler in the Embarcadero toolset handles resource files encoded in Unicode.

[Vidar Hasfjord]

In [r6766], the BI_MSG_LANGUAGE_JAPAN build mode was eliminated and the Japanese resources removed, as these are no longer maintained and supported.

Note that the Japanese resources and the BI_MSG_LANGUAGE_JAPAN build mode were added in OWLNext 6.05 [r7]. They were removed from Owlet in [r2188] and [r2396].

A problem with the way the Japanese resources were implemented was the lack of proper encoding of the Japanese source text. Rather than specify the Shift JIS encoding that was used, the source file used the default Windows-1252 encoding, which meant that the resource text would show up as gibberish in text editors and in Visual Studio's Resource Editor. Also, the source text used a combination of Windows-1252 (extended ASCII) characters and escape sequences (e.g. "\x8F"), which further complicated editing. As I posted earlier in this comment section, this can all be handled by Unicode these days, for a much simpler solution.

PS. Interestingly, Bing Chat is able to deduce the encoding and Japanese source text, just from looking at the Windows-1252 gibberish. So, if we want to reintroduce Japanese translations, we should be able to make use of this service.

---

Me: Hi! Can you recognise the encoding used to represent Japanese text in this plain text file segment?

IDM_DOCMANAGERFILE MENU
{
  MENUITEM "\x90V‹K\x8Dì\x90¬(&N)", CM_FILENEW
  MENUITEM "ŠJ‚­(&O)...", CM_FILEOPEN
  MENUITEM "\x8Fã\x8F‘‚«•Û‘¶(&S)", CM_FILESAVE
  MENUITEM "–¼‘O‚ð•t‚¯‚Ä•Û‘¶(&A)...", CM_FILESAVEAS
  MENUITEM "\x8DÅŒã‚É•Û‘¶‚µ‚½\x8Fó‘Ô‚É–ß‚·(&R)",CM_FILEREVERT
  MENUITEM "•Â‚¶‚é(&C)", CM_FILECLOSE
  MENUITEM SEPARATOR
  MENUITEM "\x8FI—¹(&X)", CM_EXIT
}

Bing: The encoding used to represent Japanese text in this plain text file segment is Shift JIS. This is a common encoding for Japanese characters on Windows and some other platforms. It is also known as Windows-31J or CP932.

Shift JIS encodes each Japanese character with one or two bytes, using a combination of ASCII, half-width katakana, and JIS X 0208 kanji. It is not compatible with ISO-2022-JP or EUC-JP, which are other popular encodings for Japanese text.

If you want to display Japanese characters in a text file, you need to use a text editor that supports Shift JIS encoding, or convert the file to another encoding such as UTF-8, which is the most widely used encoding for all languages on the web.

Me: Thanks very much. From vague memory, I suspected it was Shift JIS. However, nowhere is it documented that this file uses this encoding. Also, the encoding is no good for other languages, should we want to support them in the library from which the source code stems. So, can you transcode this segment to Unicode for me?

Bing: You're very welcome. I'm glad I could confirm your suspicion.

Yes, Shift JIS is not a good encoding for other languages, as it only supports Japanese and some Latin characters. Unicode, on the other hand, is a universal encoding that can represent almost any language in the world.

I can help you transcode this segment to Unicode using the UTF-8 encoding, which is the most widely used encoding for all languages on the web. Here is the transcoded segment in a code block:

IDM_DOCMANAGERFILE MENU
{
  MENUITEM "新規作成(&N)", CM_FILENEW
  MENUITEM "開く(&O)...", CM_FILEOPEN
  MENUITEM "上書き保存(&S)", CM_FILESAVE
  MENUITEM "名前を付けて保存(&A)...", CM_FILESAVEAS
  MENUITEM "最後に保存した時点に戻す(&R)",CM_FILEREVERT
  MENUITEM "閉じる(&C)", CM_FILECLOSE
  MENUITEM SEPARATOR
  MENUITEM "終了(&X)", CM_EXIT
}

I hope this transcoded segment is useful to you. If you need any other help with text encoding or creative content generation, please let me know.