National language keyboards and Tk

Logged In: YES
user_id=68433

Notes (from conversations on the tcl'ers chat with zly):
this seems to affect all Tk applications, not just aMSN. It
does not appear to be SCIM related. Tk 8.5, 8.4.11, and
8.4.13 are all affected (and I'm guessing the problem goes
back further than that).

One user reports that the problem appeared when upgrading
Ubuntu from Dapper to Edgy -- that's probably an important clue.

Logged In: YES
user_id=1336486
Originator: NO

The problem persists as of Tcl/Tk 8.5a6

However, to puzzle the problem further:
Everything about my system is set up to use UTF-8, if i start wish or another tk app (in this case amsn) defining e.g. iso8859-15 lang like this:
>LANG=ISO-8859-15 amsn
and at the same time configures amsn to use utf-8, the input starts working.

I can live with it like this, but it still puzzles me, what exactly goes wrong when system is set to UTF-8.

Logged In: YES
user_id=68433
Originator: YES

I might finally have a lead on this. Unfortunately I've not been able to replicate the problem on any machine here (Debian sarge, Ubuntu 7.04, CentOS 5.1, various Fedora systems; tried with various locales.)

Attached program ximinfo.c prints some potentially-useful information that might help to track this down further.

File Added: ximinfo.c

ximinfo.c - list available input styles. Build instructions in header comments.

Logged In: YES
user_id=1336486
Originator: NO

Thanks for commenting on this again jenglish,

My system has changed quite a lot over this period of time, and a part of the original problem no longer applies to me.
That is, Tk applications now *does* accept the input from (most?) UTF-8 locales, but not the one I use in particular.
And the program you attached, leads to why this might be:
./ximinfo: locale=en_DK.UTF-8
./ximinfo: locale en_DK.UTF-8 not supported

My guess would be that Tk handles unsupported locales from the X server in a different way than other toolkits, since those others still let me input the characters.

Changing to a locale which is supported *does* let me input the characters in Tk, and ximinfo yields the following:
./ximinfo: locale=en_US.UTF-8
./ximinfo: XSetLocaleModifiers -> @im=local
./ximinfo: style XIMPreeditNone|XIMStatusNone
./ximinfo: style XIMPreeditNothing|XIMStatusNothing

FYI my system is now openSUSE 10.3 (x86) with the following package version:
tcl-8.4.15
tk-8.4.15
xorg-x11-7.2

Logged In: YES
user_id=68433
Originator: YES

The first line:

./ximinfo: locale=en_DK.UTF-8

indicates that your system has LIBC support for en_DK.UTF-8 (if it didn't, ximinfo would have printed "locale=(null)" instead).
The next line means that the X system doesn't think it supports that locale. (It might be client-side or server-side; the man page for XSupportsLocale() is unclear.)

Tk doesn't even bother to test XSupportsLocale() (and it's possible that other toolkits don't either). Could you delete the "return EXIT_FAILURE" line from ximinfo.c and run it again with your preferred locale?

What is XMODIFIERS normally set to? Is it usually "@im=local", or do you need to change this specially for Tk apps?

I can confirm that Gtk+ and other toolkits are doing _something_ different from Tk. I can't tell what it is though.

Logged In: YES
user_id=1336486
Originator: NO

./ximinfo: locale=en_DK.UTF-8
./ximinfo: locale en_DK.UTF-8 not supported
./ximinfo: XSetLocaleModifiers failed

I guess XMODIFIERS is usually "@im=local"
since all I did on the previous run was LANG=en_US.UTF-8 ./ximinfo

ximinfo.c - version 2

Logged In: YES
user_id=68433
Originator: YES

File Added: ximinfo.c

Logged In: YES
user_id=1336486
Originator: NO

[zly@GizMo]~>./ximinfo
./ximinfo: locale=en_DK.UTF-8
./ximinfo: codeset=UTF-8
./ximinfo: WARNING: locale en_DK.UTF-8 not supported
./ximinfo: WARNING XSetLocaleModifiers failed
./ximinfo: XSetLocaleModifiers -> (null)
./ximinfo: XOpenIM failed

[zly@GizMo]~>LANG=en_US.UTF-8 ./ximinfo
./ximinfo: locale=en_US.UTF-8
./ximinfo: codeset=UTF-8
./ximinfo: XSetLocaleModifiers -> @im=local
./ximinfo: style XIMPreeditNone|XIMStatusNone
./ximinfo: style XIMPreeditNothing|XIMStatusNothing

Logged In: NO

There seems to be problems with the en_DK locale in general;

QT is now also affected (kwrite, kate, etc.) (cannot write accented characters in qt apps)
Java apps (zendstudio) (same as above)
OpenOffice has problems (filed a bugreport some time ago: http://www.openoffice.org/issues/show_bug.cgi?id=82579\)

All of the above (and amsn) works when defining LANG=en_US.UTF-8

GTK(1/2/+) has no problems.

Logged In: YES
user_id=68433
Originator: YES

The output from ximinfo:

> ./ximinfo: locale=en_DK.UTF-8
> ./ximinfo: codeset=UTF-8

indicates that your libc has support for that locale, but:

> ./ximinfo: WARNING: locale en_DK.UTF-8 not supported

... but your X server doesn't.

Look in /usr/lib/X11/locale (or /usr/share/X11/, or /usr/X11R6/, or wherever your distro puts it). There should be one directory for each supported locale. See if there's an en_DK.UTF-8 directory (there's probably not).

There's also a locale.alias file, which adds support for additionale locale names by mapping them to existing ones. Check if en_DK.UTF-8 appears there too (again, probably not, otherwise you wouldn't be seeing this error...)

On Debian systems, you can (must) use the `locale-gen` command to add support for additional locales. Most other Linuxes I've seen install all locales by default, or split them up into separate packages. Don't know about other Unices.

Logged In: YES
user_id=1336486
Originator: NO

Oh my, this has finally been solved.

Looking at /usr/share/X11/locale/ I see a lot fewer locales than those supported by libc.
In my case, there's also locale.alias, locale.dir, and compose.dir.
locale.alias seems to be aliases of the locales (hint taken)
locale.dir was the mapping file, and almost all *.UTF-8 locales was mapped to en_US.UTF-8,
but I could not find the en_DK.UTF-8 mapping. This was also the case for compose.dir.

I added the lines;
locale.dir:
en_US.UTF-8/XLC_LOCALE en_DK.UTF-8
en_US.UTF-8/XLC_LOCALE: en_DK.UTF-8

compose.dir:
en_US.UTF-8/Compose en_DK.UTF-8
en_US.UTF-8/Compose: en_DK.UTF-8

And everything is good :-)
No more "locale not supported by X server" etc, and typing of accented letters in both QT and TK apps alike, now functions.
./ximinfo
./ximinfo: locale=en_DK.UTF-8
./ximinfo: codeset=UTF-8
./ximinfo: XSetLocaleModifiers -> @im=local
./ximinfo: style XIMPreeditNone|XIMStatusNone
./ximinfo: style XIMPreeditNothing|XIMStatusNothing

I thank you very much for the pointers, and appreciate the great effort you have put into this jenglish, even if at the end, it has not so much to do with TK.

Logged In: YES
user_id=68433
Originator: YES

(reported on the chat: this seems to have been resolved, closing.)