#39 [PATCH] Compose key support

[Anonymous]

Originally created by: justanot...@googlemail.com

This adds support for a compose key via a long press of the dpad centre button (✓). The compose strings are adapted from X.org, with dead-key and non-Latin combinations removed (for testing reasons, as much as anything); but still there are ‘quite a few’ combinations.

It may make sense to put the compose sequences in a separate file; this is essentially a proof-of-concept patch.

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

Hm, I have mixed feelings about this change. While the compose functionality makes sense on a hardware keyboard, I'm not convinced it's a good fit for the soft keyboard. Were you missing specific characters, or is this mainly because you're very used to the compose mechanism and want it to work equivalently on Android?

For example, the current method to get an "à" is:
- long-press the "a" key
- select the "à" character from the popup mini keyboard.

If I'm understanding your change right, the new sequence would be:
- long-press the "✓" key
- type "`"
- type "a".

It's more complicated on keyboards that don't have "`" on the main layout, for example for the Czech layout:
- long-press the "✓" key
- long-press the ";" key, select "`" from the pop-up mini keyboard
- type "a".

I think it's much easier to select the characters visually from a list instead of memorizing the combinations needed to get them. The current layout of course doesn't contain all possible composable combinations, but the language-specific layouts should have the important ones.

[Anonymous]

Originally posted by: justanot...@googlemail.com

I'm used to Compose, but I got used to dead accent support on Acorn hw before that. I use both, though I tend to use Mode_shift a lot too (highly-customised keymap).

This patch doesn't remove the existing long-press; use whichever you prefer :-)

But it probably needs some work and some configurability. It's clone time, I think...

[Anonymous]

Originally posted by: justanot...@googlemail.com

http://code.google.com/r/justanotherdsalt-hackerskeyboard/source/list?name=compose-key

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

I'm not saying your patch is wrong, I'm just a bit skeptical if it's a net usability improvement. Also, I've had several hairy bugs due to unexpected side effects of stateful key handling, and that makes me wary of adding even more complications to those code paths...

I think that dead keys may be worth revisiting, it seems that the text handling is fairly smart at least for newer Android versions. Honeycomb does interesting things with Arabic text and vowel diacritic combining dead keys, though according to an Arabic speaker it's not entirely correct yet. If I'm understanding the Unicode standard right (which I wouldn't bet on), the text handling should take care of treating a composing accent together with a letter as being fully equivalent to the pre-composed character, but I'm unsure if that's really true in practice.

[Anonymous]

Originally posted by: justanot...@googlemail.com

Well... the combining is with the preceding glyph, but it's usual (IME) to type the accent first. I can see (some of) my compose code being useful here, either to swap things around (but that can equally well be special-cased differently) or to provide the equivalent single character.

'à' (U+0065 & U+0300) and 'à' (U+00E0) – definitely different, though only really if you're dealing with byte counts or file names…

[Anonymous]

Originally posted by: eborrem...@gmail.com

I think it would be nice to support key composition like normal keyboards, just because people are used to it.

No special "V" or dpad key press required, simply hit the dead key followed by the character.

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

I think using this code to provide dead key support would be useful. There's partial support in the input framework for composing Unicode accents but it doesn't work properly, so some help from the keyboard would be needed.

Status: Accepted

[Anonymous]

Originally posted by: justanot...@googlemail.com

X's own dead keys support is implemented via the compose support.

I've updated my compose-key branch to handle some dead accents. It currently only recognises U+0300 to U+036F, but does work with multiple accents; and the compose key works within a dead accent sequence too. The following all work for ‘ǖ’:

dead_macron + dead_diaresis + ‘u’
dead_macron + ‘ü’
dead_macron + compose + ‘"’ + ‘u’
compose + ‘_’ + ‘"’ + ‘u’

(Note that the third one, though more key presses, is identical to the second one: the compose sequence becomes ‘ü’ before the dead accent code gets to see it.)

I've not looked at the accent composition functions in the input framework. That would probably simplify things a bit...

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

I'll take a closer look at this as soon as I find time - I'm thinking it would be a nice option to provide the composing accents in the keymap, but still not convinced about the usefulness of a dedicated "compose" key - using the DPAD_CENTER (checkmark) key for that isn't really suitable since it's supposed to be available for other purposes, though admittedly it's not very frequently used.

About the input framework - the expected behavior would be that the Android OS should correctly normalize Unicode sequences so that a dead diaeresis + "u" would automatically be treated as an "ü". My experience is that this doesn't work correctly. Support is improving in recent Android versions but even Honeycomb doesn't handle accents quite right.

Short of patching the Android OS, a better alternative could be to use Java's java.text.Normalizer to do canonicalization within the keyboard, so that the Android input layer gets appropriately pre-composed characters. I haven't tried how well this works in practice. More here: http://developer.android.com/reference/java/text/Normalizer.html

However, as far as I know for some languages such as Arabic the canonical form uses separate non-composed characters, so this could get tricky to get right, though I think it would be valuable even if it only works for a subset of languages.

[Anonymous]

Originally posted by: justanot...@googlemail.com

I've committed some changes which make use of this code. It's not perfect – some combinations don't produce correct results (combining tilde is broken, at least on Froyo, where there's no suitable single code point).

For valid combinations which don't work via java.lang.Normalizer, we can add (or restore) entries in the compose table in DeadAccentSequence.java.

Incidentally, there are valid, working combinations which are properly replaced but which appear as boxes. This is a font coverage issue.

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

Thank you, this is looking promising. FYI, code reviews appear switched off in your clone, is that intentional? Nitpick: it should work to look for Character.getType(...) == Character.NON_SPACING_MARK instead of checking the numeric ranges.

The potential lack of font coverage is expected, and is also an issue with the hardcoded pre-composed characters to some extent.

[Anonymous]

Originally posted by: justanot...@googlemail.com

I've committed a change which uses NON_SPACING_MARK as described. However, I wonder if the other combining & enclosing marks should be handled too (though I note that GTK will handle them as combining for cursor movement purposes, given something suitable to combine them with).

(Code reviews are switched on but were restricted.)

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

I've merged this (finally!), with a few minor tweaks in [r9b6403bb5223], and the latest test version includes it - try v1.23rc4 (or later) from http://code.google.com/p/hackerskeyboard/downloads/list . Thanks again to Darren for the contribution.

I've also added some true dead keys to the Danish keyboard, to help test. (Greek and Arabic also have dead keys, though Arabic works inconsistently on pre-Honeycomb systems.)

Known issues: I'm showing the dead keys using a dotted circle placeholder char, and that doesn't consistently compose right. I'll probably need to disable the placeholder on pre-Gingerbred.

Status: Fixed

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

Forgot to add - I may end up making the compose key support conditional on an option if it causes issues for people, though I don't think that there's currently much of a reason to do long presses on DPAD_CENTER.

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

(Keeping the bug open for visibility, since the fix is not in the official release yet and several wiki pages still point to it.)

Status: FixInTest

[Anonymous]

Originally posted by: justanot...@googlemail.com

The placeholder character seems fine here (froyo), and the display issues aren't really a problem here. As for long presses on dpad centre: I don't know what might make use of short presses on it.

[Anonymous]

Originally posted by: bob.walr...@gmail.com

As far as the French typing habits are concerned (AZERTY layout), the typing order of dead key followed by the actual character is such a high-speed typing reflex that any other implementation would be hard to get used to!
The ê in particular is very frequent. (not to forget the other vowels, of course)

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

I've updated the AZERTY keymap in [r1266cdfe761f] to add dead keys. Please try v1.23rc6 (or later) from http://code.google.com/p/hackerskeyboard/downloads/list and let me know if that's working as expected.

Changes to the 5-row layout:

- dead circumflex and diaeresis on the key right of "p"
  (and removed the redundant tilde there)

- dead acute and tilde as é (top row) long-press alternates

- dead grave as è (top row) long-press alternate

I've also added the dead key modifiers to the 4-row layout as long-press alternates on the "'" key.

Accessing the dead grave and acute accents via long-press popups is clunky, but that's the closest I could get to the standard layout - apparently real AZERTY keyboards also require the use of AltGr or similar modifiers to access those? Please let me know if something is misplaced.

Once the customizable key support is ready (see issue 13), it should be possible to provide alternative layouts, for example replacing less-frequently-used precomposed accented character keys with directly accessible dead accents.

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

This is included in today's v1.24 Market release (same as v1.23rc9). Let me know in case of remaining issues.

Thanks again to Darren for contributing the patch! (And apologies for mangling it with my changes ;-) )

Status: Fixed

[Anonymous]

Originally posted by: bob.walr...@gmail.com

Sorry for the delay.
I haver now tested the AZERTY layout with dead key.
Excellent work, well done. For ^ and ¨

The long key way for the acute and grave accents is indeed clunky. Standard AZERTY keyboards do vary in the implementation of these diacritics. Basically they are not part of the french alphabet. So, they're just "nice to have".

Great work you're doing!

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

Bulk update - changing "Fixed" to "Verified" for old bugs.

(Background: I'm changing the "Fixed" status to be considered open, the next steps in the lifecycle will be the closed states "FixInTest" and "Verified". This lets me mark issues as "Fixed" in commit messages without hiding them from the issue tracker.)

[Anonymous]

Originally posted by: Klaus.We...@gmail.com

Bulk update - changing "Fixed" to "Verified" for old bugs.

Status: Verified