#2135 IME candidate window is at wrong position on GTK3

[Jiri Techet]

On macOS it is possible to long-press a key to get a popup offering to
select characters with additional accents. This popup should appear
at the position of the caret. Scintilla however only sets the
caret position using gtk_im_context_set_cursor_location() for some
input methods (which exclude im-quartz) so the caret position isn't
set correctly and macOS always displays the popup at the very bottom
of the screen.

Moving SetCandidateWindowPos() into PreeditChanged() seems to fix
the problem.

[Neil Hodgson]

IME code has been through a lot of tweaks to handle different language cases and IME choices with some changes involving the candidate window position. See, for example, [feature-requests:#1300] which is a Win32 issue but could come to GTK. East Asian input requires the IME and some users of these languages want particular behaviour.

For GTK, Linux is much more popular than macOS and a more tightly targeted change may be safer. Its most likely the situation on macOS is for the windowed variant of IME since most paths through PreeditChangedInlineThis will move the candidate window.

[Jiri Techet]

I tried with Chinese and the quartz IM basically just forwards the input to the macOS candidate window and doesn't move the caret in Scintilla until a selection is made so I think for quartz what I did should work.

Do you know if there is a way to detect which IM module is being used so I can restrict this to the quartz module?

[Neil Hodgson]

First try inline IME mode to see if it works OK. If it does then adding the movement to PreeditChangedWindowedThis avoids changing the inline code in a way that will be more dificult to refine.

There is a GtkIMContextInfo::context_name but I can't work out how to navigate to it. Could just make it a compile-time change with #if defined(GDK_WINDOWING_QUARTZ) or similar.

[Jiri Techet]

After more experimenting with the quartz im module, it looks it's quite buggy and I haven't found a way to reliably pop up the candidate window at the right position. I guess I'll just patch Scintilla every time I make a release of Geany on macOS and won't add code to Scintilla which isn't 100% reliable. So I think this bug report can be closed.

[Neil Hodgson]

macOS works differently from other platforms and its likely some of the changes to the input are not going through PreeditChanged. It may be worthwhile understanding how Cocoa (or Carbon (if GTK is using that old API) gets mapped to GTK. In particular, the firstRectForCharacterRange method which returns the screen position of text may be involved.

Another thing to check is how the standard GTK text widget behaves with this form of input.

[Zufu Liu]

The similar bug can be reproduced on Ubuntu 18.04 with fcitx:
Start SciTE with window mode IME, candidate window is at left bottom of screen, like Jiri's fist screenshot.
Change IME to inline mode, candidate window is under caret line.
Change back to window mode, candidate window is at wild position near caret.

Jiri's patch is incomplete, IME candidate window sometime overlapped with current line (see screenshots made by Ctrl+Alt+Shift+R).

@johnsonj can you check whether my patch break Korean IME?

[johnsonj]

it is not likely work for korean and japanese.
candidate window should follow caret.
I will report it tomorrow.

[johnsonj]

Sorry I can not test this on Mac, So I guess:
for accented input with windowed ime mode,
Caret position moving by arrow keys can be detected with "retrieve-surrounding" signal.

Does accented input work well in inline ime mode?

[Neil Hodgson]

I don't have GTK installed on my Mac as its not really solid.

The macOS native (Cocoa) version of Scintilla only supports inline IME mode.

In SciTE as well as platform-bundled apps like TextEdit, the Chinese, Japanese and Korean language modes do not support accented input with long press like European languages.

[johnsonj]

diff -r c924b214edf5 gtk/ScintillaGTK.cxx
--- a/gtk/ScintillaGTK.cxx  Fri Dec 13 16:29:24 2019 +1100
+++ b/gtk/ScintillaGTK.cxx  Fri Dec 13 16:05:05 2019 +0900
@@ -277,6 +277,9 @@
             G_CALLBACK(Commit), this);
    g_signal_connect(G_OBJECT(im_context), "preedit_changed",
             G_CALLBACK(PreeditChanged), this);

+   g_signal_connect(G_OBJECT(im_context), "retrieve-surrounding",
+       G_CALLBACK(RetrieveSurrounding), this);
+
    gtk_im_context_set_client_window(im_context, WindowFromWidget(widget));
    GtkWidget *widtxt = PWidget(wText); //  // No code inside the G_OBJECT macro
    g_signal_connect_after(G_OBJECT(widtxt), "style_set",
@@ -2483,6 +2486,17 @@
    }
 }

+gboolean ScintillaGTK::RetrieveSurroundingThis() {

+   // Gtk has called gtk_im_context_get_surrounding().
+   // No selection may pop up candidate box for replacement.
+   SetCandidateWindowPos();
+   return TRUE;
+}
+
+gboolean ScintillaGTK::RetrieveSurrounding(GtkIMContext *, ScintillaGTK *sciThis) {
+   return sciThis->RetrieveSurroundingThis();
+}
+
 void ScintillaGTK::StyleSetText(GtkWidget *widget, GtkStyle *, void *) {
    RealizeText(widget, nullptr);
 }

[johnsonj]

reconvert key triggers CandidateBox to show up
when quick phrase input on fcitx (or maybe accented input on mac)

Demo:
https://www.youtube.com/watch?v=2WfqPA9rHiI

[Neil Hodgson]

Committed fix with [b694d5].

[Zufu Liu]

Adding "retrieve-surrounding" fixed fcitx in windowed mode, no overlapping in both mode.
Compared to gedit, IME caret in SciTE is not shown in windowed mode.

There are tow deprecated warnings when building on Ubuntu 19.10 (GTK 3.24.12):

ScintillaGTKAccessible.cxx:1226:75: warning: ‘void g_type_class_add_private(gpointer, gsize)’ is deprecated
SciTEGTK.cxx:4383:66: warning: ‘void gtk_container_set_focus_chain(GtkContainer*, GList*)’ is deprecated

[Neil Hodgson]

Changing "retrieve-surrounding" without actually setting the surrounding text seems out-of-spec:
https://developer.gnome.org/gtk3/stable/GtkIMContext.html#GtkIMContext-retrieve-surrounding

[johnsonj]

It may be the time to introduce "retrieve-surrounding"?
Mailing list message.

[Zufu Liu]

Above retrieve-surrounding.patch not fix ibus-libpinyin (the default GNOME IME for Chinese, added in Settings -> Region & Language -> Input Sources) in windowed mode, but ime-gtk.patch (i.e. adding SetCandidateWindowPos for PreeditChangedWindowedThis) seems fixed it (except for caret, which is not shown), the updated patch is attached.

both changes not fix ibus-libpinyin in Fedora 31 in windowed mode, no Chinese characters can be typed. fcitx is not tested, seems there are no Language Support and Input Method utils on Fedora.

[Neil Hodgson]

The patch doesn't document what each portion is achieving. Examining with reference to Win32 changes from [feature-requests:#1300] and [feature-requests:#1304], there appear to be 3 changes:

1. Set candidate position for windowed IME
2. Set candidate position with box height to prevent overlapping
3. Move SetCandidateWindowPos() to before insertion of composition to prevent movement

This appears to be following the above Win32 feature requests except for (1) change to windowed IME. If (1) is a benefit by itself then it could be the solution of this bug report so should be a separate isolated patch that can be tested by the original poster. However, if this degrades behaviour on any tested operating system,, language, or IME then that should be noted and avoided if possible.

Its possible that each of the issues that occurred on Win32 for [feature-requests:#1300] and [feature-requests:#1304] will be applicable on GTK. They should have a new feature request that details which of the Win32 changes are beneficial and which do not work on GTK.

[johnsonj]

It seems to be relevant for this ticket.
I think it is better that other things should be discussed as separate issues like windows patch.

@@ -2447,6 +2450,8 @@
try {
PreEditString pes(im_context);
if (strlen(pes.str) > 0) {

• SetCandidateWindowPos();
  +
  PangoLayout *layout = gtk_widget_create_pango_layout(PWidget(wText), pes.str);
  pango_layout_set_attributes(layout, pes.attrs);

[Zufu Liu]

Attachment contains info on GTK IME setup in different systems.

Window mode IME on Fedora can't be tested, because of the error (don't know why this not happens on Ubuntu):
Gdk-Message: 22:51:24.266: Window 0x28661e0 is a temporary window without parent, application will not be able to position it on screen.

macOS and Windows are not tested (see [feature-requests:#1331]).

current result: SetCandidateWindowPos.diff works on Ubuntu.

[Neil Hodgson]

Fedora is more likely than Ubuntu to be using Wayland which differs strongly in window positioning.

[Zufu Liu]

Add a separated bug [bugs:#2149] for GTK Wayland.

[Zufu Liu]

Updated patch based on test results (in the zip) on Windows and macOS.
signal handler for "preedit_start" is added.

Candidate window moving on typing ([feature-requests:#1300]) is not observed in Scintilla, but in GtkEntry on Windows (see screenshot for Chinese IME in GtkEntry on Windows).

IME cleanup on end composition ([bugs:#2137]) is not observed, but there are behavior difference: assuming we currently typing on line 1, then move mouse to click on line 2 (no "preedit_end" signal), the continue typing, pre-edit string and candidate window will move to line 2; on Windows the pre-edit string will be added into document on clicking.

IME caret moving ([feature-requests:#1304]) is not observed, the caret is hard to see (especially for Japanese IME), it likely it's stay at the beginning of pre-edit string. after pre-edit string been committed, caret is at end of it.

I think the commented out block (imeBox.y -= 3*vs.lineHeight;) in SetCandidateWindowPos() is bug in GTK on Windows, see screenshot for GtkEntry in Geany and Meld Merge. Application build against specific GTK version, can test and enable that code.

[Zufu Liu]

there is a similar issue for GTK on Windows (opened 8 years ago, but still not committed/fixed) https://gitlab.gnome.org/GNOME/gtk/issues/374 , if that patch is applied, we will need to set imeBox.width (which currently has no effect).

from the code at https://gitlab.gnome.org/GNOME/gtk/blob/master/gtk/gtkimcontextime.c#L1036
they added some thing in y position cf.ptCurrentPos.y = wy + context_ime->cursor_location.y, and x position cf.ptCurrentPos.x = wx + context_ime->cursor_location.x;.

[Zufu Liu]

Updated patch to reflect the GTK bug on Windows.
imeBox.x is not adjusted, which is just a small offset.