#1310 Support IMR_DOCUMENTFEED to improve IME conversion accuracy

[Zufu Liu]

The document for IMR_DOCUMENTFEED is at https://docs.microsoft.com/en-us/windows/win32/intl/imr-documentfeed
This was originally requested at https://github.com/zufuliu/notepad2/issues/127, mostly used by Japanese IMEs. From these links on issue, it seems can be handled very similar to IMR_RECONVERTSTRING (without last loop).

[johnsonj]

Is there any ime supporting document feed feature?
Or working use cases?

[Zufu Liu]

I don't known yet. What I guessed is it's similar to retrieve-surrounding on GTK [feature-requests:#1066] ; i.e.: once we received the request, we return some text around caret or target text, and IME then can use these text to find better/valid candidate words, thus improved conversion accuracy.

As commented on https://github.com/zufuliu/notepad2/issues/127#issuecomment-521087537, it was at least used by Japanese IME, so called 前後フィードバック or 前後参照 in Japanese, but I don't have any steps to generate the request.

[Neil Hodgson]

To make a change here we really need to be able to test it.

[Zufu Liu]

Some tests/steps are available at https://github.com/zufuliu/notepad2/issues/127 , briefed as bellow:

sample text (ignore line numbers):

1 が痛い
2 が咲く
3 危機
4 ファイト

expected result:

1 鼻が痛い
2 花が咲く
3 危機一髪
4 ファイト一発

1. type hana before each word on line 1 and line 2
2. type ippatu after each word on line 3 and line 4
3. after typing hana or ippatu press space to select candidate item.

(tested on Win10 with MS Japanese IME) IMR_DOCUMENTFEED is sent (in WM_IME_REQUEST) after typing hana or ippa.
the behavior looks similar to GTK's “retrieve-surrounding” signal: https://developer.gnome.org/gtk3/stable/GtkIMContext.html#GtkIMContext-retrieve-surrounding

[Neil Hodgson]

I'm not seeing the desired behaviour (different characters with different context) with any application including Firefox 72 on Windows 10 using either the Microsoft or Google Japanese IMEs. For Firefox:

Microsoft IME:
花が痛い
花が咲く
危機一髪
ファイト一髪

Google IME:
花が痛い
花が咲く
危機一発
ファイト一発

In Sakura, also seeing "花" for "hana" in all cases and different "ippatu" based on IME, not context.

Tried clearing history for both Microsoft and Google. Perhaps there are other language options that need setting.

[johnsonj]

I can not tell the difference. same as Neil.
https://groups.google.com/g/scintilla-interest/c/A6Rhbg7JP7c

[Neil Hodgson]

Committed implementation from @johnsonj with [dcbeef].

Demonstration video:
https://www.youtube.com/watch?v=Kj3lTrCcS7M&authuser=0