#1703 UTF-8 in margin text

[Jacob Slusser]

I'm using Scintilla with the SC_CP_UTF8 code page to display Unicode (UTF-8). However, it doesn't appear that the margin text (SCI_MARGINSETTEXT/SCI_MARGINGETTEXT) uses the same code page for rendering. I assumed the behavior of margin text would be the same as annotation text (SCI_ANNOTATIONSETTEXT/SCI_ANNOTATIONGETTEXT) which renders the UTF-8 string correctly.

I've attached a screenshot showing the same Unicode text displayed in the document, annotation, and margin for comparison.

Is this a bug? Shouldn't the margin text be rendered using the same code page that the document text and annotations do?

Thanks,
Jacob

[Neil Hodgson]

Seems fine to me with current version. See attachment.

[Jacob Slusser]

That's so strange. I just can't seem to get it to work even with version 3.5.4. I must be doing something wrong.

[Neil Hodgson]

More likely the opposite: I write calling code in the pattern that I was expecting people to do when I wrote the implementation so it works.

You could try using a debugger, placing a breakpoint inside MarginView::PaintMargin just when its about to call DrawStyledText and see what the encoding of the surface is. If its not UTF-8 then track back to see where it is set incorrectly.

[Jacob Slusser]

I discovered this is related to the rendering technology used. When I use SC_TECHNOLOGY_DIRECTWRITE the text renders correctly, but when I use SC_TECHNOLOGY_DEFAULT (GDI) it does not.

Can you confirm?

I don't think all my users will have DirectWrite support and so I'd like to get the GDI renderer working if possible.

Thanks in advance.

[Neil Hodgson]

This doesn't occur for me with SciTE where the Unicode text appears with either technology=0 or 1. This could be a sequencing issue where SciTE makes calls in a different order to your application so gets a different result. Using a debugger to track the problem down in your application could help.

Here is some Lua code used in SciTE to check this:

:::lua
function showAnnotations()
    scite.SendEditor(SCI_MARGINTEXTCLEARALL)
    scite.SendEditor(SCI_SETMARGINTYPEN, 3, 4)
    scite.SendEditor(SCI_SETMARGINWIDTHN, 3, 80)
    scite.SendEditor(SCI_MARGINSETSTYLEOFFSET, 200, 0)
    scite.SendEditor(SCI_STYLESETBACK, 200, 0xFFFFE4)
    local line = 50
    local text = "märgîñ 文件"
    scite.SendEditor(SCI_MARGINSETTEXT, line, text)
end

[Jacob Slusser]

Neil,

Thanks for looking into this. I apologize for my last report because I gave you the wrong information. Upon closer inspection it is the SCI_SETBUFFEREDDRAW property that causes the change. Enable buffered drawing and the margin text renders incorrectly--regardless of technology. In my previous test I had disabled buffered drawing when I enabled DirectWrite--leading me to incorrectly think it was related to GDI/DirectWrite.

I've traced the code myself and can see where margins are drawn to a pixmap instead of directly on the surface when using buffered mode, but the exact cause of the problem still alludes me.

Please try the following to replicate:

function showAnnotations()

    scite.SendEditor(SCI_SETBUFFEREDDRAW, 1, 0)

    scite.SendEditor(SCI_MARGINTEXTCLEARALL)
    scite.SendEditor(SCI_SETMARGINTYPEN, 3, 4)
    scite.SendEditor(SCI_SETMARGINWIDTHN, 3, 80)
    scite.SendEditor(SCI_MARGINSETSTYLEOFFSET, 200, 0)
    scite.SendEditor(SCI_STYLESETBACK, 200, 0xFFFFE4)
    local line = 50
    local text = "märgîñ 文件"
    scite.SendEditor(SCI_MARGINSETTEXT, line, text)
end

[Neil Hodgson]

Fix committed as [4ae329].

[Jacob Slusser]

Thanks Neil.