#1365 Issue with DirectWrite and U+FFFD (replacement character)

Bug
open-accepted
Scintilla (793)
5
2013-04-10
2012-05-18
No

When DirectWrite is used and defined font is fixed width font, if a line contains unicode replacement character U+FFFD, DirectWrite will change the font of the whole line to some other proportional font, probably due to some glitch in fallback algorithm. However, this doesn't happen always, if the line consists only of the U+FFFD character, the font will not be changed. With that in mind, changing BreakFinder to add breaks on replacement characters should probably fix the problem.

Attached is a sample screenshot.

Discussion

  • Marko Njezic

    Marko Njezic - 2012-05-18

    More testing has revealed another interesting quirk. When fixed width font is used, DirectWrite will use different font substitutions depending on position of replacement character U+FFFD. I have encountered two different situations so far:

    1. If replacement character is the first character in a line, one type of substitution will be used
    2. If there are some other characters in line before replacement character, another type of substitution will be used

    Also, while changing BreakFinder to add breaks on replacement character will improve text rendering, it is only a partial solution. There are other places where BreakFinder is not used when drawing text, so the proper solution would be to change the underlying platform code (PlatWin.cxx), so that text is measured and drawn in segments if there is a replacement character somewhere in the string.

     
  • Neil Hodgson

    Neil Hodgson - 2012-05-19
    • assigned_to: nobody --> nyamatongwe
    • status: open --> open-accepted
     
  • Neil Hodgson

    Neil Hodgson - 2012-05-19

    Reproduced. I'd consider that changing the font of the other characters is a bug in DirectWrite. It is unlikely that the replacement character is the only one that causes font substitution. Perhaps there is an API to reveal if a font will be substituted and where in the text.

     
  • Marko Njezic

    Marko Njezic - 2012-05-19

    That's a definitive bug in DirectWrite, but I doubt that it will be fixed anytime soon by Microsoft.

    I have been testing various Unicode characters in an attempt to provoke more substitution quirks, but it appears that only replacement character is problematic. In every other case that I tried so far, DirectWrite used familiar square character when encountering a non existing character. I did not test behavior with invalid Unicode data, since Scintilla passes only valid data to underlying platform code, while drawing invalid bytes as blobs, so there certainly may be more cases that can cause substitution problems, which are not triggered by Scintilla.

    There is no direct API which will reveal if a font substitution occurred. However, there is a way how to determine this, but it is not pretty. You have to create custom text renderer by subclassing IDWriteTextRenderer and then analyze every glyph from created text layout in IDWriteTextRenderer::DrawGlyphRun().

     
  • Neil Hodgson

    Neil Hodgson - 2013-04-09

    I can no longer reproduce this on Windows 7 or 8. Tried with current Scintilla and with 3.1.0 and 3.2.0 so may have been fixed by Windows.

     
  • Marko Njezic

    Marko Njezic - 2013-04-09

    I've just tested the latest version of SciTE 3.3.0 and the problem is still present on my installation of Windows 7 (with all the latest updates installed).

     
  • Neil Hodgson

    Neil Hodgson - 2013-04-10

    The sample appeared to be Consolas 10 pt and couldn't reproduce with that or with the other common fixed width fonts Consolas, Courier New, Cousine, DejaVu Sans Mono, Lucida Console, or Ubuntu Mono.
    Monospaced DirectWrite sample

     
  • Marko Njezic

    Marko Njezic - 2013-04-10

    I've attached a new screenshot that shows the same problem using the latest version of SciTE. All default settings, only DirectWrite is turned on.

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks